Data Engineer, Databricks

KeyLogic is seeking Databricks Engineers to join our growing team supporting the Department of Energy (DOE).

The position is Remote and requires U.S. Citizenship

Position Requirements:

• U.S. Citizenship
• 2+ years of engineering experience on Databricks and large-scale big data projects (Delta Lake, DLT, Apache Spark, unity Catalog).
• Bachelor's Degree
• 4+ years of experience in Python, Java, or equivalent. Comfortable with evaluation tooling, logging, monitoring,  and observability
• Experience with government contracting

Position Description:

As a Data Engineer in this role, you will be responsible for designing, building, and maintaining robust data pipelines that support the organization’s analytical and operational needs. You will work closely with cross-functional teams to ensure data is collected, processed, and made available in a reliable and scalable manner. Leveraging your expertise, you will optimize data workflows and implement best practices for data integration, transformation, and storage. Your contributions will directly impact the efficiency and effectiveness of data-driven decision-making across the business.
To excel in this position, you will utilize tools such as Databricks for advanced data processing and analytics, and Terraform for infrastructure automation and management. Familiarity with cloud-based environments, version control systems, and CI pipelines will further enhance your ability to deliver high-quality solutions. Experience with large-scale data architectures, real-time data streaming, and performance tuning will help you address complex challenges and drive continuous improvement. Your background in collaborating with diverse technical teams and managing end-to-end data solutions will be instrumental in achieving success in this dynamic environment.

Position Responsibilities:

• Design, build, and maintain scalable data pipelines using Databricks
• Develop and implement infrastructure as code solutions with Terraform
• Collaborate with cross-functional teams to understand data requirements and deliver robust solutions
• Optimize data workflows for performance, reliability, and scalability
• Monitor, troubleshoot, and resolve issues related to data processing and infrastructure
• Ensure data quality and integrity throughout the data lifecycle
• Automate repetitive tasks to improve efficiency and reduce manual intervention
   

See Job Description