Lead Data Engineer

INSPYR Solutions

Doraville, GA(remote)

Apply

JOB DETAILS

SALARY

SKILLS

Access Control, Apache Kafka, Apache Spark, Artificial Intelligence (AI), Business Intelligence, Cataloguing, Cloud Computing, Continuous Deployment/Delivery, Continuous Integration, Data Analysis, Data Management, Data Quality, Data Sets, Enterprise Architecture, Finance, Hubs, Identify Issues, Information/Data Security (InfoSec), Maintain Compliance, Performance Engineering, Performance Tuning/Optimization, Professional Services, Root Cause Analysis, Technical Delivery, Training Data Sets, United States Citizen

LOCATION

Doraville, GA(remote)

POSTED

3 days ago

Title: Lead Data Engineer
Location: GA, Remote
Duration: temp to perm
Compensation: 60hr. to 70hr.
Work Requirements: US Citizen, GC Holders or Authorized to Work in the U.S.

Skillset / Experience:
Architect, build, and optimize distributed data pipelines using Apache Spark in a highvolume, missioncritical environment.
Design and maintain enterprise Lakehouse architecture with Delta Lake, ensuring ACID compliance, lineage, auditability, and data governance.
Develop automated ingestion frameworks (batch, streaming, and eventdriven) across multiple cloud services and integration points.
Enable machinelearning workflows by preparing featureready datasets and establishing reproducible ML deployment patterns.
Lead platformwide data quality, access control, and cataloging frameworks.
Implement advanced costoptimization, cluster tuning, and performance engineering strategies.
Collaborate with Finance, BI, Operations, and ML teams to translate complex business needs into scalable data solutions.
Own production reliability, troubleshooting, and rootcause analysis for data and ML pipelines.

7+ years of experience in advanced data engineering with distributed compute technologies.
Expertlevel Spark engineering (performance tuning, cluster configuration, partition strategies, optimization of large datasets).
Handson experience with Lakehouse architectures including ACID transactions, schema evolution, and governance frameworks.
Deep proficiency in Python and SQL for largescale data transformation.
Experience supporting machinelearning pipelines or model operationalization.
Proven experience architecting cloudnative data platforms (Azure, AWS, or GCP).
Strong background integrating diverse, complex data sources at enterprise scale.
Demonstrated ability to own missioncritical production systems.

Core Tools
Databricks (Spark, Delta Lake, MLflow, Notebooks)
Python & SQL
Apache Spark (via Databricks)
Delta Lake (for lakehouse architecture)
Cloud Platforms
Azure, AWS, or GCP
Cloud Storage (ADLS, S3, GCS)
Data Integration
Kafka or Event Hubs (streaming)
Auto Loader (Databricks file ingestion)
REST APIs
AI/ML
MLflow (model tracking/deployment)
Hugging Face Transformers
LangChain / LlamaIndex (LLM integration)
LLMs: Anthropic Claude, Meta LLaMA, Google Gemini
DevOps
Git (GitHub, GitLab, Azure Repos)
Databricks Repos
CI/CD: GitHub Actions, Azure DevOps
Security & Governance
Unity Catalog
RBAC

Experience with distributed streaming frameworks (Kafka, Event Hubs, or similar).
Experience building or supporting ML platforms, feature stores, or experimenttracking systems.
Background in data security, compliance controls, or auditready governance.
Experience automating data operations with CI/CD and infrastructureascode.

Our benefits package includes: (EXCLUDE on perm placements)

Comprehensive medical benefits
Competitive pay
401(k) retirement plan
…and much more!

About INSPYR Solutions

Technology is our focus and quality is our commitment. As a national expert in delivering flexible technology and talent solutions, we strategically align industry and technical expertise with our clients’ business objectives and cultural needs. Our solutions are tailored to each client and include a wide variety of professional services, project, and talent solutions. By always striving for excellence and focusing on the human aspect of our business, we work seamlessly with our talent and clients to match the right solutions to the right opportunities. Learn more about us at inspyrsolutions.com.

INSPYR Solutions provides Equal Employment Opportunities (EEO) to all employees and applicants for employment without regard to race, color, religion, sex, national origin, age, disability, or genetics. In addition to federal law requirements, INSPYR Solutions complies with applicable state and local laws governing nondiscrimination in employment in every location in which the company has facilities.

#IND-Telecom

Information collected and processed through your application with INSPYR Solutions (including any job applications you choose to submit) is subject to INSPYR Solutions’ Privacy Policy and INSPYR Solutions’ AI and Automated Employment Decision Tool Policy: https://www.inspyrsolutions.com/policies/. By submitting an application, you are consenting to being contacted by INSPYR Solutions through phone, email, or text.

About the Company

INSPYR Solutions

Resume Resources

Free Resume Templates Free Resume Builder