Title: Big Data Engineer
Location : Rockville, MD or McLean. VA
Target Start Date : ASAP
Type: contract
Pay Rate : DOE
We are seeking an experienced Big Data Engineer to design, build, and optimize large-scale data processing systems. This role partners closely with engineering, data, and business teams to deliver scalable, high-performance data solutions across modern cloud environments.
Key Responsibilities
- Design, develop, and maintain scalable data pipelines using technologies such as Spark, Hadoop, Python, and Scala
- Architect and deploy big data workloads using Amazon EMR on EKS (Kubernetes)
- Build and manage Kubernetes-based infrastructure for distributed data processing
- Implement data ingestion, transformation, and storage solutions for large-scale datasets
- Optimize pipelines for performance, scalability, and reliability
- Monitor, troubleshoot, and resolve production data issues
- Develop and execute automated testing frameworks (unit, integration, E2E)
- Collaborate with data scientists, analysts, and cross-functional teams to support data-driven initiatives
- Continuously evaluate and adopt emerging technologies and best practices
Core Technical Skills
Big Data & Distributed Systems
- Strong experience with Spark, Hadoop, Hive, Trino
- Expertise handling large-scale (petabyte-level) datasets
- Deep understanding of performance challenges (e.g., data skew, resource constraints)
Kubernetes & Containerization
- Hands-on experience with Kubernetes (EKS) including pods, services, deployments
- Experience running Spark on Kubernetes (EMR on EKS)
- Knowledge of Helm, YAML manifests, autoscaling, and cluster troubleshooting
Cloud (AWS)
- Experience with S3, EMR, Glue, Lambda, Athena
- Strong understanding of EKS architecture, IAM roles (IRSA), networking, and monitoring (CloudWatch/CloudTrail)
- Familiarity with serverless architectures (Lambda, Fargate)
Programming
- Proficiency in Python or Scala
- Strong understanding of data structures, concurrency, and performance optimization
SQL
- Advanced SQL skills including window functions, joins, aggregations, and query optimization
AI & Modern Development Practices
- Experience with AI-assisted development tools (e.g., ChatGPT, GitHub Copilot)
- Prompt engineering and AI-driven workflow optimization
- Ability to incorporate AI insights into engineering practices and team productivity
Preferred / Nice-to-Have
- CI/CD tools (Jenkins, GitLab CI, GitHub Actions, ArgoCD)
- Infrastructure as Code (Terraform, CloudFormation)
- Docker and container optimization
- Observability tools (Prometheus, Grafana, ELK)
- Service mesh technologies (Istio, Linkerd)
- AWS or Kubernetes certifications
Qualifications
- Bachelor's degree in Computer Science or related field (Master's preferred)
- 5+ years of experience in big data engineering or related roles
- Strong background in distributed systems, cloud platforms, and data architecture
- Excellent communication and collaboration skills
- Ability to operate effectively in a fast-paced environment
What Success Looks Like
- Reliable, scalable data pipelines powering business insights
- Strong partnership across engineering and business teams
- Continuous improvement of data platforms and development workflows
- Effective use of automation and AI to enhance productivity and outcomes
Welcome to ConsultNet, a premier national provider of technology talent and solutions. Our expertise spans across project services, contract-to-hire, direct search, and managed services onshore, nearshore, and hybrid. For over 25 years, we have connected thousands of consultants with meaningful roles through a personal, communication-driven approach, partnering with a diverse client base to build high-performing teams and create lasting impact. Our comprehensive service offerings cover a wide range of technology and engineering positions across key markets nationwide. Learn more at www.consultnet.com .
We champion equality and inclusivity, proudly supporting an Equal Opportunity Employer policy. We welcome applicants regardless of Race, Color, Religion, Sex, Sexual Orientation, Gender Identity, National Origin, Age, Genetic Information, Disability, Protected Veteran Status, or any other status protected by law.