Job Description
Job Title: Senior Data Engineer / Lead Data Engineer
Location: Hybrid , NJ
Employment Type: Full-Time
NEED LOCAL CANDIDATE, NY OR NJ
EXP 12+
Key Responsibilities
- Design, develop, and maintain scalable ETL/ELT data pipelines for large-volume structured and unstructured data.
- Develop data processing solutions using Python, PySpark, Apache Spark, and SQL.
- Build and optimize data pipelines using Databricks, Azure Data Factory, AWS Glue, and Snowflake.
- Work with both Azure and AWS cloud platforms to implement modern data engineering solutions.
- Work with Delta Lake, Azure Data Lake, Amazon S3, Azure Blob Storage, Azure Synapse, and Snowflake.
- Develop pipeline orchestration and scheduling using Airflow, Control-M, Databricks Workflows, Azure Data Factory, and AWS services.
- Implement monitoring, logging, alerting, and troubleshooting processes for production data pipelines.
- Work with CI/CD processes using GitHub, Bitbucket, Jenkins, Docker, and Azure DevOps.
Required Skills
- 8+ years of experience in Data Engineering or related roles.
- Strong hands-on experience with Python, PySpark, Apache Spark, and SQL.
- Strong experience with Azure and/or AWS cloud environments.
- Experience with Databricks and Delta Lake.
- Strong knowledge of Azure Data Factory, Azure Synapse, Azure Data Lake/Blob Storage.
- Experience with AWS services such as S3, Glue, Athena, DMS, Lambda, SNS, SQS, and EventBridge.
- Strong experience with Snowflake and cloud data warehousing.
- Strong understanding of ETL/ELT, data warehousing, data modeling, and data integration.
- Experience working with Oracle, SQL Server, MySQL, Hive, and other relational databases.
- Strong programming and scripting experience using Python, SQL, and Shell scripting.
- Experience with Git, Bitbucket/GitHub, Jenkins, Docker, and Azure DevOps.
- .
Preferred Skills
- Experience with Kafka and real-time streaming.
- Experience with Airflow and Control-M.
- Knowledge of MLOps and machine-learning data pipelines.
- Experience with Kubernetes and containerized data workloads.
- Experience with API data integration and modernization.
- Knowledge of LLM/AI-assisted data engineering solutions.
- Experience with Power BI or Tableau for data reporting and analytics.
- Experience with data migration and reverse engineering of legacy data models.
Numbers & Facts
| Location | New York, NY |
| Job Type | Full-time, Employee |
| Salary | $0–$110,000 Per Year |
| Headquarters | New York, NY, US |
Skills
AWS Lambdaunmatched
Amazon Simple Storage Service (S3)unmatched
Amazon Web Services (AWS)unmatched
Apache Sparkunmatched
Application Programming Interface (API)unmatched
Artificial Intelligence (AI)unmatched
Cloud Computingunmatched
Computer Programmingunmatched
Continuous Deployment/Deliveryunmatched
Continuous Integrationunmatched
Data Analysisunmatched
Data Lakeunmatched
Data Managementunmatched
Data Migrationunmatched
Data Modelingunmatched
Data Processingunmatched
Data Warehousingunmatched
Database Extract Transform and Load (ETL)unmatched
DevOpsunmatched
Dockerunmatched
Gitunmatched
GitHubunmatched
Identify Issuesunmatched
Jenkinsunmatched
Machine Learningunmatched
Microsoft SQL Serverunmatched
Microsoft Windows Azureunmatched
MySQLunmatched
Oracle Databaseunmatched
Power BIunmatched
Python Programming/Scripting Languageunmatched
Relational Databases (RDBMS)unmatched
Reverse Engineeringunmatched
SQL (Structured Query Language)unmatched
Sales Pipelineunmatched
Scalable System Developmentunmatched
Scripting (Scripting Languages)unmatched
Simple Queue Service (SQS)unmatched
Snowflake Schemaunmatched
Software Engineeringunmatched
Structured Dataunmatched
Tableauunmatched
Unix Shell Programmingunmatched
Unstructured Dataunmatched
Level up your application
Browse dozens of recruiter approved resume templates, layouts and formats. Choose your favorite and make it your own in minutes.
Free resume templatesImprove your existing resume or start from scratch and create a standout, ATS-friendly resume. Add job-specific content, download and apply.
Free resume builder