Job Description
| Job Description: | Local to Cary NC only! • Design and build batch and streaming data pipelines using Azure, Databricks, and Kafka. • Develop ETL/ELT solutions using Spark, SQL and Python/Pyspark, or Scala • Implement and support Data Lake / Lakehouse architectures • Integrate data from legacy and modern platforms. Preferred postgres DB migration to Azure Cloud. • Expertise in handling Data migration using API's. • Apply data quality checks and support data governance basics (metadata, lineage, access controls) • Optimize Spark jobs for performance, reliability, and cost • Collaborate with cross-functional teams, Product owners, Admin teams, BSAs, Scrum & Project managers Required Skills & Experience: • 8+ years of experience in data engineering in azure using Azure Databricks. • Strong experience with Azure Data Lake, Databricks, ADF, Postgres. • Strong Data Engineering experience in Big Data (Cloudera) • Hands-on expertise in SQL, Python and PySpark or Scala • Proven experience with batch & real-time ingestion • Experience with Kafka and big data processing • Added good to have experience with: • Informatica PowerCenter • Informatica IDMC or equivalent cloud data integration tools • SQL Server, Oracle, Azure SQL • Cosmos DB • Unix/Linux • Maestro (or similar enterprise schedulers) • Data tools such as Precisely or equivalent Added Advantage • Insurance domain knowledge (Policy, Claims, Billing, Underwriting, Actuarial, Regulatory data) • Experience with Azure DevOps (AzDO) and GitHub-based CI/CD • Practical usage of Copilot for data engineering and development productivity • Exposure to Generative AI / GenAI use cases in data platforms and analytics Role Descriptions: Microsoft Azure (ADB/ADF) Essential Skills: Microsoft Azure (ADB/ADF) Desirable Skills: Keyword: Skills: Digital : Microsoft Azure Experience Required: 6-8 |
|---|
Numbers & Facts
| Location | Aurora, IL |
| Industry | Computer/IT Services |
| Company Size | 500 to 999 employees |
| Year Founded | 1990 |
| Website | http://q1tech.com/ |
About Company
Q1 consists of experienced and recognized experts providing the capability to respond to market demand in order to provide professional services for our clients including Enterprise software implementations, application integration and technical / functional support.
Q1 has steadily grown into a Quality IT services and solutions organization with the average experience of our team being over 10 years. We have continuously met or exceeded client expectations by delivering professional services and project implementations on time and under budget to help clients truly recognize return on investment.
Skills
- Access Controlunmatched
- Actuarial Skillsunmatched
- Application Programming Interface (API)unmatched
- Artificial Intelligence (AI)unmatched
- Big Dataunmatched
- Billingunmatched
- Cloud Computingunmatched
- Clouderaunmatched
- Continuous Deployment/Deliveryunmatched
- Continuous Integrationunmatched
- Cross-Functionalunmatched
- Data Analysisunmatched
- Data Lakeunmatched
- Data Managementunmatched
- Data Migrationunmatched
- Data Processingunmatched
- Data Qualityunmatched
- Database Extract Transform and Load (ETL)unmatched
- DevOpsunmatched
- Engineeringunmatched
- GitHubunmatched
- Informaticaunmatched
- Insuranceunmatched
- Linux Operating Systemunmatched
- Metadataunmatched
- Microsoft SQL Serverunmatched
- Microsoft Windows Azureunmatched
- Oracle Databaseunmatched
- Performance Tuning/Optimizationunmatched
- PostgreSQLunmatched
- Project/Program Managementunmatched
- Python Programming/Scripting Languageunmatched
- Regulationsunmatched
- SQL (Structured Query Language)unmatched
- Scala Programming Languageunmatched
- Scrum Project Management and Software Developmentunmatched
- Underwritingunmatched
- Unix Operating Systemsunmatched
- Use Casesunmatched
Level up your application
Professional resume templates
Browse dozens of recruiter approved resume templates, layouts and formats. Choose your favorite and make it your own in minutes.
Free resume templatesFree resume builder
Improve your existing resume or start from scratch and create a standout, ATS-friendly resume. Add job-specific content, download and apply.
Free resume builder