Title: Sr. Engineer 2 (Python)
Location: Fort Worth, TX - 76155 (3 days hybrid onsite)
Duration: 12+ months Contract
Top Required Skills and Qualifications:
Candidate must have 7 - 10 years of experience for this role.
Python
Databricks
Azure Cloud
Spark
Azure ML and good Application development skills.
Key Responsibilities:
Design & Architecture: Participate contribute in end-to-end design and implementation of scalable data and application design using Python and Azure ML.
ETL/ELT Development: Develop and implement complex ETL/ELT processes to process and transform large-scale datasets from diverse sources using Spark and Python,
Performance Optimization: Optimize Python jobs for performance and cost-efficiency using advanced techniques such as partitioning, caching, cluster configuration tuning, and troubleshooting bottlenecks.
Collaboration & Leadership: Collaborate with data scientists, analysts, and business stakeholders to translate complex business problems into technical solutions.
Data Governance & Security: Implement data security strategies, compliance standards, and governance frameworks like Unity Catalog, including managing access controls, encryption, and data lineage.
DevOps & Automation: Champion code quality and implement DevOps (CI/CD) pipelines using tools like Azure DevOps, GitHub Actions, or Jenkins to automate deployment and management of data assets and infrastructure.
Troubleshooting & Monitoring: Develop and implement robust monitoring, logging, and alerting systems for production-grade pipelines to proactively identify and resolve issues.
Nice to have skillsets:
Java, Spring, Good knowledge of different services in Azure. Excellent communication skills
Describe a great candidate that you are looking for and what skills and experience they will have:
• Should have 7+ years of experience in Python, ETL data handling, SQL.
• Should have experience in using AI tools for better productivity.
• Advanced proficiency in Python, Scala, and/or SQL for data processing and application development.
• 5 years of experience in Spark and Databricks programming
• Should have very good experience in software development life cycle
• Must be able to design and develop CICD pipelines that can push the code to different development and prod environments preferably to Azure using git actions
• Must have 5 years of experience in Logging, Monitoring and Alerting systems.
What is the team environment and structure like?:
Team is divided into pods with each of the pod handling different projects. The resource should be able make all the pods follow Agile methodologies and track deliverables.Candidate will be helping with All pods project planning and deliverables.