Data Engineer

Pay Rate: $77.00 - $82.00 Negotiable based upon years of experience

Global Financial Firm located in IRVING, TX has an immediate contract opportunity for an experienced Data Engineer

Responsibilities:

• Partner with project managers, business stakeholders, and senior managers to translate complex business requirements into technical solutions.
• Drive cross-functional collaboration to ensure seamless integration of functions.
• Identify and implement necessary system enhancements for new products and process improvements.
• Lead the resolution of high-impact problems through the evaluation of business processes and system architectures.
• Serve as a subject matter expert in application programming and ensure adherence to architectural blueprints.
• Provide mentorship and technical guidance to mid-level developers and analysts.
• Design, develop, and maintain data pipelines using PySpark.
• Collaborate with data scientists and analysts to understand data requirements.
• Optimize Spark jobs for performance and efficiency.
• Implement data quality checks and ensure data integrity.

Requirements:

• 6-10 years of experience in systems analysis and programming of software applications.
• Strong proficiency in Java application technologies, including TDD, Spring framework, and Microservices architecture.
• Extensive hands-on experience with PySpark and Python programming skills.
• Experience with Big Data ecosystems, including Cloudera and/or Data Bricks.
• Proficiency in SQL and experience with relational and non-relational databases.
• Experience with distributed query engines like Starburst (Trino/Presto).
• Proficient in designing and managing workflows using scheduling tools like Apache Airflow.
• Experience in RESTful API development and integration.
• Experience with job scheduling and Linux environments.

Preferred Skills:

• Experience with AI development tools.
• Interest in the financial services industry.
• Experience in real-time data streaming and processing using PySpark Structured Streaming.
• Knowledge of machine learning concepts and MLOps practices.
• Familiarity with data visualization tools like Tableau or Power BI.