Data Engineer

The ideal candidate must have 8+ years of hands-on data engineering experience building large-scale ETL pipelines using Apache Spark, Hadoop, Python, and SQL. They also need a strong background in the payments or financial sector, paired with excellent communication skills to effectively influence and partner with cross-functional teams.
***Only qualified Big Data Engineer candidates located near Arlington, VA to be considered due to the position requiring an onsite presence. ***

Required Education
" Bachelor's degree in a quantitative discipline such as Engineering, Mathematics, Finance, Business, or a related field. Equivalent practical experience may also be considered.

Required Qualifications/Skills/Experience:
" Experience as a Data Engineer or in a similar role, with a strong understanding of data engineering concepts and methodologies.
" Strong knowledge of writing and optimizing SQL queries to retrieve, manipulate, and analyze data efficiently.
" Hands-on experience with big data technologies such as:
" Apache Spark (PySpark, Spark SQL, Spark Streaming)
" Hadoop ecosystem (HDFS/ Ozone, Hive, YARN)
" Understanding data modeling concepts and database design to support scalable data solutions.
" Familiarity with Python.
" Ability to analyze and troubleshoot data issues and provide solutions with minimal supervision.
" Basic knowledge of testing and validating data to ensure accuracy and consistency in data pipelines.
" Excellent verbal and written communication skills, with the ability to articulate complex ideas clearly and concisely to both technical and non-technical stakeholders.

Role:
This role focuses on designing, implementing, and maintaining scalable enterprise ETL processes and robust data pipelines for a global client base. You will leverage big data frameworks like Apache Spark and Hadoop, along with SQL and Python, to optimize data processing and ensure high data quality. Working closely with cross-functional teams, you will automate routine tasks and deliver accurate, high-value data solutions across various industries.

Responsibilities:
" Support the design, implementation, and maintenance of enterprise ETL processes for data platforms, for a global client base.
" Develop scalable and efficient code to process data, ensuring availability and accessibility in a timely manner.
" Leverage big data processing frameworks such as Apache Spark and Hadoop to build and optimize data pipelines.
" Collaborate with senior engineers to address data challenges, contributing to solutions that maintain high data quality.
" Assist in the data delivery process, working alongside Data Engineers and Analysts to support accurate, high-value data solutions across various clients and industries.
" Build strong working relationships with team members and clients, contributing to both local and global projects.
" Learn and apply industry best practices, including version control, code reviews, and data validation, to ensure quality in data processes.
" Use SQL and other database technologies to help optimize data processing and reduce the time required to handle large data sets.
" Design, implement, and maintain data pipelines using ETL frameworks, orchestration tools, and distributed data processing engines.
" Participate in efforts to automate routine data tasks and streamline processes.
" Comply with all Client internal policies and adhere to external regulations.