Senior Data Software Engineer/ Databricks, Apache Spark, PySpark

Back to Search

Senior Data Software Engineer/ Databricks, Apache Spark, PySpark

Remote in Georgia, & 4 others

Data Software Engineering

apply

FacebookLinkedInSend via email

Looking for something else?

Find a vacancy that works for you. Send us your CV to receive a personalized offer.

Find me a job

Location-specific conditions & benefits*

Choose an option

We are seeking a skilled Senior Data Software Engineer with strong expertise in PySpark, SQL, and unit testing to join our data engineering team. The ideal candidate will have hands-on experience with Apache Spark, preferably within the Databricks environment, and will be responsible for building scalable data pipelines, optimizing data workflows, and ensuring code quality through rigorous testing practices.

Responsibilities

• Design, develop, and maintain scalable data pipelines using Apache Spark (Databricks preferred)

• Write efficient and optimized PySpark code for data transformation and processing

• Develop and execute complex SQL queries for data extraction, validation, and reporting

• Implement unit tests using pytest to ensure code reliability and maintainability

• Collaborate with data scientists, analysts, and other engineers to deliver high-quality data solutions

• Monitor and troubleshoot data workflows and performance issues

• Document technical designs, processes, and best practices

Requirements

• 3+ years of experience in Data Software Engineering

• Proven experience with Apache Spark, ideally in a Databricks environment

• Proficiency in PySpark and SQL

• Background in unit testing frameworks, especially pytest

• Understanding of data engineering principles and ETL processes

• Familiarity with version control systems (e.g., Git)

• Ability to work independently and in a collaborative team setting

• Excellent problem-solving and communication skills

• Proficiency in English at an Upper-Intermediate level (B2) or higher

Nice to have

• Experience with cloud platforms (e.g., Azure, AWS, GCP)

• Knowledge of CI/CD pipelines and DevOps practices

• Familiarity with Delta Lake, MLflow, or other Databricks-native tools