Sr Apache Spark Developer

Description

Job Title: Senior Apache Spark Developer

Duration: 18 Month W2 Contract

Location: Jersey City, NJ

Required Pay Scale: $65-$73/hour

• Due to client requirements this role is only open to USC or GC candidates*

Senior Apache Spark Developer:

We are seeking a Senior Apache Spark Developer to help build and expand our next-generation data processing platform. This role will focus on designing and developing distributed data processing solutions using Apache Spark while leveraging an existing enterprise Kafka ecosystem.

The successful candidate will play a critical role in scaling our data processing capabilities, enabling cloud adoption, and supporting strategic migration initiatives involving Databricks and AWS. This position offers the opportunity to work on high-volume, mission-critical data platforms processing large-scale real-time and batch workloads.

Key Responsibilities

• Design, develop, and optimize Spark-based applications for large-scale data processing.
• Build streaming and batch data solutions leveraging Apache Kafka and Apache Spark.
• Develop reusable frameworks and components supporting enterprise data engineering initiatives.
• Support migration efforts from on-premises data platforms to Databricks and AWS.
• Collaborate with architects and platform engineers to implement cloud-native data solutions.
• Optimize Spark jobs for performance, scalability, reliability, and operational efficiency.
• Participate in system design, code reviews, testing, and production support activities.
• Implement CI/CD, monitoring, and observability practices for distributed data applications.
• Troubleshoot and resolve complex issues across Spark, Kafka, Databricks, and cloud environments and with other enterprise messaging systems.

Required Qualifications

• 5+ years of hands-on development experience with Apache Spark.

• Strong expertise in:

• Spark SQL

• Structured Streaming

• DataFrames/Datasets

• Performance tuning and optimization

• Experience integrating Spark with Apache Kafka.

• Strong development skills in:

• Java

• Python

• Experience designing and supporting large-scale distributed systems.

• Solid understanding of software engineering principles, object-oriented design, and testing practices.

• Experience working with relational and distributed data platforms.

• Bachelor's Degree

Preferred Qualifications

• Databricks experience (SQL).

• AWS experience, including services such as:

• S3

• EMR

• Glue

• Lambda

• ECS/EKS

• Experience supporting large-scale cloud migration initiatives.

• Experience with lakehouse architectures and modern data platform patterns.

• Experience handling multi-terabyte or petabyte-scale datasets.

• Financial services or regulated industry experience.

Ideal Candidate Profile (for both roles) will stand out if you have:

• Delivered large-scale cloud migration programs involving hundreds of terabytes or petabytes of data.
• Built or modernized enterprise data platforms using Spark, Kafka, Databricks, and AWS.
• Strong software engineering fundamentals and expertise in Java and/or Python.
• Experience designing highly scalable, resilient, and cost-efficient distributed systems.
• Proven ability to influence architecture decisions and work across multiple engineering teams.
• Ability to come up with iterative O(n) runtime O(1) space solutions to Easy DP problems

About Matlen Silver

Experience Matters. Let your experience be driven by our experience. For more than 40 years, Matlen Silver has delivered solutions for complex talent and technology needs to Fortune 500 companies and industry leaders. Led by hard work, honesty, and a trusted team of experts, we can say that Matlen Silver technology has created a solutions experience and legacy of success that is the difference in the way the world works.

Matlen Silver is an Equal Opportunity Employer and considers all applicants for all positions without regard to race, color, religion, gender, national origin, age, sexual orientation, veteran status, the presence of a non-job-related medical condition or disability, or any other legally protected status.

If you are a person with a disability needing assistance with the application or at any point in the hiring process, please contact us at email and/or phone at: [email protected] // 908-393-8600

At The Matlen Silver Group, Inc., W2 employees are eligible for the following benefits:

• Health, vision, and dental insurance (single and family coverage)
• 401(k) plan (employee contributions only)