System Engineer

Indent : PSL307991-15-1

Role : System Engineer

Location : Charlotte, NC (Hybrid)

Bill Rate : $57/hr

Role Summary

- Senior backend engineer building the core integrations between Unity and the enterprise Spark engine and SparkFlow framework

- Designs distributed, high-throughput systems that power data processing and integration workflows at enterprise scale

- Operates with significant autonomy, setting technical direction within a regulated, large-scale engineering environment

Key Responsibilities

Platform & Integration Engineering

- Design, build, and operate integrations between Unity and the Spark engine and SparkFlow framework

- Develop robust APIs, SDKs, and connectors enabling reliable data movement and workflow orchestration

- Implement abstractions that simplify SparkFlow adoption for downstream platform consumers

Distributed Systems Design

- Architect scalable, fault-tolerant systems handling large data volumes and concurrent workloads

- Apply best practices for performance tuning, caching, partitioning, backpressure, and resilience

- Drive design reviews, RFCs, and architectural decisions for core platform components

Reliability & Operations

- Build systems with first-class observability (metrics, logs, traces) and SLOs

- Establish operational excellence: runbooks, alerting, capacity planning, and incident response

- Champion automated testing, and progressive delivery practices

Required Qualifications

- 6+ years of backend engineering experience, with deep expertise in distributed systems

- Expert-level Python; strong API design (REST, gRPC) and asynchronous programming

- Proven track record delivering large-scale, production-grade systems

- Strong fundamentals in concurrency, data structures, and systems performance experience

Preferred Qualifications

- Hands-on experience with Apache Spark (PySpark, Spark SQL, Structured Streaming)

- Experience building or integrating with workflow/data orchestration frameworks (Airflow, Autosys)

- Exposure to greenfield platform builds in regulated financial enterprises

- Experience with event-driven architectures (Kafka) and lakehouse formats (Iceberg, Delta, Hudi)

Technical Skills

- Languages: Python (expert), Scala or Java (plus), SQL

- Distributed Compute: Apache Spark

- Storage & Data: S3/ADLS/GCS, Iceberg, Delta Lake, Parquet, Postgres

- Streaming & Messaging: Kafka, Pub/Sub

- APIs & Services: REST, gRPC, GraphQL, OpenAPI

- Containers & Cloud: Docker, Kubernetes, Azure/GCP, Terraform

- CI/CD: GitHub Actions

Soft Skills & Leadership Expectations

- Strong technical judgment and ability to operate