Senior Data Engineer (Databricks) NO 2C

inSync Staffing

Atlanta, GA

Apply

JOB DETAILS

SKILLS

Agile Programming Methodologies, Amazon Simple Storage Service (S3), Amazon Web Services (AWS), Analysis Skills, Apache Cassandra, Apache Kafka, Backlog Prioritization, Best Practices, Business Intelligence, C Programming Language, Cisco Unity, Computer Science, Data Modeling, Data Science, Data Sets, Database Extract Transform and Load (ETL), Dental Insurance, Ecosystems, Financial Analysis, IBM Product Family, Identify Issues, Information Technology & Information Systems, Kanban, Machine Tool, Messaging Middleware, NoSQL, Operational Audit, Python Programming/Scripting Language, SQL (Structured Query Language), Scrum Project Management and Software Development, Snowflake Schema, Systems Analysis, TIBCO Enterprise Message Service, Testing

LOCATION

Atlanta, GA

POSTED

30+ days ago

Senior Data Engineer (Databricks)

Client Corporation is seeking a Senior Data Engineer who enjoys collaborating
across the organization to deliver reliable, scalable data solutions. Join a skilled team
focused on modern data platforms, where you will help shape and operate our Databricks-
based on Lakehouse and streaming analytics capabilities.

This role blends strong business and analytical judgment with hands-on engineering:
you will partner with stakeholders to understand problems, translate them into clear
requirements, design pragmatic architectures, and deliver production-grade ingestion
and transformation pipelines - with ownership through deployment and steady-state
operations.

You will work with teams across the company to clarify data and BI needs, perform
structured requirements discovery alongside Data Modelers, align with BI
development on scope and estimates, and help guide delivery through completion while
promoting engineering quality and consistency.

Responsibilities

Define data requirements; discover, integrate, and wrangle large volumes
of structured and semi/unstructured data; validate outcomes using appropriate
tooling in our data environment.
Support standardized datasets and ad hoc analysis needs; build mechanisms
to ingest, validate, normalize, cleanse, and enrich data for downstream
consumption.
Contribute to data policies and technical controls where needed (access patterns,
retention considerations, and synthesizing/anonymizing sensitive attributes in line
with enterprise standards).
Implement rigorous data quality approaches for new and evolving sources; iterate
with analytics partners to improve sourcing, preparation, and trust in datasets used
for insights and modeling.
Champion data engineering best practices (testing, observability, operational
readiness) and contribute practical guidance on analytics preparation and
visualization-friendly dataset design where applicable.
Partner closely with data science and business intelligence teams to deliver data
models and pipelines that support reporting, research, and machine learning
workflows.
Build pipelines that clean, transform, aggregate, and publish data from
disparate systems into curated layers suitable for BI and advanced analytics.
Use Databricks, SQL, Spark, scripting, and AWS services to integrate systems
reliably and efficiently.
Apply solid knowledge of data architecture principles and help lead initiatives from
requirements through implementation—balancing correctness, performance, cost,
and maintainability.

Qualifications Required

5+ years of professional experience in data engineering (or equivalent depth),
including manipulating, processing, and extracting value from large datasets in
production settings.

Databricks (must have): hands-on delivery on the Databricks platform, including
developing and operating Databricks Jobs/Workflows, notebooks, and production
pipelines using Apache Spark (Spark SQL and/or PySpark) on Databricks.
Delta Lake on Databricks (must have): building and maintaining Delta
Lake tables (for example: incremental processing, merges/upserts, schema
evolution, and performance patterns aligned with Delta best practices).
Delta Live Tables / DLT (must have): designing, implementing, and
operating DLT pipelines (Python or SQL as used in your standards), including
expectations around pipeline dependencies, incremental processing, and operational
monitoring as supported by DLT.
Databricks governance (must have): practical experience with Unity Catalog (or
equivalent Databricks governance patterns strongly aligned to UC), including secured
sharing concepts and catalog/table governance as implemented in your environment.
4+ years hands-on Apache Spark engineering for analytics workloads (batch
and/or streaming), with ability to implement reliable transformations and
troubleshoot performance issues.
3+ years working with Apache Kafka (or managed equivalents such as Confluent
Kafka) for high-volume event/stream processing, including operational
considerations (throughput, lag, scaling, replay scenarios).
3+ years building and operating AWS capabilities supporting analytics platforms
(for example: S3, IAM-aligned access patterns, integration with compute/storage
patterns common to lakehouse architectures; familiarity with services such
as Glue/Lambda/MSK as applicable).
Strong SQL: ability to write and optimize intermediate-to-advanced queries and
translate business logic into reliable datasets.
Demonstrated experience delivering ETL/ELT pipelines in a Databricks
lakehouse environment; comfort with incremental loads, schema evolution,
and data quality checks.
Experience working in Agile delivery (Scrum/Kanban/SAFe or similar): backlog
refinement, iterative delivery, dependency coordination.

Preferred

Experience with Snowflake , or other large-scale analytical databases (helpful for
hybrid/platform comparisons).
NoSQL exposure (Cassandra or similar) where operational systems feed analytics
platforms.
Experience with enterprise messaging/integration ecosystems (TIBCO EMS, IBM
MQ, etc.) in addition to Kafka—helpful when bridging operational feeds into analytics
landing zones.

Education
Bachelor’s degree preferred in Information Systems, Computer Science, Computer
Information Systems, or a related field (or equivalent practical experience).

Benefits (employee contribution):

Health insurance
Health savings account
Dental insurance
Vision insurance
Flexible spending accounts
Life insurance
Retirement plan

All qualified applicants will receive consideration for employment without regard to age, race, color, religion, sex, sexual orientation, gender identity, national origin, disability, or status as a protected veteran. Rate of pay within the stated range will depend on the qualification of the applicant.

About the Company

inSync Staffing

We recognize the VMS program management team is our customer and needs to be serviced with integrity, so we built and continue to improve upon our delivery methods as we strive to provide the highest quality service possible. inSync Staffing’s management team recognized ten years ago the inevitable changes to the staffing industry being brought about by technology and the growing trend of Fortune 1000 corporations to outsource management of their contingent workforces to meet compliance and cost control goals. Rather than swim upstream against the changes, inSync Staffing has embraced MSP and VMS programs as our customers, not competitors. We asked program managers how they want to be serviced. The result of their input is that we have structured inSync Staffing as a recruiting and customer service organization, unlike traditional staffing companies who sell directly to the end client. Our delivery model allows us concentrates our resources on how to best supply candidates in a very competitive MSP/VMS program environment.

COMPANY SIZE

50 to 99 employees

INDUSTRY

Staffing/Employment Agencies

FOUNDED

2014

WEBSITE

http://www.insyncstaffing.com/default.html

Resume Resources

Free Resume Templates Free Resume Builder