Pay Rate - $60-70/Hr
Location: Malborough, MA / Hybrid with travel on an as needed basis
Duration: 6 months
Skills: Digital : Databricks~Data Architecture and Modeling
Experience Required: 10 & Above
Responsibilities
- Design, code, and deploy batch and streaming data pipelines using PySpark, Spark SQL, and Lakeflow.
- Ingest data from Point-of-Sale (POS), e-commerce platforms, loyalty systems, and marketing clouds.
- Implement data transformations and business logic within the Medallion Architecture (Bronze, Silver, Gold layers).
- Build and Eligible to workimize Gold dimension tables as the single source of truth.
- Define governed business KPIs using Unity Catalog Metric Views.
- Implement data quality frameworks and cleansing routines to ensure accurate Customer 360 data.
- Monitor, debug, and tune Databricks jobs and Spark clusters for performance and cost-efficiency.
- Implement best practices for partitioning, caching, and data layout in Delta Lake.
- Manage Databricks environments, clusters, and job deployments using Terraform and AWS DevOps/GitHub Actions.
- Champion and implement CI/CD best practices for data pipelines.
- Implement data governance features within Databricks Unity Catalog, including data lineage tracking, fine-grained access controls, ABAC policies, tags, and data masking.
- Partner with Functional Consultants, Data Scientists, and Analytics Engineers to deliver consumption-ready datasets.
- Build and govern Unity Catalog Business Semantics, including Metric Views, Domains, Pages, Glossary Terms, and trusted assets.
- Configure and curate AI/BI Genie Spaces and the Genie Ontology to enable conversational analytics for business users.
- Enable consumption through AI/BI Dashboards and Databricks One.
Required Skills
- PySpark
- Spark SQL
- Lakeflow
- Lakeflow Connect
- Lakeflow Declarative Pipelines
- Terraform
- AWS DevOps
- GitHub Actions
Disclaimer: Diverse Lynx LLC is an Equal Opportunity Employer. All applicants and employees are evaluated without discrimination, based solely on their qualifications, ability, competence and performance. This email and its attachments may contain confidential or proprietary information and is intended only for the recipient(s). If you received this message in error, please disregard it and notify the sender. If you no longer wish to receive our communications, you may unsubscribe at any time.
Security Notice: Our official website is www.diverselynx.com We do not operate or authorize any other websites representing Diverse Lynx LLC.