Access Control, Cisco Unity, Data Management, Data Sets, Database Extract Transform and Load (ETL), Python Programming/Scripting Language, Reporting Dashboards, SQL (Structured Query Language), Sales Pipeline, User Interface/Experience (UI/UX)
Databricks Data Engineer - Remote, Washington, DC
Duration: 3 months
Seeking for a skilled Databricks Data Engineer to develop and maintain data solutions using the Databricks platform. This role involves building user interfaces and dashboards, as well as managing data pipelines for critical data products.
Key Responsibilities:
- Build and maintain data ingestion and transformation pipelines using PySpark and Databricks SQL.
- Migrate datasets and ETL logic to Delta Lake on Databricks, ensuring data parity with source systems.
- Implement medallion architecture (Bronze/Silver/Gold) with lineage and access controls in Unity Catalog.
- Utilize Databricks Autoloader, Lakeflow Connect, and Structured Streaming for incremental data ingestion.
Skills/Qualifications:
- Hands-on experience with data engineering on Databricks.
- Strong skills in PySpark, Python, and SQL.
- Experience with Delta Lake, medallion architecture, Autoloader, Structured Streaming, and Unity Catalog.