Amazon Simple Storage Service (S3), Amazon Web Services (AWS), Best Practices, Data Analysis, Data Lake, Data Management, Data Processing, Data Quality, Data Sets, Documentation, Oracle, People Management, Problem Solving Skills, Project Planning, Reconciliation, Requirements Management, SAP NetWeaver Business Warehouse, Strategic Planning, Warehousing
- Expert on Databricks and AWS, with minimum 10+ years of experience
- Provide guidance and support to teams and analysts.
- Databrick certification is a must
- Collaborate on data strategy with business and IT partners.
- Identify and address data issues.
- Lead architecture, design, and implementation of scalable and efficient data pipelines and analytics solutions on Databricks.
- Work closely with data engineering, platform, and analytics teams to integrate Databricks with AWS S3 Data Lake, Redshift, Talend, SAP BW/HANA, Oracle, and other tools.
- Architect solutions supporting batch and real-time data processing using Delta Lake, Spark, and ML Flow.
- Collaborate with business stakeholders to understand data requirements, assess current state, and provide strategic direction for future-state architecture.
- Ensure best practices for data quality, governance, and security are implemented.
- Own design, development, and delivery of pipelines and models as per the project plan.
- Ensure best practices are followed in performance, security, and governance.
- Provide project documentation and conduct KT sessions
- Ensure following acceptance criteria is met, by working along with current development and pool partners
- Successful ingestion and transformation of in-scope objects.
- Validated and reconciled transformed data within defined thresholds.
- Fully operational Snowflake warehouse with business-validated datasets.
- Documented operational playbooks and KT completed.
- No critical issues post go-live during hyper-care window
T
The Dignify Solutions, LLC