Leading architectural and design decisions and managing ETL projects and providing complex ETL and backend solutions; Designing, developing and managing large-scale Data warehouse, Data Lake solutions and Data infrastructure across multiple systems following SDLC best practices; Utilizing GitHub for version control, JIRA for task tracking, Jenkins for CI/CD pipeline automation, and Apache Airflow for scheduling and monitoring batch job executions across distributed environments; Coding and deploying applications using DevOps by utilizing GitLab, AWS, and Splunk; Developing and maintaining applications, and extract, transform, and load (ETL) leveraging Scala, Python, PySpark, AWS services, PL/SQL and Unix Shell Scripting to manage complex data workflows, automate processes, and enable reliable communication between enterprise systems and applications; Working with Hive, SQL databases including MySQL, Presto, Amazon DynamoDB, RedShift and visualization using Tableau; and . Additionally, employees who have access to the compensation information of other employees or applicants as a part of their essential job functions cannot disclose the pay of other employees or applicants to individuals who do not otherwise have access to compensation information, unless the disclosure is (a) in response to a formal complaint or charge, (b) in furtherance of an investigation, proceeding, hearing, or action, including an investigation conducted by the employer, or (c) consistent with the Company's legal duty to furnish information.