Skills Required: Must have experience with: Designing and implementing highly scalable, optimizing, and distributing big data pipelines and ETLs using frameworks like Hadoop, Hive, Spark, and Databricks; Developing ETL pipelines using Scala, Python, Spark, and SQL; SQL and Spark SQL querying for data analysis and validation; Spark shell scripting for automated data validation; Documenting data pipelines and workflows like Confluence, Jira, and Azure DevOps; Orchestrating and automating implemented pipeline solutions using tools and frameworks like Automic, AutoSys, Oozie, and Apache Airflow; Writing SQLs for data querying from RDBMS and Data warehousing frameworks like Apache Hive and Snowflake; Developing and maintaining modules in cloud ecosystem in Tech like Google Cloud, AWS, and Azure; Working with Parquet, AVRO, CSV, ORC, and JSON; Working in different file-based storage systems, including HDFS, GCS, Azure Blob Storage, and S3; Creating and automating dashboards by using Power BI, Tableau, and Looker; Leading large-scale projects and providing guidance to team members; System architecture, HLD to detailed design, and disaster recovery; Resource modeling, optimization, and forecasting; Applying incident management, change management, defect-reduction, and system-reliability processes to improve operational performance. Minimum education and experience required: Bachelor's degree or the equivalent in Computer Science or related field plus 4 years of experience in software engineering or related experience; OR Master’s degree or the equivalent in Computer Science or related field plus 2 years of experience in software engineering or related experience.