Build data pipeline frameworks to automate high-volume and real-time data delivery for our Spark and streaming data hub Transform complex analytical models in scalable, production-ready solutions Provide support and enhancements for an advanced anomaly detection machine learning platform Continuously integrate and ship code into our cloud production environments Develop cloud based applications from the ground up using a modern technology stack Work directly with Product Owners and customers to deliver data products in a collaborative and agile environment Your Responsibilities Will Include Developing sustainable data driven solutions with current new generation data technologies to drive our business and technology strategies Building data APIs and data delivery services to support critical operational and analytical applications Contributing to the design of robust systems with an eye on the long-term maintenance and support of the application Leveraging reusable code modules to solve problems across the team and organization Handling multiple functions and roles for the projects and Agile teams Defining, executing and continuously improving our internal software architecture processes Being a technology thought leader and strategist Knowledge, Skills & Experience Education BS degree in Computer Science, Data Engineering or similar Intermediate to senior level experience in an Apps Development role. Demonstrated strong execution capabilities Required 5+ years of experience on designing and developing Data Pipelines for Data Ingestion or Transformation using Java or Scala or Python At least4 years of experience in the following Big Data frameworks: File Format(Parquet, AVRO, ORC), Resource Management, Distributed Processing and RDBMS 5+ years of developing applications with Monitoring, Build Tools, Version Control, Unit Test, TDD, Change Management to support DevOps At least 2 years of experience with SQL and Shell Scripting experience Experience of designing, building, and deploying production-level data pipelines using tools from Hadoop stack (HDFS, Hive, Spark, HBase, Kafka, NiFi, Oozie, Apache Beam, Apache Airflow etc).