We have following urgent role with our DIRECT client
Title: Sr. Data Engineer ( Big Data)Location: San Bruno, CA Duration: 6+ Months Rate: Market
Technical Requirements• Development experience with
Java, Scala, Flume, Python• Development background with
Spark or MapReduce/YARN is a must have • This candidate is expected to have significant application development experience in the past either with Java or Python. We are looking for someone who was a Software Engineer in the past and turned into a Data Engineer and build end to end data pipelines from ground up• Knowledge/experience on Teradata Physical Design and Implementation,
Teradata SQL Performance Optimization• Advanced SQL (preferably Teradata)• Experience working with large data sets, experience working with distributed computing (
MapReduce, Hadoop, Hive, Pig, Apache Spark, etc.).• Strong Hadoop scripting skills to process petabytes of data• Experience in Unix/Linux shell scripting or similar programming/scripting knowledge• Experience in ETL/ processes• Real time data ingestion (Kafka)
Nice to Have• Experience with Teradata Tools and Utilities (FastLoad, MultiLoad, BTEQ, FastExport)• Cassandra• Automic scheduler• R/R studio, SAS experience a plus• Presto• Hbase• Tableau or similar reporting/dash boarding tool• Modeling and Data Science background• Retail industry background Education BS degree in specific technical fields like computer science, math, statistics preferred
Position Summary• Very Strong engineering skills. Should have an analytical approach and have good programming skills.• Provide business insights, while leveraging internal tools and systems, databases and industry data• Minimum of 5+ years’ experience. Experience in retail business will be a plus.• Excellent written and verbal communication skills for varied audiences on engineering subject matter• Ability to document requirements, data lineage, subject matter in both business and technical terminology.• Guide and learn from other team members.• Demonstrated ability to transform business requirements to code, specific analytical reports and tools• This role will involve coding, analytical modeling, root cause analysis, investigation, debugging, testing and collaboration with the business partners, product managers other engineering team.