Responsible for the documentation, design,
development, and architecture of Hadoop
Write MapReduce coding for Hadoop clusters; help
to build new Hadoop clusters
Converting hard and complex techniques as well as
functional requirements into the detailed designs
To design web applications for querying data and
swift data tracking at higher speeds
Experience on Google Cloud infrastructure with big
data and big query utilities
To propose best practices and standards; handover
to the operations
Perform the testing of software prototypes and
transfer to the operational team
Pre-processing of data using Pig and Hive
To maintain data security and privacy
Performing the analysis of large data stores andderive insights
Write scripts/program to interact with HDFS and
Assess requirements and evaluate existing solutions.
Build, operate, monitor, and troubleshoot Hadoop
Develop tools, libraries, and maintain processes for
other engineers to access data and write
Evaluate and use hosted solutions on AWS / Google
Cloud / Azure.
Write scalable and maintainable ETLs.
Understand Hadoop’s security mechanisms and
implement Hadoop security.
Write software to ingest data into Hadoop.
Scripting experience using Shell, Python or PowerShell
Good understanding of High availability
Implement and manage Cluster security
Kerberos and Kerberos encryption
Ability to shell script with Linux
Ability to troubleshoot problems and quickly resolve
Performance tuning of Hadoop MapReduce routines
Integration to other Hadoop platforms