Big Data Engineer
In the Market Regulation Surveillance Patterns technology team, an expert in this role is vested with responsibilities arising from Client's mission to protect the integrity of US Securities Capital markets. Client has a portfolio of 'surveillance patterns' that look for manipulative and non-compliant behavior in the database of all transactions that occur in the stock market. The database itself consists of tens of billions of records per day. Engineers work with these large volumes of data using state-of-the-art and industry standard technologies, all of which are wholly operated in a cloud computing environment.
In order to operate with ever improving effectiveness, the team operates in a rich culture of performance and innovation. Collaboration within the team and with business stakeholders is frictionless, with significant independence offered in order to experiment with better ways of conducting technology for business value. Technological and career growth opportunities are a natural and every day part of the working environment.
Please read further to get a better feel for the role and the skills that lead to success in this team.
- Analyze system requirements and design responsive algorithms and solutions
- Use big data and cloud technologies to produce production quality code
- Engage in performance tuning and scalability engineering
- Work with team, peers and management to identify objectives and set priorities
- Perform related SDLC engineering activities like sprint planning and estimation
- Work effectively in small agile teams
- Provide creative solutions to problems
- Identify opportunities for improvement and execute
- Experience with cloud based Big Data technologies
- Proficiency in Hive / Spark SQL
- Experience with Spark
- Experience with one or more programming languages like Scala, Python, and/or Java
- Ability to push the frontier of technology and independently pursue better alternatives
About the Company
DIVERSANT LLC is the largest African-American owned IT staffing company in the US and a fully-certified MBE (Minority–owned Business Enterprise). Our growth rate for the past several years has been more than double that for our industry and reflects our ability to successfully provide IT workforce and diversity spend for our Fortune 1000 and midmarket clients. Much of our success is due to our consultative approach. We view our clients and consultants as business partners, and strive to understand their individual needs and challenges. This leads to the development of strategic solutions, rather than the tactical delivery of commoditized goods and services.
We named our company DIVERSANT for a reason - to reflect our belief that the promotion of diversity, in all its forms, is the right way to do business. All activity transacted with and through us constitutes diverse spend and contributes towards your corporate diversity goals.