By applying and enforcing standard and best practices, you will design and develop scalable, 24×7 data and reporting services to process billions of events per day. Specific duties will include developing enhancements to our Hadoop/Hive and Redshift based data pipeline and extend to streamed data processing; contributing to data reporting projects including generating internal and external reports or visualizations (CSV, Excel, PDF, interactive web graphics and others) and data distribution / reporting components, real-time bidding or optimization algo using a combination of SQL, Ruby, R, Go, Java, Javascript, D3, Bourne shell scripts, cron jobs, and other relevant technologies. Will also be expected to implement new services and algorithms to improve our product.
The ideal candidate has hands-on data processing experience. Candidate must be familiar with current distributed computing data technologies on commodity servers and have experience with 24×7 production needs.
Requirements
Strong SQL skills on large-scale databases, knowledge of any of the following RDBMS such as MySQL, Postgres, Redshift, Hive or others.
Experience with Linux or Unix based systems – including Bourne shell, cron and other Unix utilities.
Strong software development skills, including experience with Java.
Degree in Computer Science or a related field.
Desired Technology Experience
Experience with at least one of Ruby/Python, Bourne Shell or other scripting languages.
Experience with Hadoop/MapReduce and/or EMR, including experience in developing MapReduce Jobs in Java or developing Hive UDF.
ETL Experience maintaining multiple data systems.
Experience with Ooozie or other Hadoop workflow solutions and experience developing complex data processing pipelines, including experience developing regressions tests and deployment strategies for such environments.
Experience with data reporting solutions – either developed in house or with 3rd party solutions.
Working experience developing and supporting 24×7 production data services and pipelines on Linux systems – including experience being on-call supporting such services. Experience with AWS preferred.
Challenges we are tackling
Reliably processing billions of events per day with no data loss 24×7 on commodity hardware.
Processing events in near real-time.
Building for the fragility of cloud and distributed services.
Complex processing of large amounts of data in an efficient manner.
Reporting, distribution of data, data analysis, data visualization and machine learning algorithms.
Low latency data stores for use in bidding or algo optimization.
Comp & Benefits
Competitive comp based on experience level
Healthcare HMO & PPO
Stock options and 401k
Flexible Spending and Transit Reimbursement Accounts
Numbers & Facts
Location
Boston, Massachusetts
Skills
Adobe Acrobatunmatched
Algorithmsunmatched
Amazon Web Services (AWS)unmatched
Apache Hadoopunmatched
Apache Hiveunmatched
Apiary/Beekeepingunmatched
Best Practicesunmatched
Biddingunmatched
Cloud Computingunmatched
Computer Scienceunmatched
Cron Job Schedulingunmatched
Data Analysisunmatched
Data Managementunmatched
Data Processingunmatched
Data Visualizationunmatched
Database Extract Transform and Load (ETL)unmatched
Distributed Computingunmatched
Electronic Medical Recordsunmatched
Go Programming Language (Golang)unmatched
Health Maintenance Organization (HMO)unmatched
Healthcareunmatched
Javaunmatched
JavaScriptunmatched
Linux Operating Systemunmatched
Machine Learningunmatched
MapReduceunmatched
MySQLunmatched
On Callunmatched
PostgreSQLunmatched
Preferred Provider Organization (PPO)unmatched
Python Programming/Scripting Languageunmatched
R Programming Languageunmatched
Relational Databases (RDBMS)unmatched
Rubyunmatched
SQL (Structured Query Language)unmatched
Scalable System Developmentunmatched
Scripting (Scripting Languages)unmatched
Sh (Bourne Shell) Scriptingunmatched
Software Developmentunmatched
Software Engineeringunmatched
Systems Maintenanceunmatched
Test Plan/Scheduleunmatched
Test Strategyunmatched
Unix Operating Systemsunmatched
Unix Shell Programmingunmatched
Web Graphicsunmatched
🎯
Be found by employers
5,500+ employers search our resume database daily. Add yours to get found by recruiters looking for candidates like you.
Level up your application
Professional resume templates
Browse dozens of recruiter approved resume templates, layouts and formats. Choose your favorite and make it your own in minutes.