Software Engineer

Software Engineer

Location: Mountain View CA (5 days working)

Minimum Basic Requirements

• Python Proficiency: Write, update, and maintain Python frameworks and libraries to support data processing and integration tasks.
• Composer / Apache Airflow: Hands-on experience with Apache Airflow, including updating DAGs, managing multi-node rollouts, and troubleshooting issues
• Code Management: Use Git and GitHub for source control, code reviews, and version management.
• GCP Proficiency: Extensive experience working with GCP services (e.g., Big Query, Cloud Dataflow, Pub/Sub, Cloud Storage, Monitoring). Knowledge of resource labeling and automation through SDKs or APIs.
• Software Engineering: Strong understanding of software engineering best practices, including version control (Git), collaborative development (GitHub), code reviews, and CI/CD.
• Problem-Solving: Excellent problem-solving skills with the ability to tackle complex data engineering challenges.
• Communication: Excellent stakeholder communication skills, with the ability to interface directly with data scientists, platform engineers, and other clients to explain complex technical details, coordinate rollouts, triage issues, and provide updates
• Bachelor's or master's degree in computer science, Engineering, Computer Information Systems, Mathematics, Physics, or a related field or software development training program

What You'll Do

• Develop and enhance Python frameworks and libraries to support cost tracking, data processing, data quality, lineage, governance, and MLOps.
• Implement data processing optimizations to reduce the cost of our larger training data and features pipelines.
• Build scalable features and training data batch pipelines leveraging Big Query, Dataflow and Composer scheduler/executor framework on Google Cloud Platform.
• Implement monitoring, logging, and alerting systems to ensure the reliability and stability of our data and ml infrastructure and pipelines.
• Plan and oversee infrastructure rollouts, including phased deployments, validation, and rollback strategies.
• Act as primary point of contact for Data Scientists, ML Engineers, and other stakeholders handling rollout coordination, communications, and issue resolution.
• Collaborate with ML Platform engineers to ensure seamless integration of updates into workflows.
• Document processes and changes, providing clear runbooks and handoffs for ongoing support.

Preferred Requirements

• Python Mastery: Strong Python development background, with demonstrated experience maintaining and extending production-grade SDKs or internal libraries
• Change Management: Experience with infrastructure rollout and change management, including phased deployments, validation, and rollback strategies
• Resiliency: Comfortable working in fast-moving ML/AI platform environments, where reliability, transparency, and client experience are key
• Batch Pipelines: Experience with building, deploying, and maintaining production batch pipelines processing and publishing petabytes of data.