Model Routing Engineer, zPLATFORMSkip to main content
Menu
About UsOur Products
Product OverviewzPLATFORMzWAREzFABRICzMARKETPLACEzCLOUDEnterprise GPU cloud for AI workloadszEDGEzDATA
Services
Sovereign AI Data CentersHybrid AI Data CenterEnterprise AI Data CentersAI Data CenterDistributed Cell Tower ComputingGPU Financing
Press & Publications
News & PressNewsroom GalleryPublicationsPress Inquiry
Careers
Open RolesFellowship Program
Contact
Buy Compute
Back to all roles
Model Routing Engineer, zPLATFORM
Palo Alto, USAHybridEngineering
190,000-245,000USD / year
About Zettabyte
- AI computing company: we turn GPU hardware into reliable, high-utilisation AI compute through software, orchestration and full lifecycle operations
- What we offer: enterprise GPU cloud and AI inference, software for data centre owners to run their own GPU clouds, turnkey and sovereign AI data centres, edge computing and GPU financing
- Who we work with: GPU owners, data centre operators, enterprises, OEMs, integrators and financiers across multiple markets
- How we work: fast-moving and high-accountability; we value speed, ambition, clarity, ownership and measurable outcomes
Role Summary
The Model Routing Engineer, zPLATFORM builds the services that route each inference request on zPLATFORM to the right model and capacity, balancing quality, latency and cost.
The role works on the request path, the policies that guide routing decisions and the measurement that shows whether those decisions are good.
The Model Routing Engineer partners with inference, platform and product teams to give customers one simple interface across many models.
Reporting Line
Reports to: Co-CTO and CPO
Key Responsibilities
- Routing services: build and operate low-latency, highly available services that direct inference requests across models and regions.
- Routing policies: implement policies that weigh quality, latency, cost and customer preferences, and make them configurable.
- Evaluation: build evaluation sets and online metrics that show the effect of routing choices on output quality and cost.
- Model onboarding: add new open and commercial models to zPLATFORM with consistent APIs and documentation.
- Reliability: handle failover, rate limits and graceful degradation when models or capacity are unavailable.
- Regional and compliance rules: apply data residency and customer restrictions in routing, and follow export-control and sanctions screening requirements with the Export Compliance team.
- Observability: instrument the request path so that latency, errors and cost per request are visible.
- Customer feedback: work with product and solutions teams to turn customer needs into routing features.
Required Qualifications
- 4+ years of backend or distributed systems engineering in production
- Strong programming skills in Go, Rust or Python
- Experience building low-latency APIs, gateways or proxies at scale
- Working knowledge of LLM APIs, inference serving and evaluation methods
- Comfort with experimentation, A/B testing and data-driven decisions
- Clear written communication of design trade-offs
Bonus Qualifications
- Experience with LLM gateways or model routing products
- Background in recommendation, ranking or search systems
- Familiarity with inference serving frameworks such as vLLM
- Experience with multi-region service deployments
What Success Looks Like (First 90 to 180 Days)
- Routing decisions are measured against clear quality, latency and cost metrics
- New models are added to zPLATFORM through a documented, repeatable process
- Failover behaviour is tested and documented
- Customers can set routing preferences through the API
Why This Role Matters
- Product core: routing between models is a defining capability of zPLATFORM.
- Customer value: good routing gives customers better answers for less money without extra work on their side.
Working Norms at Zettabyte
- Bias to action: deliver drafts quickly, then iterate on feedback
- Be precise: double-check numbers and facts; sloppy analysis creates downstream cost
- Make it usable: package outputs for decision-making, not data dumps
- Own the process: whoever drives a deliverable manages stakeholders and timelines
- Learn the ecosystem: understand GPUs, cluster operations and the buyer landscape
Details
- Workplace: Hybrid, 3 days in office and 2 days remote, Palo Alto
- Employment type: Full-time
- Compensation: USD 190,000 to 245,000 a year base salary, plus equity
- Benefits: Medical, dental and vision insurance; 401(k) retirement plan; paid time off and US public holidays; equity
- Work authorisation: Required in the United States
- Equal opportunity: Zettabyte is an equal opportunity employer; qualified applicants with arrest or conviction records will be considered in line with the California Fair Chance Act
Model Routing Engineer, zPLATFORM
LocationPalo Alto, USA
Work arrangementHybrid
Employment typeFull-time
TeamEngineering
190,000-245,000USD / year
Apply
Share
Model Routing Engineer, zPLATFORM
190,000-245,000USD / year
Apply
Products
Product OverviewzPLATFORMzWAREzFABRICzMARKETPLACEzCLOUDzEDGEzDATA
Services
Sovereign AI Data CentersHybrid AI Data CenterEnterprise AI Data CentersAI Data CenterDistributed Cell Tower ComputingGPU Financing
Company
About UsPress ReleasesCareersFAQPricingContact Us
Resources
PublicationsGlossary of TermsBOTT Methodology
Give us a Call
+886-2-2790-6877Taipei, Taiwan
+1 650 260 1009Palo Alto, California
Get in Touch
[email protected]
Memberships & Commitments
Zettabyte is a supplier of the United Nations Global Marketplace (UNGM #1117361) and is committed to aiding the United Nations in achieving its Sustainable Development Goals.
Zettabyte 2026. All Rights Reserved.
Privacy PolicyReturn & Cancellation PolicyTerms of Service