Inference Optimization EngineerSkip to main content
Menu
About UsOur Products
Product OverviewzPLATFORMzWAREzFABRICzMARKETPLACEzCLOUDEnterprise GPU cloud for AI workloadszEDGEzDATA
Services
Sovereign AI Data CentersHybrid AI Data CenterEnterprise AI Data CentersAI Data CenterDistributed Cell Tower ComputingGPU Financing
Press & Publications
News & PressNewsroom GalleryPublicationsPress Inquiry
Careers
Open RolesFellowship Program
Contact
Buy Compute
Back to all roles
Inference Optimization Engineer
Palo Alto, USAHybridEngineering
200,000-260,000USD / year
About Zettabyte
- AI computing company: we turn GPU hardware into reliable, high-utilisation AI compute through software, orchestration and full lifecycle operations
- What we offer: enterprise GPU cloud and AI inference, software for data centre owners to run their own GPU clouds, turnkey and sovereign AI data centres, edge computing and GPU financing
- Who we work with: GPU owners, data centre operators, enterprises, OEMs, integrators and financiers across multiple markets
- How we work: fast-moving and high-accountability; we value speed, ambition, clarity, ownership and measurable outcomes
Role Summary
The Inference Optimization Engineer makes AI models served on Zettabytes platform faster and more cost-efficient, improving latency, throughput and GPU utilisation for zPLATFORM and zCLOUD customers.
The role works across model serving frameworks, kernels and hardware to find where time and memory are spent, and turns that analysis into measurable gains in production.
The Inference Optimization Engineer partners with product, platform and customer-facing teams to bring the best-performing serving configurations to customers.
Reporting Line
Reports to: Co-CTO and CPO
Key Responsibilities
- Profiling: profile inference workloads end to end to find bottlenecks in compute, memory, networking and scheduling of requests.
- Serving frameworks: tune and extend open-source serving engines such as vLLM, TensorRT-LLM and SGLang for the models customers use.
- Model optimisation: apply quantisation, speculative decoding, batching and caching techniques while protecting output quality.
- Benchmarking: build repeatable benchmarks that measure latency, throughput and cost per token across models and GPU types.
- Hardware fit: recommend the best GPU and configuration for each model and traffic pattern, and document the trade-offs.
- Production rollout: move optimisations from experiments into production safely, with monitoring and rollback plans.
- Customer support: help solutions engineers and customers diagnose performance issues on dedicated deployments.
- Knowledge sharing: publish internal guides and, where approved, external technical content on inference performance.
Required Qualifications
- 4+ years in performance engineering, ML systems or high-performance computing
- Deep experience with at least one LLM serving framework (vLLM, TensorRT-LLM, SGLang or equivalent)
- Strong Python and C++ skills; CUDA or Triton kernel experience
- Solid understanding of transformer architectures and GPU memory hierarchy
- Experience with profiling tools such as Nsight Systems, Nsight Compute or PyTorch Profiler
- A rigorous approach to benchmarking and to reporting results honestly
Bonus Qualifications
- Contributions to open-source inference or compiler projects
- Experience with multi-GPU and multi-node inference for large models
- Familiarity with non-NVIDIA accelerators
- Published work or talks on model serving performance
What Success Looks Like (First 90 to 180 Days)
- A benchmark suite covers the most-served models and GPU types
- Measured improvements in latency or cost per token are live in production for priority models
- Serving configuration guides are available to solutions and customer teams
- Performance regressions are caught by automated checks before release
Why This Role Matters
- Customer economics: faster inference at lower cost is a direct competitive advantage for zPLATFORM and zCLOUD.
- Utilisation: every gain in efficiency means more value from each GPU Zettabyte operates.
Working Norms at Zettabyte
- Bias to action: deliver drafts quickly, then iterate on feedback
- Be precise: double-check numbers and facts; sloppy analysis creates downstream cost
- Make it usable: package outputs for decision-making, not data dumps
- Own the process: whoever drives a deliverable manages stakeholders and timelines
- Learn the ecosystem: understand GPUs, cluster operations and the buyer landscape
Details
- Workplace: Hybrid, 3 days in office and 2 days remote, Palo Alto
- Employment type: Full-time
- Compensation: USD 200,000 to 260,000 a year base salary, plus equity
- Benefits: Medical, dental and vision insurance; 401(k) retirement plan; paid time off and US public holidays; equity
- Work authorisation: Required in the United States
- Equal opportunity: Zettabyte is an equal opportunity employer; qualified applicants with arrest or conviction records will be considered in line with the California Fair Chance Act
Inference Optimization Engineer
LocationPalo Alto, USA
Work arrangementHybrid
Employment typeFull-time
TeamEngineering
200,000-260,000USD / year
Apply
Share
Inference Optimization Engineer
200,000-260,000USD / year
Apply
Products
Product OverviewzPLATFORMzWAREzFABRICzMARKETPLACEzCLOUDzEDGEzDATA
Services
Sovereign AI Data CentersHybrid AI Data CenterEnterprise AI Data CentersAI Data CenterDistributed Cell Tower ComputingGPU Financing
Company
About UsPress ReleasesCareersFAQPricingContact Us
Resources
PublicationsGlossary of TermsBOTT Methodology
Give us a Call
+886-2-2790-6877Taipei, Taiwan
+1 650 260 1009Palo Alto, California
Get in Touch
[email protected]
Memberships & Commitments
Zettabyte is a supplier of the United Nations Global Marketplace (UNGM #1117361) and is committed to aiding the United Nations in achieving its Sustainable Development Goals.
Zettabyte 2026. All Rights Reserved.
Privacy PolicyReturn & Cancellation PolicyTerms of Service