Role: AI Systems & Infrastructure Engineer Location: San Jose, CA (Hybrid) Duration: 3+ months
Summary: We are seeking an expert AI Systems & Infrastructure Engineer to design, build, and optimize the end-to-end environment for large-scale AI models. You will bridge the gap between GPU kernels and production serving, focusing on distributed inference orchestration, hardware-software co-optimization, and the elimination of system-level bottlenecks to ensure maximum throughput and minimum latency.
Key Responsibilities:
End-to-End Infrastructure: Architect the full lifecycle from GPU resource allocation to high-performance serving layers.
Distributed Orchestration: Implement Tensor and Pipeline Parallelism to deploy massive models across multi-GPU/multi-node clusters.
System Optimization: Maximize hardware utilization via advanced AI memory management (KV cache, PagedAttention, quantization).
Profiling & Benchmarking: Systematically identify AI bottlenecks (NVLink, PCIe, HBM bandwidth) and establish rigorous TPS/TTFT benchmarking suites.
Deployment & Testing: Build AI-specific CI/CD pipelines for automated performance gating, model validation, and regression testing.
Co-Design: Align model architectures with target hardware constraints to optimize underlying inference engines.
Technical Qualifications:
Frameworks & Acceleration
Core: Expert PyTorch and TensorFlow; proficient in Jupyter/Colab for profiling.
Serving Engines: Deep expertise in vLLM, SGLang, Ollama, and Nvidia NIM.
Acceleration: Advanced knowledge of Nvidia Dynamo (TorchDynamo) and graph-compilation for execution path optimization.
Systems & Infrastructure
Distributed Compute: Proficiency in multi-node communication (NCCL, MPI) and distributed inference strategies.
Memory & Precision: Expert in GPU memory layouts, quantization (INT8, FP8, NF4), and PagedAttention.
Profiling Tools: Mastery of NVIDIA Nsight, PyTorch Profiler, and Triton for kernel and memory access analysis.
Deployment: Experience building AI-centric CI/CD pipelines with automated performance gating and canary deployments.
Hardware & Architecture
GPU Architecture: Deep understanding of H100/A100 internals (SMs, Tensor Cores, HBM, NVLink/InfiniBand).
Bottleneck Analysis: Ability to diagnose and resolve compute-bound, memory-bound, and I/O-bound workloads.
Validation: Experience in stress testing, load balancing, and failover validation for distributed AI nodes.
Preferred Qualifications:
Custom CUDA kernel development or Triton optimization.
Kubernetes (K8s) for GPU orchestration and scheduling.
Experience with distributed training (DeepSpeed, Megatron-LM).
OS-level knowledge of memory paging and asynchronous I/O.
Soft Skills:
Systems Thinking: Ability to map computational operations directly to physical hardware.
Analytical Rigor: Data-driven approach to tuning based on profiles rather than intuition.
Collaboration: Ability to translate researcher requirements into concrete infrastructure specs.
Numbers & Facts
Location
San Jose, CA
Skills
Analysis Skillsunmatched
Artificial Intelligence (AI)unmatched
Benchmarkingunmatched
CUDA (Compute Unified Device Architecture)unmatched
Concreteunmatched
Continuous Deployment/Deliveryunmatched
Continuous Integrationunmatched
Failoverunmatched
GPU (Graphics Processing Unit)unmatched
Hardware Architectureunmatched
Identify Issuesunmatched
Inference Engineunmatched
Input/Outputunmatched
Kernel Programmingunmatched
Load Balancingunmatched
Load Testingunmatched
MPIunmatched
Memory Hardwareunmatched
Memory Managementunmatched
Model Validationunmatched
Operating Systemsunmatched
PCI Express (PCI-E)unmatched
Performance Modelingunmatched
Process Improvementunmatched
Regression Testingunmatched
Research Skillsunmatched
Resource Managementunmatched
Short Messaging Service (SMS)unmatched
Stress Testingunmatched
Systems Engineeringunmatched
Team Playerunmatched
Testingunmatched
🎯
Be found by employers
5,500+ employers search our resume database daily. Add yours to get found by recruiters looking for candidates like you.
Level up your application
Professional resume templates
Browse dozens of recruiter approved resume templates, layouts and formats. Choose your favorite and make it your own in minutes.