Overview
The AI Frameworks team at Microsoft accelerates and optimizes large language model deployment on Microsofts MAIA AI accelerators and GPUs. We build software across the stack, from PyTorch and inference systems such as vLLM and SGLang to performance-critical runtime and kernel components. Our team operates at the intersection of AI algorithmic innovation, purpose-built AI hardware, systems, and software, with a highly collaborative and inclusive culture.
We are seeking a self-motivated Senior Software Engineer - AI Frameworks who thrives on technical innovation, enjoys diving deep into technical details, and adapts quickly in a fast-moving environment. This is a unique opportunity to directly shape the software that powers Microsofts most advanced AI infrastructure-from custom silicon to the models running on it.
Microsofts mission is to empower every person and every organization on the planet to achieve more. As employees, we come together with a growth mindset, innovate to empower others, and collaborate to realize our shared goals. Each day we build on our values of respect, integrity, and accountability to create a culture of inclusion where everyone can thrive at work and beyond.
Responsibilities
Architect and implement efficient tensor computation primitives and software abstractions for custom AI accelerators.
Develop and extend PyTorch features for model onboarding, optimization, and execution on custom AI accelerators.
Contribute to and improve AI inference stacks such as vLLM and SGLang, including scheduling, KV cache management, and serving pipelines.
Design, develop, profile, and optimize high-performance kernels for NPUs (MAIA) and GPUs to accelerate LLM inference and training workloads.
Collaborate across disciplines to define requirements and
Deliver practical solutions to new technical challenges.
Qualifications
Required Qualifications:
Preferred Qualifications:
Masters Degree in Computer Science or related technical field AND 6+ years technical engineering experience with coding in languages including, but not limited to, C, C++, or Python OR Bachelors Degree in Computer Science or related technical field AND 8+ years technical engineering experience with coding in languages including, but not limited to, C, C++, or Python OR equivalent experience.
Experience with PyTorch internals, custom operators, hardware backend, or torch.compile/Dynamo-based optimization flows.
Experience with AI inference stacks such as vLLM, SGLang, or similar large-scale model serving systems.
Experience with NPU or GPU kernel development and optimization (e.g., CUDA, Triton, or accelerator-specific toolchains).
Familiarity with common LLM concepts such as attention mechanisms, KV caching, quantization (PTQ/QAT), and distributed parallelism strategies (TP, PP, DP).
#AIInfra
Software Engineering IC4 - The typical base pay range for this role across the U.S. is USD $119,800 - $234,700 per year. There is a different range applicable to specific work locations, within the San Francisco Bay area and New York City metropolitan area, and the base pay range for this role in those locations is USD $158,400 - $258,000 per year.
Certain roles may be eligible for benefits and other compensation. Find additional benefits and pay information here:
https://careers.microsoft.com/us/en/us-corporate-pay
This position will be open for a minimum of 5 days, with applications accepted on an ongoing basis until the position is filled.
Microsoft is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to age, ancestry, citizenship, color, family or medical care leave, gender identity or expression, genetic information, immigration status, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran or military status, race, ethnicity, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable local laws, regulations and ordinances. If you need assistance with religious accommodations and/or a reasonable accommodation due to a disability during the application process, read more about requesting accommodations.