Research Engineer/Scientist

Location: San Francisco, CA (on-site 5 days/week)

About The Center for AI Safety (CAIS)

The Center for AI Safety (CAIS) is a leading research and advocacy organization focused on mitigating societal-scale risks from AI. Some of our past achievements include: releasing the most widely used measure of AI capabilities used by all major AI companies, running a large compute cluster to facilitate AI safety research which has been cited over 16,000 times, and publishing a global statement on AI Risk signed by Geoffrey Hinton, Yoshua Bengio and top AI CEOs.

What distinguishes us is what we choose to work on. Our work is aimed at reducing real-world risks from advanced AI systems. We deliberately pursue research directions that the field is not yet paying attention to, and we move on once the rest of the field catches up. Our focus is on problems that are both highly important and highly neglected—and our track record is built on getting to them first.

• In 2022–2023, we focused on AI honesty, robustness, transparency, and trojan/backdoor behaviors.
• In 2023–2024, we turned to malicious use and weaponization capabilities, introducing the first state-of-the-art benchmarks for measuring it.
• More recently, we've been working on AI value systems and the functional well-being of AI systems.

This is a research philosophy as opposed to a fixed agenda: we go where the important, unworked problems are. Because our work tends to be timely and to open up territory rather than crowd into it, our papers have repeatedly gone on to become widely cited and to set the standard for underexplored research areas. Our work is regularly used by AI safety institutes and frontier AI labs, and they have shaped real policy outcomes, including being presented directly to senators and policymakers.

The Role

As a Research Engineer (RE) or Research Scientist (RS) at CAIS, you'll lead and execute high-impact research that advances the safety and reliability of frontier AI systems. This is the general posting for both roles—if you're interested in either, apply here, and we'll determine which is the better fit based on our judgment during the process.

You will design and run experiments on large language models, build the tooling to train and evaluate models at scale, and turn results into publishable research. You'll work closely with CAIS researchers and external academic and commercial partners, using our compute cluster to run large-scale training and evaluation.

Our work centers on empirical deep learning research with large language models and/or multimodal models. If your background is primarily theoretical, this role may not be a good fit.

Research directions at CAIS are set by our Research Director, who selects and prioritizes projects for their importance, neglectedness, and timeliness — a major reason our work has been so impactful. In practice, this means that you will consistently be working on interesting, high-impact problems, with substantial freedom in how you pursue them: designing and running experiments, iterating, and chasing the threads you find most promising.

What You'll Do

• Own research experiments end-to-end.
• Train and fine-tune large transformer models across domains.
• Build and maintain datasets and benchmarks.
• Run distributed training and evaluation at scale.
• Write and ship research, collaborating with co-authors, and supporting submissions of papers to top conferences.
• Collaborate with researchers and external partners while contributing to shared research direction and responding quickly in research cycles.
• Support research infrastructure as needed, such as internal tooling, documentation, and reproducibility practices for the team.
• Mentor, guide, and support others on the team.

What We're Looking For

• Have co-authored multiple papers published at a top ML conference venues (e.g., NeurIPS, ICML, ICLR, ACL, CVPR). Workshop papers are considered, though peer-reviewed conference publications are strongly preferred. Publications in journals such as IEEE or Springer Nature are typically given less weight. Alternatively, have made meaningful research contributions at a leading AI lab.
• Are a current PhD student or researcher in machine learning or a related field. Exceptional candidates with a strong publication record may be considered regardless of degree level.
• Have a track record of empirical research in AI or ML, particularly in AI safety-relevant areas (e.g. adversarial robustness, calibration, benchmarking). We weight empirical research heavily; candidates with primarily theoretical backgrounds are generally not a strong fit.
• Are able to read an ML paper, understand the key result, and understand how it fits into the broader literature.
• Are comfortable setting up, launching, and debugging ML experiments.
• Are familiar with relevant frameworks and libraries (e.g., PyTorch).
• Communicate clearly and promptly, and take ownership of your part of a project.

Salary Range

$140,000 - $200,000 annually, based on experience.

Benefits

• Health insurance for you and your dependents
• 401K plan + 4% matching
• Unlimited PTO
• Lunch and dinner at the office
• Annual Professional Development Stipend
• Access to some of the top talent working on technical and conceptual research in AI safety