Security Engineer, Frontier AI Evaluation (Contract)

About Cobalt:

\n

Cobalt is seeking security engineers who write code to produce the expert reasoning, task environments, and evaluation data used to train and assess frontier AI models on security work.

\n

This opportunity is suited to engineers who break and defend real systems: penetration testers, red team operators, security engineers, product and application security engineers, and detection engineers, whether from consultancies, internal security teams, or independent practice.

\n

You do not need prior experience in data annotation or AI research. What matters is that you can find and reason about real weaknesses in software and infrastructure unaided, that you can write your own tooling rather than only running someone else's, and that you can document how you got there clearly enough for another engineer to follow.

\n

All work is performed against sandboxed environments, purpose-built targets, and model endpoints supplied by us or by the lab. We do not accept work performed against systems you are not authorized to test, and we do not accept material covered by a client agreement or obtained without authorization.

\n

\n

What you'll do:

\n

Depending on the project, you may:

\n

• \n
• Produce written reasoning traces on security tasks, capturing how you form and test hypotheses, what you rule out and why, and how you arrive at a working approach, rather than only the end result
• \n
• Author novel security problems, capture-the-flag style challenges, and sandboxed lab environments with verifiable success criteria
• \n
• Evaluate model-generated security content and code, ranking responses, explaining what makes the stronger one stronger, and identifying the specific step at which the technical reasoning breaks down
• \n
• Assess whether stated findings are supported by the underlying evidence, and identify inconsistencies between what is reported and what the target actually does
• \n
• Design rubrics and partial-credit criteria for scoring multistep offensive and defensive tasks
• \n

\n

Projects follow their own guidelines, scope rules, and quality standards, and you will work with feedback from reviewers and lab research teams.

\n

\n

Required qualifications:

\n

• \n
• Demonstrable hands-on security experience, evidenced by professional engagements, published vulnerability research or CVEs, a substantive bug bounty record, competitive CTF results, or comparable work
• \n
• Strong coding ability in at least one of Python, C, C++, Go, Rust, or JavaScript, sufficient to read unfamiliar codebases and build your own tooling
• \n
• Depth in at least one area, for example web and API security, cloud and container security, network and infrastructure testing, or binary exploitation and reverse engineering
• \n
• Ability to explain each step of your reasoning clearly in writing, and to produce documentation another practitioner could reproduce
• \n
• Willingness to work strictly within defined scope and authorization, and to sign a confidentiality agreement covering project materials
• \n

\n

Certifications such as OSCP, OSWE, OSEP, GPEN, or GXPN are useful but not required.

\n

\n

Why join Cobalt AI:

\n

• \n
• Advance frontier AI where it counts. Apply your security expertise to data that frontier labs cannot obtain any other way, where your judgment directly shapes how the next generation of models reasons about security.
• \n
• Grow professionally. Expand your influence through evaluation projects, advisory roles, and research collaborations, while developing a working understanding of how frontier models are trained and assessed.
• \n
• Work with a top-tier network. Collaborate with security engineers and researchers from leading organizations on high-impact, flexible work.
• \n
• Set your own schedule. Flexible 10 to 40 hour weeks that fit around your existing engagements and your life.
• \n
• Competitive pay. Rates vary by project and are determined by a number of factors, including scope, skillset, and experience.
• \n