Application Security Engineer, Frontier AI Evaluation (Contract)

About the role:

\n

Cobalt is seeking application security engineers to produce the expert reasoning and evaluation data used to train and assess frontier AI models on secure code: writing it, reviewing it, and fixing it.

\n

This opportunity is suited to engineers who sit between security and software development: application and product security engineers, secure code reviewers, security-focused senior engineers, and consultants who do source-level assessments rather than only network testing.

\n

You do not need prior experience in data annotation or AI research. What matters is that you can read unfamiliar code, see the vulnerability class before the scanner does, and articulate why a given fix is correct rather than merely sufficient to close the ticket.

\n

All work is performed against sandboxed environments, purpose-built targets, and model endpoints supplied by us or by the lab. We do not accept work performed against systems you are not authorized to test, and we do not accept material covered by a client agreement or obtained without authorization.

\n

\n

What you'll do:

\n

Depending on the project, you may:

\n

• \n
• Produce written review traces on real code, capturing how you orient in an unfamiliar codebase, which paths you follow, what you rule out and why, and how you confirm a finding is exploitable rather than theoretical
• \n
• Author paired secure and insecure implementations, along with the tests and exploit conditions that distinguish them
• \n
• Evaluate model-generated code and remediations, ranking solutions, explaining what makes the stronger one stronger, and identifying fixes that close the report without closing the underlying flaw
• \n
• Assess whether a proposed fix introduces regressions, breaks intent, or shifts the vulnerability elsewhere, and identify results that pass tests but remain unsafe
• \n
• Design rubrics and partial-credit criteria for scoring multistep review and remediation tasks, and classify findings into a consistent taxonomy
• \n

\n

Projects follow their own guidelines, scope rules, and quality standards, and you will work with feedback from reviewers and lab research teams.

\n

\n

Required qualifications:

\n

• \n
• Demonstrable application security experience, evidenced by professional secure code review or assessment work, published research, a substantive bug bounty record in software targets, or comparable work
• \n
• Strong software engineering ability in at least one of Python, TypeScript or JavaScript, Go, Rust, Java, C, or C++, and comfort reading unfamiliar codebases across more than one language
• \n
• Depth in at least one area, for example web and API security, authentication and authorization design, cryptographic implementation, dependency and supply chain security, or cloud and infrastructure as code
• \n
• Ability to explain each step of your reasoning clearly in writing, at a level another engineer could act on in a pull request
• \n
• Willingness to work strictly within defined scope and authorization, and to sign a confidentiality agreement covering project materials
• \n

\n

\n

Why join Cobalt AI:

\n

• \n
• Advance frontier AI where it counts. Apply your security expertise to data that frontier labs cannot obtain any other way, where your judgment directly shapes how the next generation of models reasons about security.
• \n
• Grow professionally. Expand your influence through evaluation projects, advisory roles, and research collaborations, while developing a working understanding of how frontier models are trained and assessed.
• \n
• Work with a top-tier network. Collaborate with security engineers and researchers from leading organizations on high-impact, flexible work.
• \n
• Set your own schedule. Flexible 10 to 40 hour weeks that fit around your existing engagements and your life.
• \n
• Competitive pay. Rates vary by project and are determined by a number of factors, including scope, skillset, and experience.
• \n