Application Security Engineer IV – AI Harness

The Application Security Engineer, Agentic Secure Code Harness Engineer is a hands-on role responsible for operating, monitoring, and improving an AI-enabled AppSec harness used to evaluate application and infrastructure source code for security vulnerabilities and insecure-design practices throughout the Secure SDLC lifecycle. The role focuses on harness health, observability, reliability, troubleshooting, evidence capture, and day-to-day operability of the AppSec process.

The engineer partners with AppSec, DevSecOps, platform engineering, AI governance, and application teams to ensure reliability, accuracy of findings, and recommendations are actionable, evidence is repeatable, developer workflows remain aligned to the secure SDLC, AI model governance, and financial-services control expectations.

What You’ll Do:

• Operate and maintain the AI secure-code evaluation harness for source code repositories, SDLC and lifecycle changes, and agentic security workflows.

• Monitor harness health across ingestion, orchestration, model routing, scanner integration, executions, evidence generation, remediations, and reporting.

• Build and tune observability dashboards and alerts for run success, queue depth, latency, cost management, model/API availability, regression failures, missing evidence, and integration outages.

• Troubleshoot issues across development tooling such as: Jenkins, GitHub Actions, GitHub Enterprise, Atlassian, AppSec scanners, context tools, logging platforms, artifact repositories, and harness components.

• Execute recurring operational routines, including run validation, readiness checks, benchmark refreshes, regression reviews, evidence-quality checks, and post-run reconciliation.

• Maintain runbooks, SOPs, support playbooks, recovery steps, known-error documentation, and escalation paths.

• Support secure ingestion and handling of source code, artifacts, scanner output, SBOMs, metadata, golden datasets, logs, and evidence packages.

• Maintain benchmark suites, golden test cases, prompt/model configuration records, retrieval settings, scoring rubrics, and operational test data.

• Validate that findings flow into developer workflows with context, severity, confidence, remediation guidance, traceability, and rejection rationale where applicable.

• Collect audit-ready evidence aligned to NIST SSDF, NIST CSF 2.0, NYDFS, FINRA, SOX ITGC, FFIEC, GLBA, internal AI governance, and technology risk controls.

• Report operational KPIs including run availability, failed-run rate, MTTR, validation cycle time, evidence completeness, cost per validated finding, false-positive trends, and developer remediation adoption.

• Drive automation that reduces manual triage, improves repeatability, lowers operational toil, and increases developer trust in AI-assisted AppSec outcomes.

What Experience You’ll Need:

• Bachelor’s degree in Computer Science, Cybersecurity, Software Engineering, Information Technology, Engineering, or related field, or equivalent practical experience.

• 6+ years of experience in application security, secure software engineering, DevSecOps, platform engineering, security operations, or related cybersecurity engineering roles.

• Hands-on experience supporting security capabilities across CI/CD, source control, ticketing, artifact management, logging, and AppSec reporting workflows.

• Working knowledge of secure code review, vulnerability triage, exploitability analysis, remediation validation, threat modeling concepts, and secure SDLC practices.

• Practical experience with SAST, SCA, DAST, secrets scanning, API security testing, container security, IaC scanning, SBOMs, SARIF, and findings management.

• Experience with Jenkins, GitHub Actions, GitHub Enterprise, Jira/Azure DevOps, developer portals, observability platforms, and AppSec dashboards.

• Strong understanding of logs, metrics, traces, health checks, alert thresholds, run manifests, error budgets, and incident response routines.

• Ability to automate operational workflows using Python, shell scripting, APIs, configuration files, and infrastructure or policy-as-code patterns.

• Familiarity with LLM or AI-assisted engineering concepts such as prompts, model versions, retrieval configurations, guardrails, token usage, latency, cost tracking, and drift monitoring.

• Understanding of secure handling requirements for proprietary source code, credentials, logs, telemetry, evidence packages, and regulated financial-services data.

• Working knowledge of OWASP Top 10, CWE, CVSS, NIST SSDF, NIST CSF 2.0, AI security risks, auditability, and regulated source-code handling.

What Could Set You Apart:

• Certifications such as CISSP, CSSLP, CCSP, AWS/Azure security, Kubernetes security, GIAC application security, or AI governance.

• Experience operating AI-assisted AppSec, software assurance, or vulnerability-validation platforms in a Fortune 500 or regulated financial-services environment.

• Hands-on experience with observability platforms, SIEM integrations, telemetry pipelines, operational dashboards, SLIs, and alert tuning.

• Experience supporting LLM-enabled workflows, including prompt evaluation, regression testing, guardrail monitoring, model routing, cost governance, and human-in-the-loop review.

• Experience maintaining benchmark datasets, golden test cases, validation pipelines, custom static-analysis rules, or exploitability-validation workflows.

• Track record reducing AppSec operational toil, improving evidence completeness, lowering false positives, improving run reliability, and accelerating developer remediation

**Candidates that live within a commutable distance from our Tempe, AZ and St. Louis, MO home office locations are expected to work in the office four days per week effective June 1, 2026. Before June 1, 2026, candidates that live within a commutable distance from our Tempe, AZ and St. Louis, MO home office locations are expected to work in the office three days per week, with preference for Tuesday through Thursday.**