Sign upLog in
JobsJobs in CaliforniaJobs in San Mateo, CAProduction JobsQuality Control Jobs

QA Engineer, AI

Applied Business Software

San Mateo, CA

Apply
JOB DETAILS
SKILLS
Application Programming Interface (API), Artificial Intelligence (AI), Automation, Business Solutions, Data Sets, Loan Operations, Loans, Mortgage, Product Flow, Product Testing, Python Programming/Scripting Language, Quality Assurance, Quality Metrics, Regression Testing, Software Testing, Team Player, Test Automation, Test Case, Test Design, Test Strategy, Testing, User Interface/Experience (UI/UX)
LOCATION
San Mateo, CA
POSTED
Today
About Applied Business Software, Inc.,

Applied Business Software, Inc., (ABS), the maker of The Mortgage Office , is the industry leader in private lending and loan management technology. Since 1978, our software has powered thousands of lenders, including private money firms, municipalities, CDFIs, Tribal Nations, universities, franchisors, third-party servicers and other non-bank lenders, helping them streamline, automate, and scale their lending operations with confidence.

About the role

We are hiring for a QA Engineer, (AI), who will be responsible for validating both: Deterministic systems (APIs, backend logic, UI flows) and Non-deterministic AI systems (LLM outputs, prompt behavior, edge cases). We are building an AI-first product where core functionality depends on prompt engineering and LLM behavior. This position requires someone to design, execute, and automate testing strategies for LLM-based features, and AI-driven workflows. Additionally, they will test end-to-end product flows across frontend, backend, and APIs.

What you'll do
  • Test end-to-end product flows across frontend, backend, and APIs of an event based system
  • Design structured test cases for prompt-based features
  • Build and maintain test datasets
  • Define evaluation criteria for AI outputs
  • Run regression testing when prompts or models change Automate repeatable tests
  • Track quality metrics over time (output quality, failure rate, latency issues)

Qualifications
  • 3+ years in QA / software testing, preferably for LLM based product
  • Strong Python skills (Automation, LLM, Evaluations)
  • Experience testing APIs and full-stack systems
  • Ability to think in edge cases and break systems intentionally
  • Experience working with LLM-based products
  • Experience testing prompt engineering products


Why Join Us?
  • Comprehensive health, dental, and vision coverage.
  • 401(k) with company funded match.
  • Mission-driven company with a 45+ year legacy and industry-leading products trusted by thousands of lenders.
  • Collaborative, supportive team known for long tenure, deep expertise, and a culture built on trust and integrity.
  • Help modernize and evolve the leading platform in private lending technology.
  • Beautiful San Mateo office with a tight-knit culture.
  • Competitive compensation and room for professional growth.
  • Performance-based bonuses.
  • Generous PTO.
  • Modern workspace with on-site amenities.
  • Awesome company Christmas party.


The pay range for this role is:

120,000 - 140,000 USD per year (Hybrid, San Mateo, CA)

About the Company

A

Applied Business Software

Resume Resources

Free Resume TemplatesFree Resume Builder

Similar Job Searches

Inspector Jobs in San Mateo, CA