Johnson Service Group logo

Senior AI/QA Engineer (519)

Johnson Service Group
  • Boston, MA
  • Instant Apply
30+ days ago

Job Description


Our client is seeking a Senior AI Engineer with strong AI quality experience to build, evaluate, and improve enterprise AI solutions, with a focus on GenAI, RAG, and agent-based workflows in an Azure environment. (remote US only)

Must Have skills
  • Experience building and supporting GenAI and LLM-based solutions
  • Strong skills in Python
  • Experience with Azure AI services, including Azure AI Foundry or related tools
  • Experience with RAG systems and retrieval-based AI workflows
  • Familiarity with LangChain, LangGraph, or agent-based orchestration
  • Experience validating LLM outputs for quality, accuracy, and consistency
  • Exposure to evaluation frameworks, testing, or quality processes for AI systems

Nice to Have skills
  • Experience with Azure AI Search
  • Exposure to AI test automation or monitoring workflows
  • Familiarity with Responsible AI, governance, or compliance practices
  • Experience with knowledge graphs, GraphRAG, Pinecone, FAISS, OpenSearch, or similar tools
  • Experience in enterprise or regulated environments
  • Experience supporting human-in-the-loop or audit-focused AI processes

Responsibilities
  • Build and improve enterprise AI solutions across GenAI, RAG, and agent-based systems
  • Support development of LLM-powered applications in an Azure-based environment
  • Design and enhance retrieval pipelines and grounded response workflows
  • Contribute to evaluation and validation of AI outputs for quality, safety, and consistency
  • Help develop structured approaches for testing and monitoring AI solutions
  • Work with cross-functional teams to deliver reliable, governed, and production-ready AI systems
  • Support documentation, quality standards, and Responsible AI practices
#LI-SB1
#D480

Numbers & Facts

LocationBoston, MA
IndustryStaffing/Employment Agencies
Company Size1,000 to 1,499 employees
Year Founded1984
Websitehttps://www.jsginc.com/

About Company

At JSG, We Understand People

It is our mission to empower people, help our clients reach their goals, and hire motivated individuals.

Since our start in 1984, we’ve worked hard to establish our present position as the most effective cross-industry staffing and consulting solution available.

Headquartered in Chicago, IL with local offices throughout the United States and Canada, JSG proudly serves a diverse range of industries with the right solutions to help clients reach their company goals. Our client-centric approach enables us to create personalized solutions for businesses in need of top-tier talent.  Our dedication doesn’t stop at our clients. We strive to connect candidates with opportunities that match their skills and goals. More than just a go-between, we are a dependable bridge, uniting top talent with suitable opportunities, facilitating shared opportunities and growth.

Skills

  • Artificial Intelligence (AI)unmatched
  • Cross-Functionalunmatched
  • Microsoft Windows Azureunmatched
  • Python Programming/Scripting Languageunmatched
  • Quality Assuranceunmatched
  • Quality Metricsunmatched
  • Safety/Work Safetyunmatched
  • Support Documentationunmatched
  • Test Automationunmatched
  • Testingunmatched

Be found by employers

5,500+ employers search our resume database daily. Add yours to get found by recruiters looking for candidates like you.

Level up your application

Professional resume templates

Browse dozens of recruiter approved resume templates, layouts and formats. Choose your favorite and make it your own in minutes.

Free resume templates

Free resume builder

Improve your existing resume or start from scratch and create a standout, ATS-friendly resume. Add job-specific content, download and apply.

Free resume builder