Software Engineer

Our Client, a Health Insurance company, is looking for a Software Engineer for their Columbia, MD/Hybrid location.

Responsibilities:

• Conduct deep technical analysis and testing of AI/ML and Generative AI systems, including validation and verification of model outputs, prompt behavior, and end-to-end application workflows.
• Participate in Business Requirements reviews and collaborate with systems analysts and developers to fully test application changes, including AI/ML model integrations and Gen AI features.
• Understand functional specifications and scope of releases to design robust testing harnesses that ensure quality delivery of AI-powered applications.
• Write complex SQL queries for retrieval of data from multi-database environments, including querying ML feature stores and model metadata. (Strong knowledge of Oracle database required).
• Apply expert systems analysis and design techniques to complex enterprise systems, with a focus on validating Gen AI outputs, LLM responses, and ML model predictions.
• Maintain broad knowledge of data sources/flow, interactions of complex systems, and the capabilities and limitations of AI/ML systems, including model drift, hallucination detection, and bias evaluation.
• Work closely with Product Owners, developers, Data Scientists, and stakeholders to understand AI/ML product specifics and create corresponding test strategies, including evaluation frameworks for generative outputs.
• Execute manual and exploratory testing on frontend (UI) and backend (API) systems, including AI/ML inference endpoints and Gen AI prompt pipelines.
• Develop and maintain testing suites for a large-scale Salesforce environment integrated with AI/ML capabilities such as Salesforce Einstein and external LLM services.
• Responsible for front-end and back-end testing for applications built on the Salesforce platform, including AI-driven features and automated decision systems.

Requirements:

• Detailed knowledge of Salesforce applications, ML model lifecycle, and Generative AI concepts including LLMs, prompt engineering, RAG (Retrieval-Augmented Generation), and vector databases.
• 3+ years of relevant testing experience with a strong background in Software QA with cloud technologies, including 3+ years of experience with hands-on exposure to AI/ML application testing.
• 2+ years of relational database experience including generating complex queries, setting up database connections, and automating query processing, with familiarity in querying ML feature stores or data pipelines.
• Experience in automated and manual testing in cloud environments, particularly for AI/ML and Gen AI applications, including validation of model outputs, embeddings, and LLM-generated responses.
• Experience designing test strategies for non-deterministic systems, including techniques for evaluating accuracy, relevance, and safety of Gen AI outputs.
• 1+ years of experience with BDD/Selenium and automation tools, with a plus for experience in AI testing frameworks.
• Experience using Agile team collaboration and requirements management tools (Jira, Confluence).
• Excellent written and oral communication skills and problem-solving abilities, with the ability to clearly articulate AI/ML testing findings to both technical and non-technical stakeholders.
• Strong aptitude to learn and adapt to emerging AI/ML technologies and evolving Gen AI tooling.
• Familiarity with AWS AI/ML services (SageMaker, Bedrock) or equivalent cloud AI platforms is a plus.
• Bachelor's degree required; technical degree preferred.
• ML concepts and GEN AI concepts

Why Should You Apply?

• Health Benefits
• Referral Program
• Excellent growth and advancement opportunities