Want to know if you’re a fit? Upload your resume and let our AI show you.
Skills
Artificial Intelligence (AI)unmatched
Financial Servicesunmatched
Healthcareunmatched
Logisticsunmatched
Problem Solving Skillsunmatched
Product Developmentunmatched
Python Programming/Scripting Languageunmatched
Resolve Customer Issuesunmatched
Software Engineeringunmatched
Technical Strategyunmatched
Description
The Company Headquartered in Dallas TX, our client is a technology and strategy consultancy that aims to provide a competitive edge for its clients by solving complex problems with data, software, and strategy. They specialize in areas like technology strategy, product development, software engineering, and digital transformation, with a particular emphasis on AI, MLOps, and Data Engineering. The firm's clientele spans various industries, including AgTech, Healthcare, Logistics, and Financial Services.
Platform / Stack You will work with technologies that include RAG, Agentic AI, Python, and LLMOps.
Job Responsibilities
What You'll Do:
Own the end-to-end technical quality of generative AI systems — from data preparation and retrieval infrastructure through model integration, prompt design, output evaluation, deployment, and production monitoring.
Establish and enforce GenAI engineering standards across the team: prompt versioning and management, evaluation harness design, context window strategy, output quality testing, hallucination tracking, and system documentation requirements.
Make final technical decisions on LLM selection, context architecture, retrieval strategy, fine-tuning approaches, and orchestration frameworks — with the judgment to know when a simpler system outperforms a complex one.
Own the technical risk register for each engagement — identifying context poisoning risks, hallucination failure modes, latency bottlenecks, cost overruns, and compliance exposure before they surface in production.
Design end-to-end GenAI system architectures that integrate LLMs cleanly with enterprise data platforms, application layers, and operational workflows — built for reliability, observability, and controlled evolution.
Architect retrieval-augmented generation (RAG) systems with rigorous attention to chunking strategy, embedding model selection, vector store design, retrieval quality evaluation, and reranking — treating retrieval as an engineering discipline, not an afterthought.
Design agentic AI systems with well-defined tool interfaces, error handling, state management, and human-in-the-loop controls — architectures that behave predictably under real enterprise data and user behavior.
Architect LLMOps foundations covering model gateway management, prompt registry, evaluation pipelines, A/B testing for prompts and models, cost monitoring, and production observability with output quality tracking.
Lead and mentor a team of AI engineers and data scientists — setting technical direction, unblocking delivery, and raising the engineering quality of every individual contributor on the engagement.
Represent the technical voice of the GenAI team in client-facing settings — communicating system behavior, failure modes, cost implications, and production risks with precision and candor.
Establish incident response procedures for GenAI systems — owning the technical response when output quality degrades, retrieval pipelines drift, context windows overflow, or serving infrastructure fails under load.
Ensure all GenAI systems meet client data governance, privacy, and compliance requirements — including data residency, PII handling in context, audit logging, and prompt injection defense at the architecture level.
Numbers & Facts
Location
Dallas, TX
Job Type
Full-time, Employee
Salary
$120,000–$145,000 Per Year
Company Size
51 - 200
Year Founded
2010
Headquarters
San Diego, CA, US
Additional Compensation
Bonus
Website
https://www.ctpconsulting.com
Qualifications
You could be a great fit if you have:
7+ years in software or ML engineering; 3+ years with direct hands-on ownership of production generative AI or LLM systems at enterprise scale.
Deep production experience with LLM integration patterns — RAG architectures, function calling, tool use, structured output generation, and multi-turn conversation management — beyond API wrappers and demo-grade implementations.
Strong engineering foundation in Python, software design principles, testing practices, and the discipline to build GenAI systems that engineering teams can operate, debug, and maintain without the original author present.
Proven hands-on experience with orchestration frameworks such as LangChain, LlamaIndex, or LangGraph, and vector databases including Pinecone, Weaviate, pgvector, or Chroma in production retrieval systems.
Demonstrated ability to design and operate LLM evaluation frameworks — going beyond vibe-checking to build systematic, metric-driven evaluation pipelines that catch regressions before they reach users.
Proven ability to lead and mentor technical teams on consulting timelines — setting standards, conducting reviews, and developing individual contributors under delivery pressure.
Strong communication skills — able to translate LLM behavior, system trade-offs, cost implications, and failure modes clearly for client engineering leads, product owners, and executive stakeholders.
Preferred:
Experience with LLM fine-tuning and adaptation techniques including LoRA, QLoRA, RLHF, and DPO — with an understanding of when fine-tuning earns its cost over well-engineered RAG or prompting.
Familiarity with multi-agent orchestration frameworks such as AutoGen, CrewAI, or LangGraph for complex, multi-step reasoning workflows in enterprise production contexts.
Background in one or more of the company’s core verticals where GenAI creates direct operational value: AgTech, Logistics, Financial Services, or Construction.
Experience with LLM gateway and proxy platforms (LiteLLM, Portkey, Azure AI Studio) for model routing, cost control, and observability across multi-model deployments.
Professional certifications (e.g., AWS ML Specialty, Google Professional ML Engineer, Azure AI Engineer Associate).