On-Premises LLM & Vector Database Implementation Consultant

• Current Openings
• Veterans Program
• Working With Indotronix
• Referral Program

All current openings

On-Premises LLM & Vector Database Implementation Consultant

• Location: Philadelphia, Pennsylvania 19122
• Work model: On-site
• Posted April 8, 2026
• Job ID: JPC - 202407

Easy ApplyApply Now

Job Description

On-Premises LLM & Vector Database Implementation Consultant

Location: Philadelphia, PA | Hybrid (3+ days/week onsite)

Position Type: Contract | Experience: 5-10 years

[About the Role]

Join a pioneering team as an On-Premises LLM & Vector Database Implementation Consultant. Drive the end-to-end deployment of state-of-the-art open-source large language models (LLMs) and vector databases in highly secure, enterprise-grade on-premises environments. This is a hands-on, mid-level consulting opportunity to shape cutting-edge AI infrastructure while collaborating with leading technologists and business stakeholders.

[Responsibilities]

• Deploy, integrate, and optimize open-source LLMs (e.g., Meta Llama 3, Mistral/Mixtral) in private, air-gapped environments
• Develop and implement Retrieval-Augmented Generation (RAG) pipelines using open-source vector databases (Qdrant, Chroma, Milvus, pgvector)
• Engineer Python-based solutions for LLM inference, prompt design, and system integration
• Tune model performance through CPU-based inference, quantization, and metadata management
• Ensure strict data privacy and enterprise security, including access controls and audit logging
• Deliver clear reference architecture, prototype solutions, and comprehensive documentation
• Lead knowledge transfer sessions to upskill internal engineering teams

[Required Skills and Experience]

• 5-10 years in software development with proven on-premises LLM deployment experience
• Advanced Python proficiency for LLM workflows and system integration
• Hands-on work with vector databases and RAG pipeline implementation
• Expertise in model quantization, CPU inference, and performance optimization
• Solid knowledge of enterprise security, access control, and data governance in private environments

[Preferred Skills]

• Experience with LangChain, LlamaIndex
• Familiarity with Rust, Go, or C++ for high-performance backend services
• Exposure to Docker, Kubernetes for on-premises deployments
• Knowledge of vLLM, llama.cpp, Hugging Face Transformers frameworks
• Prior work in regulated, enterprise, or security-focused environments

[Benefits]

• Work with cutting-edge AI technologies in a secure, enterprise setting
• Hybrid work model for enhanced work-life balance
• Opportunity for career growth and upskilling in advanced AI and infrastructure engineering
• Gain exposure to real-world, high-impact AI applications and top-tier professionals

[How to Apply]

Ready to make a tangible impact in the AI space? Submit your resume and a brief cover letter outlining your experience with on-prem LLM and vector database deployments. Onsite interviews required. Take the next step in your AI consulting career with us in Philadelphia!

• --

Easy ApplyApply Now

Interested in this role?

Easy ApplyApply Now

Job details

Job ID JPC - 202407

Location Philadelphia, Pennsylvania, United States

Work Model On-site

Experience 5-10 years

Posted April 8, 2026

Skills LLM RAG Vector DB

Share this job

*