Application Programming Interface (API), Artificial Intelligence (AI), Artificial Intelligence (AI) Agents, Cisco Unity, Computer Security, Continuous Deployment/Delivery, Continuous Integration, Cost Control, Data Management, Data Processing, Documentation, GPU (Graphics Processing Unit), Incident Response, Injections, Internet Application, Kernel Programming, Knowledge Transfer, Memory Hardware, Metadata, Microservices, Open Source, Performance Analysis, Performance Modeling, Performance Tuning/Optimization, Python Programming/Scripting Language, REST (Representational State Transfer), Regulatory Compliance, SQL (Structured Query Language), Strategic Planning, Testing, User Interface Design, Web Application Framework
Mandatory skill: Expertise in LangChain, LlamaIndex, Semantic Kernel, AutoGen, or equivalent
Key Responsibilities
• Agent Development: Build and orchestrate autonomous AI agents with multi-step reasoning, tool usage, and workflow chaining using frameworks like LangChain, CrewAI, AutoGen, Semantic Kernel, or LlamaIndex.
• LLM Integration & Optimization: Deploy, fine-tune, and serve open-source LLMs (e.g., Llama 3) using Databricks Model Serving; optimize latency, throughput, and cost.
• RAG & Knowledge Systems: Design advanced RAG pipelines leveraging vector search, embeddings, semantic ranking, and enterprise data sources (structured + unstructured).
• Context Engineering: Develop prompt strategies, memory frameworks, and metadata tagging to improve contextual accuracy and response quality.
• UI & Experience Design: Build intuitive AI-driven applications using Databricks Apps (Streamlit/Dash) or modern web frameworks to enable business consumption.
• Data Engineering for AI: Build reliable data pipelines (batch & streaming) supporting training, inference, and feature generation using Delta Lake.
• Security & Governance: Implement enterprise-grade controls using Unity Catalog (row/column-level security, lineage, auditability) aligned with compliance standards.
• LLM Guardrails & Responsible AI: Implement guardrails (e.g., NeMo Guardrails) for prompt injection prevention, hallucination mitigation, and safe output handling.
• MLOps & AIOps: Establish CI/CD pipelines for AI models and agents, including versioning, monitoring, drift detection, observability, and incident response.
• Performance & Cost Optimization: Optimize model performance, GPU/compute usage, and inference cost efficiency across environments.
• Testing & Evaluation
• Collaboration & Stakeholder Engagement
• Documentation & Knowledge Transfer
Required Skills and Qualifications
Databricks & Lakehouse
• Strong experience with Unity Catalog, Delta Lake, Vector Search, Databricks Workflows, and Model Serving
• Hands-on with Lakehouse architecture patterns
LLMs & Generative AI
• Experience with open-source LLMs (Llama, Mistral, etc.), prompting techniques, and fine-tuning approaches
• Strong knowledge of RAG architectures and embedding strategies
AI Engineering & Frameworks
• Expertise in LangChain, LlamaIndex, Semantic Kernel, AutoGen, or equivalent
• Experience building agentic workflows and multi-agent systems
Programming
• Advanced Python proficiency (APIs, web apps, orchestration, data processing)
• Familiarity with REST APIs and microservices architecture
MLOps & Monitoring
• Experience with MLflow, CI/CD pipelines, model lifecycle management, and observability tools
• Knowledge of drift detection and model performance monitoring
Data Engineering Foundations
• Experience with Spark, SQL, and large-scale data processing
• Familiarity with streaming frameworks (Kafka, Structured Streaming)
Security & Governance
• Expertise in AI security risks (prompt injection, jailbreaks, data leakage)
• Experience implementing governance frameworks and compliance controls.