AI Engineering Manager_Machine Learning

Role: Sr AI Engineering Manager_Machine Learning

Experience: - Min 10+ Years

Location: - Remote USA

We are looking for a Senior AI Engineer to design, build, and deploy high-quality AI-powered features with end-to-end ownership from prototyping to production, ensuring reliable, scalable, and impactful AI solutions.

Responsibilities: -

End-to-End AI Feature Ownership

• Design and implement AI-powered features (LLM workflows, copilots, and agent-based systems with tool use and multi-step reasoning)
• Own the Full Lifecycle: prototyping evaluation production deployment iteration
• Ensure solutions are reliable, performant, and aligned with product needs

AI System Implementation

• Build and optimize prompt pipelines for specific use cases
• Build retrieval systems (embeddings, chunking, ranking)
• Implement RAG-based workflows where needed
• Iterate on outputs to improve quality, accuracy, and consistency
• Design scalable and cost-efficient AI architectures for production workloads
• Select and evaluate models (hosted vs open-source) based on use case constraints

Agent-Based Systems (AgentCore)

• Design and build agentic workflows capable of multi-step reasoning and decision-making
• Integrate agents with tools, APIs, and internal systems to perform real-world actions
• Implement planning, execution, and reflection loops for complex tasks
• Manage context, memory, and state across multi-step interactions
• Balance deterministic workflows vs. agent autonomy for reliability and control

Experimentation & Evaluation

• Run structured experiments to compare approaches (prompting, retrieval, models)
• Define and track key metrics for AI performance (quality, latency, cost)
• Debug and improve non-deterministic system behavior
• Build and maintain evaluation datasets and benchmarks
• Implement automated evaluation pipelines for continuous improvement

Collaboration & Contribution

• Drive technical direction and influence AI adoption across teams
• Partner with product managers and designers to scope AI features
• Contribute to shared patterns and reusable components
• Participate in code reviews and design discussions
• Support and mentor mid-level engineers where needed

AI Reliability, Safety & Governance

• Design guardrails to ensure safe and reliable AI behavior
• Mitigate hallucinations, prompt injection, and model misuse
• Ensure compliance with data privacy and enterprise requirements
• Implement monitoring and observability for AI systems in production
• Implement guardrails for agent actions (tool access control, execution boundaries)
• Prevent failure cascades in multi-step agent

Educational Qualifications: -

• Engineering Degree BE/ME/BTech/MTech/BSc/MSc.
• Technical certification in multiple technologies is desirable.

Skills: -

Mandatory skills

Core AI Skills

• Strong understanding of LLM capabilities and limitations
• Experience with prompt engineering and structured output design
• Hands-on experience with embeddings and vector search
• Familiarity with RAG architectures and when to apply them
• Experience designing agent-based architectures (AgentCore concepts)
• Understanding of tool use, planning strategies, and memory mechanisms in LLM systems

Engineering Skills

• Min 4+ years of related work experience
• Solid backend/system design fundamentals
• Experience building and deploying production-grade systems
• Ability to debug complex issues, including probabilistic outputs
• Comfort working with APIs, pipelines, and data flows

Product Thinking

• Ability to translate user needs into effective AI solutions
• Strong intuition for balancing quality, latency, and cost
• Focus on delivering measurable product impact

Collaboration

• Communicates clearly across engineering and product teams
• Contributes to team knowledge and shared practices.

Good to have skills

• Evaluate agent performance across multi-step tasks (task success rate, error propagation)
• Debug and optimize agent decision-making and tool selection behavior