Gen AI Engineer

Position: Observability & Evaluation AI Engineer

Worksite: Charlotte, NC- 2-3 days Onsite

Duration: Contract Opportunity

JobDescription:
Observability & Evaluation Engineer will build telemetry, tracing, dashboards, evaluation suites, alerts, service objectives, runbooks, and readiness evidence for Tachyon agent releases. This role ensures production AI systems can be monitored, evaluated, improved, and supported with clear operational visibility.

Key Responsibilities
Implement observability and telemetry for LLM-powered applications, agents, tools, and platform services.
Build evaluation suites for agent behavior, prompt quality, response quality, retrieval performance, latency, reliability, and safety signals.
Develop dashboards, alerts, traces, metrics, service objectives, and reporting for production readiness.
Work with platform engineers and Product Owners to define monitoring requirements and evaluation metrics.
Automate evidence collection for release readiness, operational reviews, and governance checkpoints.
Create runbooks and support documentation for priority agent releases.
Analyze production behavior and recommend improvements to reliability, performance, and quality.

Best Regards,

Pravati Pratima

Account Manager - Recruitment

Pravati.p@ekcelinc.com

Ekcel Technologies Inc.