Experience optimizing performance using caching layers such as Redis, parallelization or async workflows, and chunking and retrieval tuning, and designing automated evaluation pipelines using benchmark datasets, LLM-as-a-judge techniques, regression testing, and human evaluation workflows. Experience with cloud-native architectures, including AWS services such as Bedrock, SageMaker, EKS or ECS, S3, IAM, Lambda, CloudWatch, and Secrets Manager, and agent frameworks and orchestration patterns such as LangGraph, tool calling, and function calling APIs.