19 August 2026 10:30 - 11:00
You can't unit test an agent: Building eval frameworks for agentic AI in financial services
Evaluating a single LLM call is hard enough. Evaluating an agent that plans, calls tools, revises its approach, and returns an answer three steps later is a different problem entirely.
In this session, Chhayank Jain will cover what a practical eval framework looks like for agentic systems in financial services, from trajectory evaluation and LLM-as-judge pipelines to the observability layer you need to catch silent degradation before your risk team catches it first.
Key takeaways:
- Why pass/fail unit tests break down for multi-step agentic workflows
- How to build trajectory evaluation pipelines for non-deterministic systems
- Observability patterns that catch silent degradation in production
- What a minimum viable eval stack looks like for regulated financial environments