Wat.ai

AI Software Engineer, Evaluation Lead

May 2026 - Present

At Wat.ai, I built deterministic checks for agent reliability: comparing claims with retrieved evidence and checking that actions stayed within policy.

An answer needs evidence

Illustrative demo

Retrieved evidence

Sample returns policy: unopened items may be returned within 30 days.

Agent response / action

You can return an unopened item within 30 days.

Ready to validate

Check the proposed answer or action against its source before accepting it.

A deterministic example of evidence and permission checks, not a live LLM or a general hallucination detector. Tool actions are simulated; no email is sent.

What I worked on

Architected TRACE’s deterministic agent-evaluation layer by defining Pydantic verifier contracts for pass/fail, evidence, severity, and affected trace steps across attribution, regression testing, and release gates.

Achieved 100% evaluation accuracy across a 25-task policy suite by validating agent execution traces against retrieved documents, expected actions, tool side effects, and final state to detect unsafe or ungrounded behavior.

Built agent guardrail evaluations measuring replay reliability and classifying recovery into six outcome categories.