CASE STUDY / AGENTIC AI
LedgerLens
Observable AI Financial Assistant
Financial answers with an inspectable tool trace.
Local live-model testing01 / The problem
Financial questions often require both structured calculations and contextual retrieval. An answer needs an inspectable trail of the tools and evidence used.
02 / My approach
The agent uses four tools over an analytical warehouse and semantic store. The interface streams responses and tool traces, with views for metrics and evaluation results.
- React interface
- Claude agent / 4 tools
- DuckDB + ChromaDB
- Citations & trace
03 / Engineering decisions
- Keep analytical queries and semantic retrieval behind explicit tools.
- Validate citation identifiers against observed tool output and expose execution traces.
- Separate deterministic scorer tests from locally executed live-model evaluations.
04 / Results & current state
Locally exercised with live model calls; an operating AWS deployment is not verified.
Historical live-model evaluation
Historical report: 2026-08-26. Timestamp of the committed report
Historical results on a small 34-case set; not general financial-answer accuracy or verification of later changes. One judge false negative and its manual adjudication are retained in the report.
Original 34-case evaluation report05 / Limits of the evidence
The August 26, 2026 report covers a small historical 34-case set. It is not general financial-answer accuracy and does not verify subsequent changes. Citation-ID validation checks whether cited identifiers appeared in tool results; it does not prove semantic support for every claim. Historical report latency includes judge execution; it is not agent-only latency. Agent-only cost must not be described as total evaluation cost. External users and an operating AWS deployment are not verified.
Public sources
Reviewed September 8, 2026. Links are pinned to the inspected public repository revision.
Project documentationReact/TypeScript frontend manifestNode.js/TypeScript agent manifestCitation-ID validator