Case Study
Health Tech: Clinical trial document processing that actually ships.
A healthcare software company running document workflows for clinical trial sites. The manual binder review was the bottleneck on everything.
- Sector
- Healthcare SaaS
- Engagement
- Fractional CAIO · 6 months
- Status
- Shipped Q1 2026
Situation
Each clinical trial site onboarded required an analyst to hand-review a 200–800 page regulatory binder. Average time per site: 40 minutes at best, 90 when things went wrong.
Three vendors had pitched AI document solutions. All three demos looked impressive on the one-page sample and fell apart on a real binder.
What we did
- Stood up a labeled evaluation set of 180 real binders spanning three therapeutic areas. Defined success as “≥ 98% field-level precision, zero silent failures.”
- Ran the three vendor tools through the eval. All three failed the silent-failure criterion within 40 documents.
- Designed an internal pipeline: frontier LLM for extraction, deterministic schema validation, a two-stage human-in-the-loop for anything flagged low-confidence, and a versioned audit log.
- Paired with their engineering team to build it in 9 weeks. Sat in on the compliance review and rewrote the workflow diagrams until the Quality officer signed off.
Outcome
- Time per site from 40 minutes to under 4, end-to-end, including QA.
- Zero silent-failure incidents in first 1,100 sites processed on the new system.
- Analyst headcount held flat while site volume grew 3.2× over the engagement.
- Compliance review documented as a reference workflow for the company’s SOC 2 audit.