Case Study
Health Tech: Clinical trial document processing that actually ships.
A healthcare software company running document workflows for clinical trial sites. The manual binder review was the bottleneck on everything.
- Sector
- Healthcare SaaS
- Engagement
- Fractional CAIO · 6 months
- Status
- Shipped Q1 2026
Situation
Each clinical trial site onboarded required an analyst to hand-review a 200–800 page regulatory binder. Average time per site: 40 minutes at best, 90 when things went wrong.
Three vendors had pitched AI document solutions. All three demos looked impressive on the one-page sample and fell apart on a real binder.
What we did
- Stood up a labeled evaluation set of 180 real binders spanning three therapeutic areas. Defined success as “≥ 98% field-level precision, zero silent failures.”
- Ran the three vendor tools through the eval. All three failed the silent-failure criterion within 40 documents.
- Designed an internal pipeline: frontier LLM for extraction, deterministic schema validation, a two-stage human-in-the-loop for anything flagged low-confidence, and a versioned audit log.
- Paired with their engineering team to build it in 9 weeks. Sat in on the compliance review and rewrote the workflow diagrams until the Quality officer signed off.
Outcome
- Time per site from 40 minutes to under 4, end-to-end, including QA.
- Zero silent-failure incidents in first 1,100 sites processed on the new system.
- Analyst headcount held flat while site volume grew 3.2× over the engagement.
- Compliance review documented as a reference workflow for the company’s SOC 2 audit.
If this looks like the kind of problem you’re sitting on, an intro call is thirty minutes and costs nothing. We’ll tell you if we’re the right firm. If we’re not, we’ll often know who is.