Private AI for Medical Transcription & Summarization

The problem

Clinicians spend a substantial share of every day writing notes rather than seeing patients. Commercial transcription would help, but a recorded consultation is among the most sensitive data a hospital holds, and sending it to a third-party API is a non-starter under HIPAA.

How it works

The whole pipeline runs on the provider's own infrastructure. Whisper handles speech recognition on-premise, and a locally hosted Llama model turns the transcript into a structured visit summary rather than a wall of text — history, findings, plan, as a clinician would expect to read them.

Summaries are written as FHIR resources, so they enter the existing electronic record as first-class clinical data rather than as an attachment. That is what makes the output usable: a note that has to be copy-pasted into the EHR saves far less time than one that is already there.

What shaped it

HIPAA set the architecture before any modelling decisions were made. Nothing leaves the building, which rules out hosted models and means the system has to perform acceptably on hardware the hospital already owns.

Clinical summarisation also has an asymmetric failure mode. Omitting something the clinician said is dangerous in a way that verbosity is not, so the system was tuned to preserve detail and leave editorial judgement to the reviewing clinician, who signs off before anything reaches the record.

Outcome

Documentation time per visit fell while every byte of patient data stayed on-premise.