Ambient AI Scribes and Clinical Speech Recognition in 2026

Speech recognition in healthcare used to mean dictation. A physician spoke into a microphone, said “comma” and “new paragraph”, and a transcript appeared. In 2026 the category is ambient clinical documentation: a system listens to the visit, separates the speakers, drafts the note, proposes codes and orders, and hands everything to the clinician for review before it reaches the EHR.

This guide is for product teams building healthcare SaaS and for provider organizations deciding what to buy. It covers what changed, how an ambient AI scribe works end to end, what HIPAA governance requires before the first recording, how to keep model errors out of the chart, and how to integrate with the EHR over HL7 and FHIR.

The documentation burden has not gone away

The numbers that made speech recognition attractive in 2020 still hold. Clinicians can spend up to 6 hours a day filling in medical records, and documentation still takes close to half of a physician’s working time even with an EHR in place. Early dictation tools added their own friction: spoken punctuation, slow speech and a manual pass to fix errors afterwards.

What changed is the buyer. According to Bain and KLAS (October 2025, n=228), 70% of providers and 80% of payers now have an AI strategy, ambient documentation is one of the top provider use cases with 1 in 5 organizations at full rollout, and fewer than 5% of providers say deployed AI has missed expectations. Ambient documentation is no longer an experiment. It is a line item.

The error argument also stands. The Journal of Patient Safety estimated some 440,000 deaths a year from preventable medical error, and Johns Hopkins researchers have ranked medical error as the third leading cause of death in the US. Incomplete or late documentation is one contributor. A scribe that produces a complete, timely note at the point of care removes one source of that risk, as long as the note is correct.

From dictation to ambient documentation

Three generations of clinical speech technology are still in use.

1. Back-end dictation. The clinician records, a transcriptionist or a speech engine produces text, and the clinician signs later. Latency is measured in hours.
2. Front-end speech recognition. Text appears as the clinician speaks, with medical vocabularies and voice commands to move between EHR fields. The clinician still structures the note by hand.
3. Ambient AI scribes. A microphone captures the conversation. Speech recognition produces a diarized transcript, and a large language model turns it into a structured note, problem-list updates, orders and codes. The clinician reviews and signs. Microsoft Dragon Copilot (the successor to DAX) and Abridge define the category, and every major EHR vendor now offers a partner or a native option.

The third generation changes the engineering problem. Speech recognition accuracy is table stakes. The hard parts are summarization quality, traceability, governance and integration.

How an ambient AI scribe works

A production scribe is a pipeline with six stages.

1. Consent and capture. The clinician starts a session from the EHR context or a mobile app, confirms patient consent, and audio streams to a private endpoint. Nothing is stored on the device.
2. Speech recognition and diarization. A medical speech model produces a transcript with speaker labels and timestamps. Options include Amazon Transcribe Medical, AWS HealthScribe, hosted vendor APIs, or a self-hosted open model behind your own firewall.
3. Structuring. A language model maps the transcript to the note format the specialty uses (SOAP, H&P, procedure note), extracts medications, allergies and orders, and proposes ICD-10 and CPT codes. Each generated sentence is linked to the transcript spans that support it.
4. Clinician review. The draft appears in a review UI with evidence links and confidence flags. The clinician edits, accepts or rejects each section. Nothing is written to the chart without a signature.
5. EHR write-back. The signed note and any accepted orders go to the EHR over FHIR or HL7 v2 (details below).
6. Audit and evaluation. Every step logs who, what and when. The edits a clinician makes are captured as feedback for evaluation and model tuning.

Stages 3 to 6 are where SaaS products differentiate. Stage 2 is increasingly a commodity.

Where the value shows up

Measure before and after on the same clinicians. The metrics that matter to a health system are operational, not technical.

• Documentation minutes per encounter, and after-hours documentation time
• Note completion time: hours between visit end and signed note
• Coding completeness and query rate from clinical documentation improvement (CDI) teams
• Clinician-reported burden and turnover intent
• Patient-facing time per visit
• Denials attributable to missing or inconsistent documentation

Set a target on two or three of these in the pilot contract. A scribe that saves 5 minutes per encounter across 20 encounters a day returns 100 minutes per clinician per day. Whether it does is an empirical question the pilot has to answer.

HIPAA governance: what has to be true before the first recording

Recording a clinical conversation creates protected health information (PHI) in a new medium. The controls below are the minimum a provider’s compliance team will ask for.

Business associate agreements at every hop. The speech provider, the LLM provider, the cloud provider and your own company each need a BAA. If a vendor cannot sign one, its service cannot touch PHI.

PHI-safe model deployment. Models run in a private network path (a VPC endpoint or on-premises), with no PHI used to train shared models, no retention beyond the configured window, and encryption at rest and in transit with customer-managed keys. Amazon Bedrock and AWS HealthScribe are HIPAA-eligible services; that eligibility still depends on how you configure them.

Consent. Several US states require two-party consent for recording. The workflow needs a documented, auditable consent step, an easy opt-out, and a fallback to non-ambient documentation when the patient declines.

Minimum necessary and access control. Audio and transcripts are more sensitive than the note. Restrict who can play audio, expire it by policy, and log every access.

Audit logs. Immutable logs of capture, model calls, edits and write-backs. These are also what you need to investigate a wrong note.

AI governance. Providers increasingly align AI programs with the NIST AI Risk Management Framework. Document the model versions in use, the evaluation results, the known failure modes and the human review policy. PHTI’s January 2026 review of administrative AI warns that AI layered on flawed workflows “exacerbates the underlying issues”, so fix the workflow first.

Erbis holds ISO 27001 certification and builds to these requirements as a default, not as an add-on.

Keeping hallucinations out of the chart

Language models can produce fluent text that the patient never said. In a clinical note that is a patient-safety issue. Five design rules contain the risk.

1. Draft, never final. The model produces a draft. A licensed clinician signs. The product should make skipping review harder, not easier.
2. Evidence linking. Every sentence in the draft links to the transcript span that supports it. Unsupported sentences are flagged and cannot be signed without explicit confirmation.
3. Structured extraction for high-risk fields. Medications, doses, allergies and orders are extracted as structured data and validated against the EHR’s own lists, not free-generated.
4. No autonomous actions. The scribe proposes orders and codes. It does not place them. An order reaches the EHR only after the clinician accepts it.
5. Evaluation and monitoring. Maintain a labeled evaluation set per specialty. Track omission rate, fabrication rate and clinician edit distance per model version. Watch for drift after model updates, new specialties, and new accents or acoustic environments.

These are the same disciplines Erbis applies to any agentic system: guardrails, evaluation and observability, built as part of the product. See Agentic AI Engineering.

EHR integration: HL7 v2, FHIR and SMART on FHIR

A scribe that lives outside the EHR gets abandoned. Integration has three parts.

Launch and context. SMART on FHIR lets the scribe launch inside Epic, Oracle Health (formerly Cerner), MEDITECH or athenahealth with the current patient and encounter already selected, and with the clinician’s identity carried over. This removes patient-matching errors.

Read. Over FHIR R4 the scribe reads Patient, Encounter, Condition, MedicationRequest and AllergyIntolerance, so the draft is grounded in the chart and not only in the conversation.

Write. The signed note goes back as a DocumentReference or Composition. Proposed orders become ServiceRequest or MedicationRequest resources that the clinician signs in the EHR. Where FHIR write is not available, HL7 v2 messages (MDM for documents, ORM for orders) or CDA documents still do the job, and older interface engines expect them.

Plan for the EHR vendor’s app program and its review process. It adds weeks to the timeline and is not optional for production use at most health systems.

Build, buy or integrate: options for a healthcare SaaS

There are three realistic paths.

• Integrate a scribe API. Fastest. You embed a vendor’s capture and note generation and own the workflow, review UI and EHR integration. You also inherit the vendor’s pricing and roadmap.
• Assemble on managed components. Use a HIPAA-eligible speech service and a private LLM endpoint, and build the structuring, evidence linking, review and integration yourself. More control over cost, data and specialty behavior; more engineering.
• Build the models too. Only justified when you have proprietary data at scale or a specialty the market ignores.

Most SaaS products land on the second path within a year. It keeps the differentiating parts, the workflow and the integration, in-house.

A reference architecture on AWS

The architecture below is the one Erbis uses as a starting point for HIPAA-governed voice products. It runs entirely in the client’s AWS account.

• Capture: a web or mobile client streams audio over TLS to an API behind a private load balancer.
• Speech layer: Amazon Transcribe Medical or AWS HealthScribe for transcription and diarization, or a self-hosted model on GPU instances inside the VPC.
• Structuring: a language model on Amazon Bedrock, called through a VPC endpoint, with prompts and schemas versioned in code and outputs validated against JSON schemas.
• Review service: the application layer that stores drafts, evidence links and edits, with row-level access control.
• Integration: a FHIR client and, where needed, an HL7 v2 interface, with retries and an idempotent write-back queue.
• Security and audit: KMS customer-managed keys, CloudTrail and application audit logs, retention policies per data class, and no PHI in observability tools.

Erbis is an AWS Partner (Select Tier) and has run production AI since 2017; this pattern is a variation of what we deploy for other regulated workloads.

Beyond the note: voice across the clinical workflow

Once the pipeline exists, it can serve more than the visit note.

• Clinical documentation improvement queries and coding suggestions from the same transcript
• Prior-authorization requests assembled from the note and the chart (our EHR implementation checklist covers how the January 2026 CMS rule changes the timelines)
• Patient-facing visit summaries in plain language
• Nursing handoffs and procedure notes
• Voice control of bedside devices and monitors, with the same constraints as before: authenticated users, secure transport and reliable recognition in noisy rooms

Each of these has the same governance profile as the note, so the controls above carry over.

What Erbis brings

Erbis has built regulated software since 2012 and production AI since 2017, with 200 engineers across San Francisco, Kraków and Dnipro. The founders’ background includes HIPAA-governed laboratory systems used by Mayo Clinic and a Pfizer-funded clinical-trial system. We built a HIPAA-compliant clinical-research EHR from idea to production, and we work on healthcare systems where the EHR, not a demo, is the deployment target.

If you are scoping an ambient documentation product or an integration, a fixed-fee 4-week AI pilot is the shortest route to a go/no-go decision with real clinicians.

FAQ

Is ambient clinical documentation HIPAA compliant?

It can be. The recording and transcript are PHI, so every vendor in the chain needs a business associate agreement, models must run in a PHI-safe deployment with no training on your data, and access, retention and audit controls must be in place. Compliance is a property of the deployment, not of the product.

Do patients have to consent to an AI scribe?

Yes, and in states with two-party consent laws the requirement is legal, not only ethical. Build consent into the session start, log it, and support opt-out with a non-ambient fallback.

How accurate are AI scribes?

Speech recognition accuracy on clinical conversations is high with current models, but the relevant measure is note quality: omissions, fabrications and edit distance after clinician review. Track those per specialty and per model version rather than relying on a vendor’s headline word error rate.

Can an AI scribe write directly to Epic or Oracle Health?

Yes, through SMART on FHIR launch and FHIR R4 write of DocumentReference or Composition resources, or through HL7 v2 and CDA where the site’s interface engine requires it. Vendor app-program review is usually required before production.

How long does it take to build a scribe product?

Integrating a vendor scribe into an existing SaaS can be piloted in about 4 weeks. Assembling your own pipeline on managed speech and LLM services, with a review workflow and FHIR integration, is typically a 3 to 6 month build to a production pilot, depending on the EHRs you need to support.

Talk to Erbis about your healthcare voice product.