Transcription and ASR

Verbatim vs Clean Transcripts

A verbatim transcript records every sound as spoken, including the stutters, false starts and “um.” A clean transcript removes them for reading. Clean is better to read and useless to edit with: you cannot cut a filler the transcript does not contain, and a transcript that skipped a 200 ms “um” has a hole in its own timeline.

How ASR Works

• Features. Audio is resampled (typically to 16 kHz mono) and converted to a log-mel spectrogram: energy per frequency band in short overlapping frames, such as a 25 ms window every 10 ms.
• Models. CTC models predict a symbol per frame and are fast; RNN-transducers add a model of the words so far and suit live streaming; attention encoder-decoders such as Whisper generate text from a window of audio and give the best offline accuracy, but can also produce fluent text with no audio support at all.
• The language model has opinions. Every system weighs how the audio sounds against what sentence is likely. That fixes obvious slips for free and is why proper nouns, product names and jargon, the words a general model finds unlikely, come back wrong.

Word Error Rate

WER = (S + D + I) / N

reference:  we shipped it on friday
hypothesis: we shipped on a friday
            -> 1 deletion ("it") + 1 insertion ("a") over 5 words = 40%

• It can exceed 100%, because insertions are unbounded. A model that invents a sentence over silence can score several hundred percent.
• Normalization moves it by points. Casing, punctuation and digits versus words must be handled identically before two WERs are comparable.
• Every word counts the same. Losing “not” costs as much as losing “the,” so WER understates errors that change meaning. Judge a transcript by whether names, numbers and the words you will search for are right.

Why Accents, Overlap and Silence Are Hard

• Accents. Error rates follow the training data. A 2020 PNAS study of five commercial systems measured an average WER of 0.35 for Black speakers against 0.19 for white speakers, traced to pronunciation rather than vocabulary.
• Overlap. A standard model emits one stream of text. When two people talk at once it transcribes the louder voice and silently drops the other. Separate microphones solve this better than any model.
• Silence. Generative models can hallucinate. A 2024 study of Whisper found that roughly 1% of transcriptions contained whole invented phrases, and long non-speech stretches made it more likely.

What a Timed Transcript Unlocks

A transcript with sentence times is a document. A transcript with per-word times is an index: search a 53-minute interview for “pricing” and you get the exact second it starts, every filler becomes a cuttable range, captions come out already timed, and text-based editing becomes possible. You can read 8,000 words in minutes to find the three moments worth keeping, which watching at 2x cannot match.

Common Misconceptions

• “ASR is solved.” It matched humans on one benchmark under specific conditions. A room microphone, three speakers and jargon are a different problem.
• “If the model is unsure, it will say so.” Generative models return fluent text whether or not the audio supports it.
• “Diarization tells you who is speaking.” It tells you when the speaker changed. Names come from you or from enrolled voice samples.
• “A better model will fix bad audio.” Reverb, noise and overlapping voices lose information at capture. Microphone placement outranks model choice.

How Valmera Uses It

In Valmera the transcript is what the editing agent works from. Every upload is transcribed once, with word-level timestamps, speaker labels and automatic language detection, and the transcription keeps filler words so they can be cut. The agent searches it to find moments, cuts on its word boundaries, reads back the transcript the edit keeps to verify cuts and retakes, and builds captions from it. The same transcript tools are available to Claude, ChatGPT and other assistants through the Valmera MCP server.

In Studio you can read the transcript, click a word to jump there, and correct misheard names, which updates the captions without changing the audio. There is no SRT or VTT export and no audio denoise. English is the best-tested language, and overlapping speakers transcribe less reliably than one voice at a time, so check names and numbers before you export. See the auto subtitle generator.

Related Terms

• Word-level timestamps — The per-word times that turn a transcript into an index.
• Text-based video editing — Editing a video through its timed transcript.
• Silence detection — Finding the pauses between the words.
• Burned-in captions vs subtitles — What happens to the transcript when it becomes captions.
• Shot detection — The picture-side index; ASR indexes the sound.

Frequently Asked Questions

Made with Valmera: real exports

Edit from the Transcript

Upload footage and it is transcribed to the word before you type anything; then search it, cut it and caption it by describing what you want. Creating an account and uploading are free; editing requires a subscription.