Text-Based Video Editing
Text-based video editing is a workflow in which a recording's transcript is the editing interface. The audio is transcribed with word-level timestamps, so every word in the text is bound to a range of time in the source media; deleting, reordering or searching the text then performs the equivalent operation on the footage. Delete a sentence in the transcript and the frames it occupies are dropped from the program. Nothing is re-encoded while you edit — the result is recorded as a list of surviving source ranges and conformed from the original file at export. Its reach is exactly the reach of the transcript: it addresses what was said, never what was shown, which is why every implementation still hands you a timeline for the rest of the edit.
How the Mapping Actually Works
The whole workflow rests on one data structure: a two-way index between positions in a text document and time ranges in a media file. Everything else is interface. Four stages build and use it.
1. Transcribe with per-word timing
A transcript alone is not enough — a subtitle file gives you a sentence per cue, and a sentence is far too coarse to cut with. What the workflow needs is word-level timestamps: a start and end for each individual word. Speech models produce these on a coarse grid. Whisper-family front ends compute a log-mel spectrogram over a 25 ms window every 10 ms — 100 frames per second — and a stride-2 convolution halves that to 50 encoder positions per second, so the native resolution is 20 ms. Any timestamp printed finer than that is arithmetic, not measurement.
2. Bind text offsets to source time
The editor holds the transcript as a list of word records, each carrying its character span in the document and its time span in the source. That gives you both directions for free: a text selection resolves to a time range, and a playhead position resolves to a word to highlight. The document you are reading is a view of a timed index, not a file in its own right.
words: [
{ "text": "we", "start": 12.437, "end": 12.556 },
{ "text": "shipped", "start": 12.556, "end": 12.921 },
{ "text": "it", "start": 12.921, "end": 13.024 },
{ "text": "on", "start": 13.024, "end": 13.160 }, <- gap of 352 ms follows
{ "text": "Friday", "start": 13.512, "end": 14.005 }
]Two details in that snippet decide the quality of every edit made from it. Inside the phrase, each word's end is exactly the next word's start — not out of confidence, but because continuous speech has no silence between "shipped" and "it" to measure. And the 352 ms before "Friday" belongs to neither word: it is a breath, and it is the most useful number on the page, because that is where a cut can hide.
3. Resolve a deletion into surviving ranges
Deleting text does not delete anything. It marks words as dropped, and the editor recomputes the keep list: the ordered set of source spans that survive. Delete "it on" above and the program becomes [0 → 12.921] + [13.512 → end]. Note that the second range starts at 13.512 rather than 13.160 — a well-built editor extends the cut through the following silence, because leaving 352 ms of dead air where two words used to be is audible, and a listener reads it as a gap rather than as an edit.
The edges also have to be snapped. Cutting inside a word truncates a phoneme: a word-final /t/ or /k/ is a silent closure followed by a release burst, and cutting between the two ends the word on a hole that reads as a swallowed syllable. A fricative like /s/ runs 100–150 ms and clips into an obvious hiss. No crossfade repairs this, because the information was never in the range you kept. So every edge moves outward to the nearest word boundary before anything is recorded.
4. Conform the picture at render
The keep list is written to an edit decision list in source seconds and nothing is re-encoded while you work — which is why deleting a paragraph is instant on a two-hour recording. At export, source seconds are quantised to whole frames: an audio cut computed at 12.921 s becomes frame 388 at 30 fps, or 12.933 s, a 12 ms shift the sound does not take unless you make it. Sound and picture must be quantised together at the same edit point, or every cut drifts by a fraction of a frame.
One consequence is worth stating plainly, because it explains why professional systems anchor decisions in source time rather than output time. Deleting a sentence at minute 3 shifts every subsequent moment in the finished video earlier. If a caption, a zoom or a music cue had been stored at "output second 400", it now points at the wrong content. Stored against the source, it still points at the same words.
The Lineage: Research, Descript, Everyone Else
The idea is much older than the product that popularised it. Human-computer interaction researchers had working transcript-driven video editors two decades ago — Carnegie Mellon's Silver editor (2002) aligned a transcript to footage so the text could drive the cuts, and Berkeley and Adobe Research published a line of tools on the same premise, notably Content-Based Tools for Editing Audio Stories (UIST 2013) and QuickCut (UIST 2016). All of them argued the same thing: for speech-driven material, the transcript is a better index than a waveform, because you can read it.
Descript turned that into a mainstream product in 2017 by inverting the application — the document is the editor and the timeline is secondary, rather than the other way round. That inversion, not the transcription, is what made it feel new to people who had never read a UIST paper.
Everyone else added it as a panel. Adobe Premiere Pro shipped Text-Based Editing after a beta announced at Adobe MAX in October 2022, reaching general availability in May 2023, working from the speech-to-text transcript Premiere already produced for captions, and later gaining filler-word and pause detection. DaVinci Resolve added transcript-driven editing in 18.5 (2023). Browser editors including CapCut, Veed and Kapwing offer transcript trimming at varying depth. In every one of these, the transcript is a second view onto a timeline that remains the real document.
A separate research branch deserves a name because it is routinely confused with this one. Text-Based Editing of Talking-Head Video (Fried et al., SIGGRAPH 2019) let you type a word the speaker never said and synthesised the face and mouth to match. That is generative video, not an index into a recording. Editing a transcript to remove what was said and editing a transcript to fabricate what was not are opposite operations that happen to share an interface.
Where It Runs Out
A transcript is a lossy projection of a video. Speech at a conversational 120–150 words per minute is a few hundred bits a second of information; the picture is orders of magnitude more. Anything that did not survive the projection cannot be addressed in the projection. Four limits follow, and none of them are implementation gaps that a better version will close.
It can only see speech
Silent b-roll, a music bed, a screen recording with no narration, a reaction shot, a long look to camera: none of these produce text, so none of them can be selected. A 12-second establishing shot is a blank in the document. This is also why silence detection is a separate index rather than a transcript feature — the gaps between words are exactly the part of the recording the transcript does not describe.
It can only express what, never how
"Punch in on the word revenue", "hold this beat two seconds longer", "grade the interview warmer than the b-roll", "drop the music under this line" — none of these are deletions, insertions or reorderings of text. A document of words has no vocabulary for them. Every text-based editor therefore keeps a timeline and a properties panel, and the moment your edit is about craft rather than content you are back in it.
One caret cannot cut sound and picture separately
A text selection is a single range, so the audio edit and the picture edit necessarily land at the same instant. That rules out the most basic polish in the craft: J-cuts and L-cuts, where sound leads or trails the picture by a beat to smooth the join. Text-based editing produces straight cuts by construction.
A word boundary is safe, not good
Snapping to word boundaries protects the audio, but inside a phrase the boundary is notional rather than acoustic — speakers coarticulate, so the end of "shipped" is already shaping the vowel of "it". A splice there is clean at the sample level and still wrong at the sentence level, because it joins two different pitch contours mid-breath. The listener hears the prosody jump even though nothing is damaged. Genuinely invisible cuts land in pauses, which the transcript does not mark and you have to look for.
And the picture still jumps
Removing a sentence from a static talking-head shot leaves the speaker in a visibly different position on either side of the join — the definition of a jump cut. The text edit was one keystroke; covering it with b-roll, a punch-in or a cutaway is a separate piece of work the document view cannot do for you. Text-based editing makes the cut cheap and leaves the consequence at full price.
Why It Matters in Practice
Take a 90-minute recorded webinar that has to become an 11-minute highlight cut. At a conversational 140 words per minute that recording contains roughly 12,500 words. Reviewing it as video costs 90 minutes at 1x, and scrubbing does not help, because the waveform of a good answer and the waveform of a tangent are the same shape. Reviewing it as text costs a little under an hour at the ~240 words per minute a meta-analysis of reading studies puts silent non-fiction reading at — and, more importantly, text is randomly addressable. You skim, jump, search a term, and read the two paragraphs around a hit without playing anything. That is the entire value proposition of the workflow, and it is a real one.
So you select the keepers, delete the rest, and the 11 minutes exist in about an hour. Then you watch it. The picture jumps at all forty joins, because you cut a locked-off shot forty times. Two of your keepers came from opposite ends of the session and splice a confident answer onto a tired one. The segment you moved earlier now references a slide the viewer has not seen. None of these are transcription errors. They are the parts of the edit that were never in the text, arriving all at once at the moment you thought you were finished.
The honest summary of the workflow is that it collapses the selection problem and leaves the construction problem untouched. Deciding what survives goes from ninety minutes to fifty; making the survivors play as one video costs exactly what it always did.
Common Misconceptions
"Text-based editing is AI video editing."
It uses AI for exactly one job — producing the transcript — and after that it is a manual editor with a document-shaped interface. You read every line, decide every deletion, and perform every edit. Nothing chooses anything for you. The two terms describe different parts of the stack, and a tool can be one without being the other.
"Editing the transcript edits the video."
Only deletion, and in some tools reordering, are bound to the media. Typing a new word does not make anyone say it; correcting a misheard name fixes the text and the captions, not the audio. This confuses people constantly because the interface is a text editor and text editors accept typing. The bound operations are the ones that map to a range of existing footage.
"It is non-destructive, so any cut is reversible."
The media is untouched — the edit is a keep list over the original file, and restoring a range is a change to a document. But an edit is also a decision, and a hundred small deletions accumulate into a structure you can no longer reason about by reading. Reversible is a property of the file, not of your understanding of the edit.
"Deleting a word deletes exactly that word."
It deletes a range that has been snapped outward to word boundaries and usually extended through adjacent silence, then rounded to a frame at export. The gap you remove is normally longer than the word. That is correct behaviour, not a bug — but it explains why a cut can feel a beat different from what the text implied.
"The timestamps are exact."
They are estimates with an error distribution. A strong forced aligner reports mean word-boundary errors in the region of 22–28 ms, with only about two thirds of boundaries inside a 20 ms tolerance. Timestamps derived from a model's attention have a systematic failure too: they tend to absorb a following pause into the preceding word, so a reported end can include several hundred milliseconds of silence. If your cuts feel consistently late, this is usually why.
"It replaces the timeline."
No shipped tool does. Descript keeps one; Premiere and Resolve present the transcript as a panel beside theirs. The transcript is the fastest way to decide which content survives, and it says nothing about pacing, framing, sound or look — so the timeline is where the rest of the edit still happens.
How This Works in Valmera
Valmera is an agentic video editor, and it takes the index from text-based editing without taking the interface. Every upload is analysed once into a word-accurate transcript with word-level timings, plus silence detection, shot detection and labelled frame tiles the agent reads directly. Every later edit resolves against that index; all cuts snap to word boundaries; transcript search resolves a phrase to a real source range rather than a scrubbed guess.
The difference is who operates it. In a transcript-deletion editor you select the words to remove. In Valmera you state the outcome — "cut the dead air and every um, then punch in when I say the price" — and the agent performs it, renders a preview, and looks at the frames it produced. That covers the two limits above that hurt most in practice: an instruction scales the same whether the recording is four minutes or two hours, and it can express how as well as what, because zooms, grades, music and framing are tools the agent holds rather than characters in a document. Decisions are written to an edit decision list in source seconds, so a cut early in the video never invalidates a caption or a zoom placed after it, and the final render always comes from the original file at source quality.
The honest limits. Valmera has no document view whose deletions cut the footage — if what you want is to delete words in a transcript by hand, Descript is the tool built for that, and our side-by-side says so. Valmera's transcript panel is editable for corrections only: fix a misheard name and the captions re-render. Captions are burned into the video, with no SRT or VTT import or export. English is the best-tested transcription path. And the timings are model estimates carrying the boundary error described above — when a cut lands a beat late, the fix is to say so and have it moved, not to assume the number was ground truth.
Related Terms
Other entries in the Valmera video editing glossary that touch this one:
- Word-level timestamps — The unit the whole workflow is built on. No per-word timing, no text-based editing.
- Transcription and ASR — Where the text comes from, and the error rate you inherit with it.
- Edit decision list — What a transcript deletion actually writes: source ranges, not pixels.
- Silence detection — The other index. Words say what survives; silence says where the join can hide.
- Jump cut — What a clean transcript edit looks like on the picture side when nothing covers it.
- B-roll — The standard fix for the visual discontinuity a text edit creates.
- J-cut and L-cut — Sound and picture cut at different moments — a thing a single text caret cannot express.
- Burned-in captions vs subtitles — The same transcript, used for display instead of for cutting.
Frequently Asked Questions
Skip the Deleting
Valmera indexes your footage word by word, then edits it from a sentence you type. 50 free credits, no card.
Start free →