How is a video editing agent different from an AI video generator?

A generator synthesizes footage that never existed. An editing agent works on footage you shot: it chooses which moments survive, in what order, at what speed and with what on top. Valmera is an editing agent; it does not generate a whole video from text.

What is the difference between an agent and an automated workflow?

A workflow runs a fixed sequence that someone wrote in advance. An agent decides the sequence at run time from what it perceives in your footage, then reacts to what it produced. The test is a request nobody anticipated, such as “cut the part where the dog barks over my pricing line, then use the second take of the intro.”

Does a video editing agent replace human editors?

It replaces the mechanical hours: removing dead air, cutting filler words, timing captions, reframing to 9:16, matching loudness and laying music under speech. Intent, story and taste still come from you, which is why Valmera’s agent works from your direction and shows you a preview.

Can Valmera’s agent learn my style?

Within a project, corrections carry forward. Valmera does not store brand kits yet, so keep a short style brief (caption style, music track, pacing, format) and paste it at the start of each project.

Is there an API for Valmera’s video editing agent?

Valmera exposes its editing tools as an MCP server at https://valmera.io/mcp/server. An external assistant such as Claude or ChatGPT becomes the planner and calls the tools directly, including export_final for the finished MP4. Delegating a request to Valmera’s own in-house agent is not available over MCP.

What can’t the agent do?

It does not generate whole videos, export SRT or VTT files, sync multicam footage, motion-track stickers, do true crossfades, denoise audio, generate music, use custom fonts, or publish to YouTube or TikTok. Choosing the emotionally right take is still best left to you.

Video Editing Agent

A video editing agent is an AI system that turns a plain-English brief into a finished edit by looping over five parts: a planner, a registry of editing tools, a state document (the edit decision list), a renderer and a verification step. It observes the result of its own edits and revises.

Valmera’s video editing agent is that system, built for footage you already have. It reads the transcript and the frames, edits, renders, looks at the result and revises. This page explains how it works inside. To make your own assistant the planner instead, see using an AI agent like Claude or ChatGPT to edit video.

Point the Agent at Your Footage

Upload a recording, write the brief, watch what it does. Creating an account and uploading are free; editing requires a subscription.

Create an account →

See pricing →

The Five Parts of a Video Editing Agent

Five components and the edge that makes them an agent. Delete the red arrow and what remains is an automated workflow: the same tools, run once, with nothing looking at the result.

1. Planner

Reads the current edit and the evidence, then picks the next tool call. It sequences dependent work itself: cuts before captions, so captions are timed to the edit, and music after both, so it fits the final length.

2. Tool registry

A fixed set of typed editing operations; its direct editing tools are the ones Valmera also exposes on its MCP server. If an operation is not in the registry, the agent says so and offers the closest option instead of improvising.

3. State document

An edit decision list. Every call writes a new version; the uploaded file is never modified, so any cut can be restored and any version reopened.

4. Renderer

Previews render quickly from a lightweight proxy so the loop can run often. The final export renders from the original file as a full-resolution H.264 MP4.

5. Verification

The agent watches its own render, audits captions and the audio mix, and checks its reply against the edits it actually recorded before reporting back.

Editing suits this shape unusually well: the state is small and explicit, every operation is reversible because the source is never touched, and the result can be rendered and looked at. The edit decision list glossary entry covers the state document in more depth.

What the Agent Perceives

Tool calling is the easy part. What decides the quality of an edit is what the agent can see before it calls anything, because “cut the bit where I fumble the pricing line” has one right answer: two timestamps that must be looked up, not reasoned about. Valmera indexes each upload once into aligned layers:

Five layers over the same minute, on one clock. An agent holding only the transcript cuts through a shot change; one holding only shots cuts a sentence in half. The alignment is the perception.

A word-level transcript with speaker labels, so cuts land on word edges and “keep the guest’s answers” can be located.
Shot boundaries, so a cut never leaves a stray frame of the previous shot.
Audio energy and silences, which separate emphasis from dead air.
Frames and burned-in text, which the agent looks at directly, including frames of its own rendered edit.

English is the best-tested transcription language. Because nearly every editing decision is anchored to the transcript, transcription quality affects the edit, not just the captions.

The Tools It Calls

The editing tools are published on Valmera’s MCP server card: 143 tools (live server card, October 2026), made up of 129 editing tools in nine groups plus 14 session tools for projects, uploads, jobs, exports and downloads. The editing groups:

Group	Tools	Examples
Reading the footage	20	`get_editorial_map` `search_transcript` `look_at`
Cutting	6	`cut_silences` `remove_filler_words` `restore_range`
Captions and on-screen text	12	`add_captions` `set_caption_style` `add_kinetic_text`
Framing and motion	17	`auto_reframe` `add_zoom` `set_speed`
Audio	20	`add_music` `beat_align_cuts` `set_master_loudness`
Colour and finishing	8	`set_color_grade` `apply_look` `set_transitions`
Media and screen capture	13	`insert_media` `add_overlay` `record_website_demo`
Repair and censoring	6	`blur_region` `erase_region`
Editing (orchestration)	27	`make_shorts` `research_broll` `render_preview`

Every tool, with what it does and refuses, is in the tool reference. External assistants call the direct editing tools over MCP; tools that delegate to Valmera’s own agent are not part of the MCP catalog.

Agent vs Workflow vs Preset

	Agent	Automated workflow	Preset
Decides what to do	The model, step by step	The author, in advance	The author, in advance
Reads your specific footage	Before every decision	Only what its steps sample	No
Handles a request nobody anticipated	Often	No	No
Reacts to its own output	Yes: that is the loop	No	No
Identical output every run	Not exactly	Yes	Yes
Best for	Briefs you can’t enumerate	High volume of the same job	One known operation

If you need a thousand identical outputs, a workflow is the right engineering choice. An agent earns its place on briefs you can’t enumerate in advance, which describes most of what an individual creator asks for. More on the category: what an agentic video editor is.

Clipping, Editing and Production Agents

	Clipping agent	Editing agent	Production agent
Input	One long recording	One recording and a brief	A brief, often no footage
Output	Many short clips	One finished cut	A whole generated video
Core problem	Selection: which 40 seconds	Execution across every part of a cut	Generation and assembly
Fails as	A clip without its context	A wrong cut	A clean video about nothing

Valmera is an editing agent that also has a clipping mode: in Studio it scouts a long recording for complete story arcs and gives each its own project to edit (make_shorts). That workflow is on the AI clipping agent page. Valmera is not a production agent and does not generate whole videos.

Further reading: VideoAgent: All-in-One Framework for Video Understanding and Editing, LAVE (Wang et al., ACM IUI 2024), and “It’s time for agentic video editing” (Justine Moore, a16z, January 21, 2026).

How to Work With Valmera’s Agent

1
Upload and let it index
Up to 100 GB and 3 hours per video. The transcript, shots, silences and frames are built once.
2
Give it the goal, not the steps
Describe the finished video. The agent decides the order of operations.
3
Let it check its own work
It renders, watches and audits the preview before replying.
4
Correct by pointing
“The second cut is too tight” is enough; it looks at its own output to find it.
5
Export
A full-resolution MP4 rendered from the original file.

Prompts That Use the Agent Well

“Before you cut anything, map the topics with timestamps and flag anything repeated.”
“After you add captions, check the preview for captions covering my face and fix them.”
“If two takes are equally good, ask me which one to keep.”

What Is Still Hard for Any Editing Agent

Choosing the best take. Finding every take is mechanical; picking the one with the right warmth is not. Name the take you prefer, or ask the agent to check with you.
Pacing. Where a moment should breathe is taste. Beat-aligned cuts help when there is music.
Story structure. If the piece is really about the third thing you mention, say so in the brief.
Memory across projects. No stored brand kits yet; paste your style brief each time.

Swap the Planner: Use Your Own Assistant

Because the planner is one of five parts, it can be replaced. Connect Claude, ChatGPT or Cursor to https://valmera.io/mcp/server and your assistant calls the editing tools itself: it uploads, edits, renders, watches the preview and, when you ask, exports the final MP4 with export_final. Editing one project from Studio and over MCP at the same time is refused, so changes never collide. The full guide is AI agent that edits video.

Limits

It edits footage you provide; it does not generate whole videos from text.
Captions are burned in, with no SRT or VTT import or export.
No true crossfades, motion-tracked stickers or custom font uploads.
No audio denoise and no AI music generation; music comes from your own file or a specific song you name.
No team seats, and no publishing to YouTube or TikTok.