Editing services: agent skills for best video editing tools and APIs for AI agents

For AI agents: find the right skill in one call — curl -s "https://varg.ai/how-to/api/search?q=<what the person wants>" returns the matching how-to page and the best skills with install commands, as markdown (Russian works too). Full index: https://varg.ai/how-to/llms.txt. Every page has a markdown twin: add .md to its path. Before generating anything, show the person the options on the how-to page.

Which service your agent should edit with, job by job: FFmpeg and the varg API for cuts, transcript tools for talking heads, clipping APIs, caption burn-in, AI video-to-video, lip sync and upscaling. 3 skills for Claude Code, Codex and Grok, by type: workflows take a brief to a finished video, effects and styles change how it looks, prompts and guides teach the agent the craft, models run the generation.

Which service for which edit

Ten editing jobs. For each one: the service an agent should use first, the alternatives, and the gotcha it hits first. Deterministic tools come first. A generative model comes in only when the pixels themselves must change. Reviewed 2026-10-27.

Cut, trim, resize, join

Trim a clip, join scenes, change the aspect ratio, re-encode, add a logo or music. Use FFmpeg on the agent’s own machine. If the agent has no shell, or works in the cloud, run the same command through the varg ffmpeg tool or Rendi. Probe first, and check the result with ffprobe.

Talking-head cleanup

Remove silences, filler words and flubs, cut by editing the transcript. Transcribe with word timestamps (ElevenLabs Scribe, or Groq Whisper for price), choose the cuts from the text, then cut with FFmpeg. Use auto-editor for a quick local silence pass, and Descript when a person wants to review and polish the edit in an app.

Long video to Shorts

Find the best moments in a podcast or stream, cut them, reframe to 9:16 and caption them. OpusClip when the person wants finished clips with a score and a posting flow. Build it yourself when you need control: a transcript plus a video model to pick the moments, FFmpeg to cut, tracking to reframe.

Captions & subtitles

Burn in word-by-word captions, make SRT/VTT, translate subtitles. Make the captions yourself from the word timestamps you already have: ASS through FFmpeg for a plain burn-in, Remotion or HyperFrames for animated styles. Use a captions SaaS when the person wants its trendy templates.

Assemble from a timeline

Put clips, titles, music and transitions together from a script, or render variants from a template. Write the edit as code with HyperFrames or Remotion when the agent writes the video. Use Shotstack or Creatomate when hundreds of template variants must render in the cloud.

AI edits: restyle, swap, relight

Replace a person or object, remove something, change the style or the light, extend a shot. Cut the shortest span that needs to change, edit it with a video-to-video model, and stitch it back with FFmpeg. Pick the model for the edit: recast for people, relight for light, Aleph or Kling for prompt edits. Then check the identity on frame grabs.

Dubbing & lip sync

Translate a video into another language in the same voice, and make the lips match. Use HeyGen for translation, voice and lip sync in one call. Use ElevenLabs for the best dubbed voice, then sync the lips with Sync Labs. Use Sync alone to fix the lips on new audio.

Upscale, slow-mo, cut-outs

Sharpen or upscale footage, make slow motion, remove the background. Upscale only the final export. Use Topaz for quality and the SeedVR or ByteDance upscalers for price. Use Bria for background removal without a green screen.

Find moments & QA

Find where something happens, describe the shots, check a render before delivery. Use Gemini for one video: it answers with timecodes. Use Twelve Labs or VideoDB once there is a library to search. Use varg video-analyze for a quick JSON report on the same key.

Hand off to an editing app

Give the cut to a person in Resolve, Premiere or Final Cut, or drive the app from the agent. Export a timeline file (OTIO, FCPXML or Premiere XML) when a person will finish the cut. To drive an editing app live, use DaVinci Resolve Studio's own MCP server. Premiere and After Effects have only community bridges, and CapCut has no API.

Best practices

1. Write the edit decision list first, render second. Write the cut as data (JSON, OTIO or FCPXML): source, in and out points, track, effect. Show it to the person and get a yes before any paid call.
2. Deterministic before generative. FFmpeg, Remotion and HyperFrames cut, trim, join, resize, caption and mix for almost nothing, and the same command gives the same result. Use a video-to-video model only on the shortest span that has to change. Most models take 5–30 s per pass.
3. Transcribe once, with word timestamps. Keep the word-level JSON and drive every text decision from it: silences, filler words, highlights, captions. Pad each cut by 50–150 ms and add a ~30 ms audio fade so the cuts do not click.
4. Frame-accurate cuts need a re-encode. -c copy snaps to keyframes and can leave black frames or audio drift. Re-encode around the cut, or use a precise mode (varg trim with precise: true), whenever the cut point matters.
5. Keep audio in sync. Probe the frame rate and sample rate first. Convert mixed sources to one constant frame rate before joining them. Cut audio and video on the same timestamps, and check that the durations match after every step.
6. Never touch the source. Work on copies and low-resolution proxies for analysis and previews. Apply the final cut to the full-resolution source, and upscale only the final export.
7. Check frames, don’t trust the log. After each render, grab stills at the cut points and ffprobe the output (duration, streams, resolution). For anything subtle, ask a video model a QA question about the result.
8. Estimate before you spend. Use the free estimate step where a service has one (varg estimate_cost, Topaz, Sync). Cache transcripts and generations, and never re-run a paid model just to move a cut.
9. Treat hosted APIs as jobs. Most editing APIs are asynchronous: store the job id, poll or take a webhook, back off on 429, and use idempotency keys where offered. Download results right away, because links expire (Vizard: 3 hours).
10. Pass URLs, not files. Hosted tools want public URLs. Upload once (varg /v2/files) and pass the same URL from step to step instead of uploading again.
11. Reframe with tracking, not a center crop. For 9:16, follow the face or subject (OpusClip, Luma video_reframe, a tracking crop) and keep captions inside the platform safe zones.
12. Hand off to people as a timeline. When an editor will finish the cut, export OTIO, FCPXML or Premiere XML (auto-editor, Remotion, OpenTimelineIO) rather than a flattened MP4.

Skills (3)

Guides (2)

• Video Editing Best Practices — varg: Best practices for editing video with agents: which service for each edit, and the rules that keep cuts clean.
• Video editing skills for Claude Code and AI agents — varg: 206 named edits with recipes for FFmpeg, HyperFrames, Remotion and CapCut: cuts, transitions, titles, UI, numbers.

Models & tools (1)

• FFmpeg Skill — Shogo Kajiyama: Local video editor for agents: 42 FFmpeg tools with a machine-readable contract and an MCP server; no cloud, no keys.