How to make a video with an AI agent in ten steps
For AI agents: this page is the route. Do the steps in order; before each step read its guide (the
.mdlink in the table) and install its skill. Each step writes its files; at every STOP show them and wait for the person's yes.
For a whole video made properly by Claude Code, Codex or Grok, not one clip. Ten guides, one skill each, in the order that works: references, moodboard, script, sound, characters, previz, takes, edit, review, GitHub. Every rule is tested or traced to its source.
Rules
- Before step 1, ask once for what is missing: what the video is, who it is for, the length, the platform and aspect ratio, the brand (name, colours, type, logo), what the person loves and hates in videos like it, a spend cap and the paid models allowed. Then set up the project with step 10's skill, so every STOP is a saved version.
- One step at a time, in order. Before a step, read its guide and install its skill. Skip a step only when the person already has its output. When a skill names a different next step, this order wins; a video drawn entirely in code still takes every step, and step 7 sends its shots to code.
- Each step writes its files. At every STOP, show them and wait for the person's yes.
- Estimate the cost before every paid call and say the running total. Never go over the cap. Nothing before step 7 needs a video model: steps 2 and 5 make a few paid stills and step 4 paid voice and music, each after its estimate and a yes.
- Use video models only for what code cannot draw: real people, crowds, weather, physical motion. Text, numbers, UI, charts and logos are built in code.
- Sound is locked before picture (step 4), and the picture is cut to it.
- Measure, do not grade yourself (step 9). Between close takes a person picks.
- Generation runs on varg with one account: the varg-ai skill (npx skills add vargHQ/skills --skill varg-ai) or the varg MCP server https://mcp.varg.ai/mcp.
The steps
| # | Step | Writes | Skill | Guide |
|---|---|---|---|---|
| 1 | References (STOP) | refs/refs.md, refs/never.md, refs/links.md, refs/board.html | find-references | https://varg.ai/how-to/find-references-for-an-ai-video.md |
| 2 | Moodboard (STOP) | look/moodboard.md, look/board.html | video-moodboard | https://varg.ai/how-to/make-a-moodboard-for-an-ai-video.md |
| 3 | Script (STOP) | script.md, STORYBOARD.md | video-script | https://varg.ai/how-to/write-a-video-script-with-an-agent.md |
| 4 | Sound and voice (STOP) | sound/plan.md, cues.json, mix.wav, beats.json | video-sound | https://varg.ai/how-to/design-sound-and-voice-for-ai-video.md |
| 5 | Characters and props (STOP) | cast/*.md, character and prop sheets | character-passport | https://varg.ai/how-to/design-characters-and-props-for-ai-video.md |
| 6 | Previz in code (STOP) | staging.json, blockout stills, previz.mp4 | code-to-video-previz | https://varg.ai/how-to/previz-a-video-in-threejs-or-blender.md |
| 7 | Animate (STOP) | takes/*.mp4 with one sidecar per take | video-animate | https://varg.ai/how-to/animate-a-previz-with-ai-video-models.md |
| 8 | Edit | cut.mp4, notes.md | edit-footage-with-code | https://varg.ai/how-to/edit-a-video-with-hyperframes-and-agents.md |
| 9 | Judge (STOP) | review.md | media-judge | https://varg.ai/how-to/judge-an-ai-video-without-bias.md |
| 10 | Keep it in GitHub | AGENTS.md, media.lock.jsonl | video-project-github | https://varg.ai/how-to/keep-a-video-project-in-github.md |
Install a step's skill: npx skills add vargHQ/skills --skill <skill>. All at once: npx skills add vargHQ/skills --skill find-references --skill video-moodboard --skill video-script --skill video-sound --skill character-passport --skill code-to-video-previz --skill video-animate --skill edit-footage-with-code --skill media-judge --skill video-project-github.
What to do at each step
1. References (STOP: the person approves)
Write the brief, then one query pack per job the piece needs: light gets the most, then framing, people, location, texture, palette, type, motion, music. Search each source in its own words, open everything you keep, prove who made it, and show three candidates per job. Turn what the obvious search repeats into never.md. Sites that forbid bots go into links.md for the person to open. The person picks the references.
- Writes: refs/refs.md, refs/never.md, refs/links.md, refs/board.html
- Skill:
find-references(https://varg.ai/how-to/skills/find-references.md) - Guide: How to find references for an AI video
- Measured: Openverse "old fisherman face weathered" returned 0, and LoC "fisherman" returned 243
- Measured: Shown the same image twice, a naive judge picked the first image 10 of 10
- Measured: A mean of 24 vision models scored 41.6% on lighting condition (ShotBench)
2. Moodboard (STOP: the person approves)
Write the board in ranked order: references, light, subject lock, framing, palette as hex taken from a reference, motion, texture, type, sound, and a Never list. Every line is a number or a named thing, and every hex comes from a picked reference, never from memory. Prove the board on three test stills before anything else is generated.
- Writes: look/moodboard.md, look/board.html
- Skill:
video-moodboard(https://varg.ai/how-to/skills/video-moodboard.md) - Guide: How to make a moodboard for an AI video
- Measured: A full board made frames 1.0 point more film-like (on 1–10) than no board
- Measured: Light was the only line that raised film-likeness: +0.50 (p 0.04)
- Measured: Posts built on "cinematic", "8k" or "stunning" got 31% fewer saves
3. Script (STOP: the person approves)
One promise, ten hooks to choose from, beats joined by BUT or THEREFORE, one spoken line per visual, every line sized to its seconds (about 2.5 words a second) and checked with the skill's check_script.py. Then the storyboard: timing and camera for every shot.
- Writes: script.md, STORYBOARD.md
- Skill:
video-script(https://varg.ai/how-to/skills/video-script.md) - Guide: How to write a video script with an AI agent
- Measured: Scripts written with the guide scored 7.54 out of 10 against 6.40 without it
- Measured: Without the guide, 5 of 12 scripts had lines too long to say in their slot; with it, none
- Measured: Reversing the reading order changed the 1–10 judge's winner in 2 of 3 briefs
4. Sound and voice (STOP: the person approves)
Lock the sound before any picture: a sound plan, two or three voice candidates to pick from, lines written with pauses and breaths, music mapped to bars, the few effects that matter, and a mix measured by script. beats.json holds the time of every cut, hit and word; the picture is cut to it.
- Writes: sound/plan.md, cues.json, mix.wav, beats.json
- Skill:
video-sound(https://varg.ai/how-to/skills/video-sound.md) - Guide: How to design sound and voice for an AI video
- Measured: ~85% of hard cuts in pro launch films get no sound of their own
- Measured: 231 → 194 words a minute: bare text vs breath-tagged on Eleven v4
- Measured: 0 of 3 music takes made the planned silence and hit
5. Characters and props (STOP: the person approves)
For every person, creature, product and prop: one passport paragraph that is pasted word for word into every later prompt, a turnaround sheet in the moodboard's look, six expressions written as a cause plus physical tells, and a measured voice line. Props get materials, wear and how they move.
- Writes: cast/*.md, character and prop sheets
- Skill:
character-passport(https://varg.ai/how-to/skills/character-passport.md) - Guide: How to design characters and props for AI video
- Measured: Same person across 4 shots: 2 → 8.5 out of 10 with passport + sheet
- Measured: Expressions that read as real: 5 → 7 with cause + physical tells
- Measured: Voice pace vs a 130 wpm target: 92 → 129 wpm after one fix
6. Previz in code (STOP: the person approves)
Block every shot in code (Three.js by default, Blender when the guide says so): geometry, camera and timing from the storyboard and beats.json. Render one still next to its reference and close the gaps, three rounds, before any motion. The previz owns camera, blocking and timing from here on.
- Writes: staging.json, blockout stills, previz.mp4
- Skill:
code-to-video-previz(https://varg.ai/how-to/skills/code-to-video-previz.md) - Guide: How to previz a video in Three.js or Blender
- Measured: In our own film, 2 of 7 measured camera moves came back reversed
- Measured: Guessing the camera and positions put key points 9 to 22% of the frame off
- Measured: Lit and wrong vs grey and right: the LLM judge picked lit 6 of 6
7. Animate (STOP: the person approves)
Route each element, not each shot: code draws what must be exact (text, numbers, UI, logos), the video model does what code cannot (faces, cloth, weather, crowds). A cheap 480p pass first; try timing and camera variants in the previz, then roll takes that vary only the performance. STOP before the first paid video call, with the estimate.
- Writes: takes/*.mp4 with one sidecar per take
- Skill:
video-animate(https://varg.ai/how-to/skills/video-animate.md) - Guide: How to animate a previz with AI video models
- Measured: Half-finished 3D leaked CG into 10/15 shots, colour-coded blocks into 0/15
- Measured: Draft mode then final stays .95-.99 correlated; a fresh 1080p render .257
- Measured: 30 s clips reached the screen at 32.2%, 5 s clips at 3.3%
8. Edit
Cut the speech first, lock the sound, then add picture, graphics last. Assemble on beats.json. The person gives notes with timestamps and you fix exactly those; an approved look or fix is saved as a reusable command so it stays the same.
- Writes: cut.mp4, notes.md
- Skill:
edit-footage-with-code(https://varg.ai/how-to/skills/edit-footage-with-code.md) - Guide: How to edit a video with HyperFrames and AI agents
- Measured: The best agent passed 15 of 56 real editing tasks; editors preferred the human edit 83.5%
- Measured: Curators and bait hold 43.5% of bookmarks. Working editors hold 4.9%
- Measured: All five test cuts fixed 7 of 7 planted defects, with or without the skill
9. Judge (STOP: the person approves)
Measure, do not grade yourself. Code reads every frame and writes a time, a number and a picture for each flag (duration, black or frozen frames, seams, sync, loudness, spelling on screen). A fresh agent reads only the flags. Takes rank on counted defects, and a person picks between close ones.
- Writes: review.md
- Skill:
media-judge(https://varg.ai/how-to/skills/media-judge.md) - Guide: How to judge an AI video: measure, don't grade
- Measured: Claude Opus caught 17.5% of temporal glitches. Best of 12 models: 54.1% accuracy
- Measured: An LLM judge picked whichever take it saw second 75% of the time
- Measured: 18 of 22 planted defects caught on footage it never saw. 6 of 8 clean controls passed
10. Keep it in GitHub
Set this up before step 1 and use it at every STOP: a private GitHub repo, text in git, media listed by hash with its bytes in Releases, keys in .env behind a commit hook. Save a version at every STOP so anything can be undone, and prove the backup restores byte for byte.
- Writes: AGENTS.md, media.lock.jsonl
- Skill:
video-project-github(https://varg.ai/how-to/skills/video-project-github.md) - Guide: How to keep an AI video project in GitHub
- Measured: A smaller model pushed live-looking keys to GitHub in 3 of 3 unaided runs
- Measured: In 96 runs, folder layout made no measurable difference to agents
- Measured: Criteria with an allowed tie gave the same answer under a nudge 82-90% of the time
Evidence
7,605 X posts read, 4,156 authors behind them, 1,707 pieces of advice kept, each with its quote, 1,470 varg how-to cards checked. Every rule with the post or test it came from:
- Playbook (Markdown): https://varg.ai/how-to/guide-pages/evidence-bank/data/PLAYBOOK.md
- Advice with quotes and numbers (JSONL): https://varg.ai/how-to/guide-pages/evidence-bank/data/advice.jsonl
- Data points (JSON): https://varg.ai/how-to/guide-pages/evidence-bank/data/data-points.json
- The page for people: https://varg.ai/how-to/guide-pages/evidence-bank/
The line a person pastes to start this route:
Make my video in ten steps. Read https://varg.ai/how-to/make-a-video-in-ten-steps.md and follow it: one step at a time, stop for my yes at every STOP, and tell me the cost before anything paid.
The video: [what it is, who it is for, how long, where it will be shown]
Run the generation on varg
varg is one API key for the models these skills drive — Seedance 2.5, Kling 4, Veo 3.1, Wan 3, Nano Banana 2.1, GPT Image 2.5, ElevenLabs v4 and 200+ more — billed in credits, with a cost estimate before anything runs.
- Any agent with a terminal:
curl -fsSL https://varg.ai/install | sh(Windows:irm https://varg.ai/install.ps1 | iex), thenvargai loginand every model is a command —vargai generate_video …,vargai estimate_cost …(https://varg.ai/how-to/cli/) - Claude Code:
claude plugin marketplace add vargHQ/skills && claude plugin install varg@varg(skill + hosted MCP) - Codex, Cursor, Gemini CLI, OpenClaw:
npx skills add vargHQ/skills --skill varg-ai -a <agent> -y - Grok and any agent that reads SKILL.md:
npx skills add vargHQ/skills --skill varg-ai, or point it at the skill's SKILL.md - Claude.ai, ChatGPT, any MCP client: connect
https://mcp.varg.ai/mcp - API key: https://varg.ai/dashboard/api · Model specs: https://varg.ai/models/llms.txt
For AI agents: find the right skill in one call —
curl -s "https://varg.ai/how-to/api/search?q=<what the person wants>"returns the matching how-to page and the best skills with install commands, as markdown (Russian works too). Full index: https://varg.ai/how-to/llms.txt. Every page has a markdown twin: add.mdto its path. Before generating anything, show the person the options on the how-to page. From a terminal:curl -fsSL https://varg.ai/install | shinstalls the vargai CLI (https://varg.ai/how-to/cli.md):vargai searchandvargai install <slug>with no key, every varg model aftervargai login.