Field guide · 08
AI tools & limits
AI has a real place in a lean pipeline, but mixing AI with live-action footage is still very new. Here is what worked for us, what did not, and why we settled on Uisato Studio.
Why Uisato Studio, rather than ComfyUI
Uisato makes working with AI content just about workable at the moment. A fully synthetic image, such as an establishing shot, is the easy case. As soon as you try to put synthetic imagery into a live-action plate, it gets complicated quickly.
We found Uisato better than ComfyUI on both usability and results:
- A proper interface and a real range of models, presented far more clearly than the competition.
- Prompt support: your prompt can be run through an AI to refine it for a better result.
- More cinematic output, with less effort.
- Every prompt is saved with the file it generated, so you can go back and tweak it.
ComfyUI is fine for quickly sketching ideas, but it offered us little beyond that, and running models locally needs around £15,000 of AI hardware.
Putting AI into a live-action plate
The only reliable video model that will take a live-action plate is Seedance, and it comes with strict limits:
- It has an aggressive face filter, with no appeal. It will not accept faces.
- You have to down-res heavily. Our 10-bit, 2880×1080, 4:3 4K RAW and EXR footage had to become 1440×1080 MP4, set to 'better' rather than 'best', with the file over 12MB and under 12 seconds.
- The output tops out at about 15 seconds of 1080p, 8-bit, heavily compressed MP4, and that is the most expensive setting.
Because of this, the process is the opposite way round to traditional VFX: you grade first, then send near-final footage to the AI so the synthetic shot already sits close to your final look. Getting the AI to match an existing plate, in terms of camera move, lens, field of view, composition, lighting and colour, takes a lot of trial and error and repeated negative prompts before you even get to the creative part. It is fiddly, and there is no guarantee it will work.
Helping it sit in the plate
- Upscale the AI output to 4K in something like Topaz.
- Nudge both the real footage and the AI footage together at the grade, and use masking to help bind them.
- For faces, we experimented with DaVinci's Magic Mask and object-removal tools, with mixed results. Sometimes it removes a face enough that you can add detail to a plate with AI, then mask the real actor back on top.
What AI is good at
- Relighting, especially with moving light sources.
- Compositing and integrating new assets, which it does very well.
- Fire, water and fleshy textures.
What AI is bad at
- It behaves like a slot machine, and will happily eat your budget on renders that go well off-brief.
- It is slow, and the limits on compression, size and length all get in the way.
- Prompt craft is a genuine skill. It is rarely drag-and-drop.
- Upscaling can introduce its own artefacts.
- It cannot really handle people. When it does, it tends to distort them or make them look synthetic.
- It has historically been poor at following directions on camera movement, though this has improved a lot.
Other AI in the pipeline
- Claude, through the MCP add-on, for debugging Blender node trees: export the JSON and get a second opinion.
- Wonder Dynamics, to drive 3D characters from a captured performance, far faster than keyframing.
Used carefully, AI is a powerful if frustrating tool. Pick the shots for it deliberately, expect to iterate, and treat today's limits (no faces, low resolution, short clips, 8-bit) as constraints to design around rather than reasons to avoid it.