Filming the launch trailer from the game itself, frame by frame

The launch film of Endurance Racing is 47 seconds long, 4K at 60 frames per second, and every frame of it comes from the game. There is no screen recording and no mock-up. The point of doing it this way is that nothing in the film can promise something the game does not do, and that I can film any moment at any speed, however slow the GPU is.

This post is how the rig works. It lives in the repository under tools/video.

A clock I can stop #

A screen recording captures whatever frame the computer managed to draw. I wanted the opposite: every frame drawn at the exact simulation time I ask for, however long it takes. So the page gets a virtual clock, injected before any of the game's scripts run:

// trimmed from tools/video/lib/vclock.js
const realNow = performance.now.bind( performance );
let frozen = null;
performance.now = () => ( frozen === null ? realNow() : frozen );

window.__vt = {
  freeze() { if ( frozen === null ) frozen = realNow(); return frozen; },
  advance( ms ) { if ( frozen !== null ) frozen += ms; return frozen; },
  live() { frozen = null; },
};

While the clock is frozen, performance.now() only moves when the rig advances it. Three's timer, and so the rain, the ripples, the sky and every TSL time node, sees exactly one step per video frame. Slow motion is simply a smaller step. The rig renders a frame, takes a screenshot, advances the clock and repeats.

A pilot and a camera crew #

The car is driven by a pilot that uses the game's own input. It follows the racing line with pure pursuit, and it has scripts for a drift, for the nitro and for launch control. The camera is a separate rig with modes (the game's chase camera, a camera that stands beside the track, a free-flying one) and time curves.

The best trick is how a trackside camera knows where to stand. The simulation is deterministic, so the rig first runs the shot once as a recording pre-pass. It then places the camera where the car will be at that moment of the real take, and the shot is framed before the car arrives. A shot is described by a few lines: the look of the sky, the car, the position on the lap, the speed, and the camera.

Frames to video #

The capture runs in headless Chrome against a build that keeps window.__app for the rig. Each frame is a JPEG at quality 98, 3840 by 2160 (a 1920 by 1080 page at a pixel ratio of 2), piped into x264, one file per shot. All shots render with one page per car, three Chromes in parallel, in about 15 minutes on an M2 Max.

Then ffmpeg does the rest: the cut, the grade, the letterbox, the sound and the end card, all in a script, so changing a shot or a timing means running it again.

Cutting to the music #

The soundtrack is Vivaldi's Summer III (the Presto), taken from the game's own radio. The film uses the last 47.1 seconds of the track, and the edit is built around where the music goes:

• the quiet strings are a slow-motion intro,
• the storm breaks at 6.43 seconds into the film, and that is where the black bars open,
• the lights go out at 36.58 seconds, on a big accent of the music,
• the end card appears at 43.37 seconds and the last chord lands at 45.78.

The film has 22 shots. From the drop to about 20 seconds in, the cuts fall on a grid of 0.441 seconds, the pulse of the music counted from the drop. The picture has a 2.39:1 letterbox that opens on the drop, and white flashes on the hits. The sound design is sparse: a rain bed under the intro, a riser into the drop, deep booms on the drop, the lights-out and the end card, whooshes when the car passes the camera, tire squeal under the drifts, and the game's own nitro and engine samples. The mix is normalized to -14 LUFS with a true peak under -1.5 dB.

Things that bit #

• One page per car. Showing car A and then car B in the same page, not counting the first swap from the default car, made three reuse a render pipeline that had been built for A's vertex layout. The error was Vertex buffer slot 3 required by [RenderPipeline "..."] was not set, every submit was invalid and the canvas simply froze. Frames repeated and there was no error in the page's own logs. I saw it with two specific cars in the night look. The rig now never swaps cars inside a page, and the garage compiles every car first.
• Color. The intermediates come from JPEG, which is full range and BT.601. The assemble script converts them once to limited BT.709 and tags them. A lost tag doubles the contrast. Also, ffmpeg's vignette gets stronger with a bigger angle, and mode=backward brightens.
• Speed blur. The post-processing zoom blur reads the car's speed, so it is off for free cameras and on for the game's chase camera.

Back to a small loop #

The short loop on the project page is cut from a 720p version of this film. To fit its shape I had to crop out the 2.39:1 bars of the first seconds, and the end card, and I kept the shots without any HUD. A film that is a real game also ships its HUD, which is not something a mock-up would have to deal with.

The same rig could film the replay of Photo Run to give a shareable video. That is an idea in the project's notes, not something I have built.

Play Endurance Racing for free in the browser.

Related projects

• Endurance Racing — Rain-soaked GT3 racing for the browser