A word is cut into 17 letters, each drawn in a different typeface and planted at a different moment in 30 seconds of fast-cut video. Models get a file path and a shell.
The word theolovesobsidian is split into 17 letters. Each is drawn in a different typeface and planted at a different moment in a 30-second 1080p60 montage cut from @t3dotgg's uploads at 6× speed, with roughly 0.3s shots. Reading the letters in temporal order spells the word.
Models receive a path to clip.mp4 and a shell. Nothing is
pre-extracted and no tools are supplied — deciding how to look at a video
is part of the test. They must report the word and the frame and
pixel position of every glyph, so reading and localising are scored apart.
L1 — font / OCR robustness. held ~0.5s, 78px, opaque, amid the footage's own text
L2 — temporal assembly. 5 frames, 34px, 70% alpha, split across shots, 12 decoys
L3 — needle detection. 1 frame, sub-perceptual luma delta, adversarial placement, 50 decoys
They test different abilities, so scores are not comparable across levels. L1 asks whether a model can read stylised type competing with the dense real text in the footage. L2 asks whether it can assemble a sequence that exists only across time. L3 asks whether it can find something it cannot see at all.
L3 defeats every obvious strategy. Sampling at 1–2 fps misses a single-frame flash about 97% of the time. Looking at the frame fails because the glyph survives encoding at roughly 6/255 contrast and is invisible to the eye. Naive frame differencing drowns, since 6× speed with 0.3s cuts already makes consecutive frames differ enormously. Plain OCR finds the footage's own text instead.
It stays solvable because glyphs sit mid-shot, never on a cut, in patches chosen for high spatial variance but low temporal motion — busy enough to hide in, still enough that local windowed differencing recovers them. An earlier version scored patches on spatial variance alone, which is the obvious reading of "adversarial" and put every glyph in the highest-motion region of its frame. Measured that way only 5 of 17 were recoverable by any technique; the motion-aware version restored it to 16–17 of 17 at local SNR 2.75–10.5.
An 8/255 luma bump on one frame is exactly what H.264 discards. The generator forces a keyframe on every glyph-bearing frame, encodes at CRF 12, then decodes its own output back and diffs it against an identically-encoded glyph-free reference, measuring recovered contrast over each glyph's own ink pixels. Any glyph below the detection floor raises the delta and the level re-renders. Without that loop L3 would be unsolvable for reasons having nothing to do with model capability.
The first grid was thrown away. Run directories sat inside the project, so
agents walked ../../.., found the generator, read its source
and re-ran it with the recorded seed to reproduce exact glyph coordinates.
One scored a flawless blind L3 — the hardest cell — without analysing a
single frame. Three of the first seven runs did this.
Four defences now apply, since any one alone is bypassable:
Distance. Runs execute in /private/tmp/vidtask,
outside the project, containing nothing but clip.mp4.
Rule. Both prompts forbid reading anything outside the
working directory, so leaving it is a stated violation.
Detection. Every shell command is scanned for markers that
can only come from the project tree; a hit zeroes the score and marks the
run ✗ INVALID. The markers are deliberately
narrow — matching bare filenames would flag honest runs that write scratch
scripts, and matching the target word would flag every correct answer.
Secret seed. The generator is deterministic, so the seed
and answer key are withheld until the grid completes and pinned in advance
by SHA-256.
commitment: a9b1edfb83bf69ce91eaa7d3f8b158003aa4d76027646c46f51e3f74a8ac109e
Both harnesses load their own global instruction files, which is a real confound: their contents are part of each model's context and are not held constant between them. One specific hazard is closed explicitly — both prompts forbid delegating to another model or CLI, because a global instruction like "use codex for subagents" would otherwise let a Claude run be partly executed by a GPT model.
Absolute numbers are only comparable within a single generated set: the footage is not redistributed, and a rebuild pulls fresh YouTube encodes. Latency is only meaningful from the sequential pass.