How it finds the first app frame
- Your browser plays the video locally and takes a 32 by 18 pixel color snapshot twice a second, for the first 3 minutes.
- It splits the video into shots: a new shot starts when a snapshot differs from the previous one by more than 6 percent on average, and every 6 seconds inside a long shot, to catch slow fades and titles over a dimmed app. Black or nearly blank frames are skipped, and a shot that looks almost the same as one already checked reuses that answer.
- For each shot, in order, it builds one 1024 by 288 picture: your screenshot on the left, a frame 0.5 seconds into the shot on the right. Only that picture is sent.
- The server asks Llama 4 Scout to describe the right half: how much of it is a software screen, and whether a large headline sits on top. If most or all of it is software and there is no headline, Mistral Small 3.1 is asked the same question to confirm. The model describes; the rule decides.
- At the first confirmed shot, your browser steps through every frame of the video at its own frame rate (measured when the video loads, then snapped to the nearest standard rate such as 23.976, 25, 29.97 or 60) between the last "not yet" snapshot and the first "app" snapshot, and picks the first frame that looks more like the app than what came before. No model calls happen in this step.
Accuracy
Hit means within 1.0 second of a frame marked by hand. The table below is from an earlier version of this method in Python (before the blank-frame skip and answer reuse were added), not this page. A browser benchmark (bench/run.mjs) that runs this page in Chrome on every video is written and will replace this table once it has run.
| Video | Set | Found | Marked by hand | Result |
|---|---|---|---|---|
| ciph | dev | 12.27 | 12.27 | hit |
| faultrehearsal- | dev | 9.27 | 9.33 | hit |
| graphhacks- | dev | 15.97 | 16.20 | hit |
| motiontracker- | dev | 13.63 | 13.62 | hit |
| boardclerk- | dev | 0.00 | 0.00 | hit |
| motiontracker- | dev | 5.27 | 0.00 | miss: a big caption over the app reads as a headline |
| h- | holdout | 0.00 | 0.00 | hit |
| h- | holdout | 0.00 | 0.00 | hit |
| h- | holdout | 18.27 | 18.33 | hit |
| h- | holdout | 9.27 | 9.27 | hit |
| rcv- | no app | 58.57 | none | false positive: this video never shows an app |
Reference run: Python pipeline, Oct 2026. Not yet the browser run.
Limits
- Only the first 3 minutes are scanned.
- A large caption over the app counts as a title, so the answer can land late (motiontracker-v5).
- A video with no app at all can still return a time (rcv-yt). Check the frame before you cut.
- Exports assume the file starts at timecode 00:00:00:00, which is true for screen recordings and most exports.
- Import into Resolve, Premiere and Final Cut has not been tested yet.
Cost
About 25 neurons per Llama look and 45 per Mistral look on the Cloudflare Workers AI free allowance, which this site shares with another project. Every model attempt is counted, including ones that fail. A picture the server has already answered is answered again from a cache at no cost. The site caps itself at 6,000 neurons a day, 40 looks per visitor per day and 25 per video. When a cap is hit, the page says so and you pick the shot yourself; the exact frame is still found on your computer.