Second Look Desk

Second Look is an open-source triage queue for image desks. A folder goes in. A ranked queue comes out, with an evidence receipt for every image. Code: https://github.com/prandata/secondlook FOR JUDGES Impact (30%): a desk gets a folder, and the queue says which images a person checks first. We ran 640 held-out images, cropped to 512 pixels. 344 of 640 were decided without a person: 227 routed as likely AI-made, 117 to likely fine. Meta tooling (25%): Video Seal runs in every receipt's watermark check. Muse Glimmer reads screenshot text on the receipt page. Both run locally. Demo (25%): the video runs 2 minutes 31 seconds. The README has the one-command setup. Technical depth (20%): thresholds frozen with a hash before the test. 0 of 314 camera photos auto-decided, upper bound 0.95%. 3 of 326 AI images landed in likely fine, upper bound 2.36%. Published failure: as shipped, it caught 0 of 500 TrueFake AI images. Three of 326 AI images landed in likely fine, upper bound 2.36%. IMPACT The user. A desk gets a folder of images and one question. Which ones should a person look at first? The desk may be a newsroom verification desk or an insurance claims team. The problem. Many detectors put one score or one label on each file. A desk with hundreds of files needs an order, not a pile of scores. Our own first build showed the gap. It checked one file at a time and said "can't tell" when it found no signed record. On 500 TrueFake AI images it said "can't tell" every time. For a desk, that sorts nothing. Evidence of the problem. In May 2026, NewsGuard gave five AI-image detectors the same 15 news photos from credible outlets. Three tools called some of them AI-made. That is 10 of 75 tool-photo decisions. It is one test on one news event, not a market rate, and not our result. We make no claim that Second Look does better. Verisk surveyed 300 US claims professionals at manager level or above. 98% agree that AI editing tools are fuelling a rise in digital media fraud. 32% are very confident that they can identify deepfakes. Verisk commissioned the survey and published it in March 2026. What the queue does. One command, secondlook queue, checks every file in a folder and writes a receipt for each. It sorts the folder into three bands. Needs a person comes first, highest detector score on top. Auto-decided holds files flagged as likely AI-made. Likely fine holds the lowest scores, and it proves nothing about an image. The queue never changes a receipt. What one held-out run showed. We ran the queue once on 640 held-out TrueFake images: 314 camera photos and 326 AI images. In the headline form, each image was cropped to 512 pixels, with metadata removed. 344 of 640 were decided without a person: 227 routed as likely AI-made, 117 to likely fine. On the files as given, 312 of 640 were. None of the 314 camera photos was auto-decided, with an exact 95% upper bound of 0.95%. Three of 326 AI images landed in likely fine, with an upper bound of 2.36%. So we make no 1% claim for likely fine. One run checked the 640 crops in under a minute, with 2 workers. Who has used it. Nobody outside our build has used Second Look. No desk has tested the queue. We have no pilot and no evidence that anyone will pay. The need comes from published studies, and we say so. The main risk. In the same Verisk survey, 39% cite weak integration between fraud tools and claims systems. The queue writes plain CSV and JSON files, with a hash for every receipt. We have not tested that they fit any desk's system. Problem statement. We enter under "Build for Your World": bring Meta's models into your own domain. We write What Survives in AI, a newsletter about AI. A newsletter must check an image before it shares it. Second Look does that check, on the user's own machine. DEMO What a judge can run. One command sets up the tool: ./scripts/demo.sh --no-serve. Then .venv/bin/secondlook queue samples --out queue-out ranks the demo folder and writes a receipt per image. It needs an Apple silicon Mac, macOS 14 or later, uv and Python 3.12. After set-up, the checks run with no network. What the video shows. It runs 2 minutes 31 seconds. One command reads a folder of 640 held-out images. The queue sorts them into three bands. One row opens its receipt. Then we show our two published failures. It holds up. The queue refuses to run if its frozen thresholds file does not match its pinned hash. A clean-clone test sets up a fresh copy and checks the nine demo samples offline. Labels. An AI voice (Google Gemini 2.5 Flash TTS) narrates the video. The images are TrueFake's, CC BY 4.0, cropped and with metadata removed. TECHNICAL DEPTH AND EXECUTION Thresholds frozen before the test. We split TrueFake's 983 Facebook-shared images by photo group, with a fixed seed. 343 went to dev and 640 to held-out. We fitted four thresholds on dev only: a high and a low for each image form. We committed them with their sha256 at 09:31 AEDT on 2 Nov 2026. The held-out bands came after that commit. The detector's scores for all 983 images existed before the split. Only the four thresholds were fitted, on dev only. The rule. The high cut-off lets at most 1 of 169 dev camera photos be auto-decided. The low cut-off lets at most 1 of 174 dev AI images into likely fine. The detector score and file records set each band. No model output sets one. Exact bounds, and what they allow. Every error count carries an exact one-sided 95% bound. With 314 held-out camera photos, "at most 1%" needs zero errors. We had zero, so the bound is 0.95%. On the likely fine side we had 3 of 326, with a bound of 2.36%. So we state that count and bound, and claim no 1% rate there. Checked by a second agent. An agent that did not write the queue re-derived all four thresholds with its own script. They matched, and the bands file was byte-identical. It recomputed every held-out count and bound, and each one matched. The full test suite passed at that code. Our published failures. First, Second Look as shipped caught none of 500 TrueFake AI images. Its detector had failed our own pre-set test on 29 Oct, so it shows a score only. The queue uses the same score with new thresholds fitted on TrueFake dev. They set the order, never the receipt's answer. Its ladder answered "can't tell" on all 983 TrueFake files. Second, three Stable Diffusion 3 images landed in likely fine. Our failures page lists each by hash and receipt, and shows no images. Where it is weakest. The detector trained on Stable Diffusion 1.x checkpoints, so the SD 1.5 and SD 2 rows are partly in-family. FLUX.1 and SD 3 are the nearer out-of-family tests. On 512-pixel crops, the queue auto-decided 23 of 57 FLUX.1 images and 27 of 70 SD 3 images. Most of the rest went to a person. One evidence contract. Each receipt holds the file's SHA-256, the tool versions, the answer and every evidence card. Every card has a mandatory "does not prove" field. The queue adds a band and a rank. It never changes the ladder's answer or a receipt. How we worked. AI agents wrote the code, and Pranjal directs. Every merged lane has a review by a different agent. Some merges landed before a PASS at the exact merged head. The private build repository keeps the merge log, with each case. More than 2,000 tests pass. Known limits. This is one public set of Facebook-shared copies. Dev and held-out share the same five generators and one camera source. New generators may behave differently. A desk's own folder may behave differently. File records can be wrong or missing. The tool has no earliest-copy finder and no reverse image search. What it does not do. It never clears an image. It does not say how an image was made unless a signed record says so. It has no face identification, no cloud upload and no accounts. A band is a reason to look first, or later. It is never a reason to reject a claim. CREDITS AND DISCLOSURES Made with AI. AI agents wrote the code. Pranjal directs. An AI voice (Google Gemini 2.5 Flash TTS) narrates the video. Second Look is a hackathon build, in development. It is not affiliated with Meta. Data. TrueFake (Dell'Anna et al., IJCNN 2025, arXiv 2504.20658), CC BY 4.0. The video shows crops with metadata removed. The TrueFake authors do not endorse Second Look. Credits. Classifier: Community Forensics ViT-S/16 (Park and Owens, CVPR 2025), MIT. Fact-check data: Data Commons ClaimReview feed (CC BY). Each review is its publisher's own work.