# AutoCut

- **Event:** [Claude Build Day](https://cerebralvalley.ai/e/claude-startups-build-day)
- **When:** Sat, Jun 13 at 9:00 AM – 10:00 PM (PDT)
- **Where:** San Francisco, CA
- **Team:** [Wei Tu](https://cerebralvalley.ai/u/weitu)
- **GitHub:** https://github.com/dw820/ClaudeFable5Build
- **Website:** https://claude-fable5-build.vercel.app/
- **Demo video:** https://canva.link/hgk15ltim3j2w40
- **Gallery:** https://cerebralvalley.ai/e/claude-startups-build-day/hackathon/gallery
- **Page:** https://cerebralvalley.ai/e/claude-startups-build-day/hackathon/gallery/132

AutoCut — an autonomous AI video editor. You upload raw footage and give a one-screen brief ("Viral Short,
  9:16, 30s, loop back to the first frame"), and an agent does the rest of the editing.

How it works

  Offline preprocess (src/preprocess/) turns raw clips into a machine-readable library:
  - ffprobe for duration/resolution, Whisper (via Replicate) for word-level transcripts, scene detection + a VLM
  for per-scene captions and tags, and now a poster frame per clip for the UI.
  - Output is clips.json — the structured contract the agent reasons over directly (no ffmpeg in the agent's
  hands).

  The agent loop (src/agent/, src/loop/) reads clips.json, picks segments, and emits a schema-validated EDL
  (edit decision list). A deterministic renderer (src/edit/, ffmpeg) compiles the EDL into the final cut, and an
  independent verifier returns a structured grade. Those zod schemas in src/loop/types.ts are the single source
  of truth at every boundary, so a bad model output fails fast instead of three modules downstream.

  The UI (Next.js, app/ + components/) is a two-screen flow: Setup (the footage library grid + the brief form)
  and Generation. Built with shadcn/ui + Tailwind.

  Infrastructure

  - Supabase Storage is the source of truth for assets: source-clips (private — videos + manifest), renders
  (public — final mp4), thumbnails (public — poster frames). Daytona persistent disk is a download cache.
  Supabase Postgres + pgvector holds caption embeddings for semantic clip search.
  - Models: Opus 4.8 drives the agent (not Fable 5); Replicate hosts Whisper + the VLM.

  Where it stands

  Preprocess and the core loop work and are well-tested (~193 unit tests). Known gaps: the agent server still
  hardcodes a SAMPLE_LIBRARY instead of loading real clips.json (agent-server.ts:141,274), the Daytona caching
  layer is specced but not built, and transcription is on base Whisper with WhisperX (true word-level) as the
  agreed next step.

  This session's work: rolled per-clip thumbnail generation into the preprocess pipeline and got the poster
  frames rendering on the Setup screen's footage grid.

---

Markdown version of https://cerebralvalley.ai/e/claude-startups-build-day/hackathon/gallery/132. Site index for agents: https://cerebralvalley.ai/llms.txt · full text: https://cerebralvalley.ai/llms-full.txt