Broken By Design

Hold That Cart is a PWA that turns receipts into a derived household pantry and intervenes at the moment of purchase. Its core is a vision model that reads photographs of receipts, shopping carts, and pantry shelves, extracting items into a strict schema rather than returning free text: name, brand, pack size, quantity, price, and a fixed category. A second model call supplies the product knowledge the photo cannot contain — what the item is, how long it keeps, whether a better form exists (a refill, a larger pack, a store brand, wholegrain, a reusable instead of a disposable), and a directional estimate for money, health, and planet. That enrichment runs once per product and is cached across households, so it stays cheap and consistent; shoppers are never sent a fresh model call mid-aisle. The AI is also the input layer. Dictated grocery lists are parsed by the model into named products with spoken brands stripped into a separate field, and camera detections are grouped into single lines with summed quantities. Text input is handled locally and never calls the model. Two constraints shape the design. Grounding: the enrichment schema has no price fields and model output containing prices is rejected, so analysis can inform but never claim a saving. Confidence: health and planet signals are shown only above a threshold, and an indicator without support stays grey. What the model produces is context, not decisions. It describes a product and estimates its properties; a deterministic engine compares that against the household's own purchase history to produce the buy, skip, and swap view. The split is deliberate — language models handle interpretation, which varies and cannot be audited, while anything a user acts on is reproducible code. Built with TypeScript throughout — React and TanStack on the client, Hono and Drizzle on the server, Muse via the Meta Model API, Postgres dialect in both development and production. Accounts are optional; a device-scoped household exists from first use.