ForeSight

ForeSight is an experimental copilot that predicts what a physical structure might do before you change it, asks for what it cannot see, and lets reality check its answer. When people experiment with physical systems, a small change can have an unexpected consequence. A technician, researcher, or student may need to understand what supports a structure before moving a component. ForeSight explores a simple but powerful question: Can AI help someone examine the evidence, explore possible outcomes, and test a prediction before making the change? Our first test environment is a Jenga tower. The user captures views of the physical tower with a phone camera. Meta SAM 3.1 analyzes each image and returns candidate detections, which ForeSight displays on the captured views. Because a camera cannot reliably reveal every hidden block, the operator confirms the tower arrangement rather than the system pretending it knows. The user then selects a proposed block removal. A simplified static model calculates a prediction, while a separate experimental 3D physics replay visualizes possible motion. If a deliberately unknown block could change the assessment, ForeSight shows the different possibilities and asks for clarification. The user can lock the prediction before touching the tower, perform the move, and record whether it stood or fell. The evidence trace distinguishes model observations, operator-confirmed inputs, predictions, simulations, and real-world outcomes. It also records whether the user steadied the tower: an assisted result is labeled not comparable, because the hand may affect the outcome. Meta Muse Voice Transcribe lets the user speak a question and see its transcription. Muse Spark can answer questions about the current scene and summarize SAM’s candidate box data. The selected block is currently entered manually, and spoken playback uses the browser’s voice. ForeSight is a research prototype, not a structural-safety tool. Its tower model is operator-confirmed, its simulation uses simplified assumptions, and its camera views do not automatically create a 3D reconstruction. Our goal is to make physical experimentation more transparent: observe, ask, predict, test, and learn from the difference between the model and reality.