# Traceback

- **Event:** [Meta Global AI Developer Hackathon](https://cerebralvalley.ai/e/meta-ai-developer-hackathon)
- **When:** Oct 26, 12:00 PM – Nov 4, 9:00 PM (EST)
- **Where:** Online
- **Team:** [Amit Chowdary](https://cerebralvalley.ai/u/amitchowdary), [devalla ashvita](https://cerebralvalley.ai/u/ashvithaxai)
- **GitHub:** https://github.com/AmitChowdary122/traceback
- **Demo video:** https://www.youtube.com/watch?v=kFGHwkIziEo
- **Gallery:** https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery
- **Page:** https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/3951

This is a trace back AI. Within Digital Forensics, the statement 'AI said so' is not admissable. All claims must be provable to an examiner. Chat assistants may be able to create a coherent summary based on the phone extraction. However this means that they can't actually prove where the information is coming from. One false phone number or incorrect time could make or break a case.

The user will ask a question, for example 'Where was the phone between 9pm and 11pm and who did it speak to?' The Traceback AI will use 6 search tools (messages, calls, locations, media, transcripts, search) within Meta's Muse Spark to search the parsed output from the phone extraction (ALEAPP output from public Android 14 reference image). It must provide a response using the submit_answer tool. This response will consist of multiple claims, each claiming to support a certain field from a certain artifact with a specific value.

The deterministic validator will check all claims are valid from the raw data. This will only accept claims that actually exist, have the correct value, are within the timeframe specified in the question and do not have any text that refers to a phone number, date, time, coordinates or contact without being supported by the claims themselves. Claims will only be accepted as findings if they have been verified. Each finding will show the source row and source path. Rejected claims will be displayed separately with a reason in plain text. The model's note will show as 'not verified'.

This system does more than just text. Using Muse Voice Transcribe video audio will be transcribed so it can be used in quotes. Objects will be detected and boxed within the photos using SAM 3.1. This means that mentions of 'license plate' will have a detection supporting it plus GPS from the photo itself. This same AI can also be run using Meta's Muse Spark Open, which uses their open weights (Baseten is used here).

Known to work:

Fault injection: 577 faults were injected into real responses from the model. These included 1 minute time offset, 1 digit swap, fake record ID and fake phone number. All were rejected!

Live testing: 22 questions asked on Muse Spark 1.3. 0 errors. 92 claims were valid and 12 were invalid. None were incorrectly rejected. All 4 questions that couldn't be answered were correctly rejected. The 12 invalid claims were actual invalid claims. Whilst the model correctly stated who sent the messages, it never actually referenced the sender field. Just being true isn't good enough, it needs to be within the cited record.

Offline testing completed: 527 tests, including a 160 case red team suite for the validator.

Try it out on the live replay site. There are 13 recorded runs. When the site is built, each run will be validated again against the evidence. No keys required. On the page it clearly states that only the highlighted values and entities will be verified. Everything else is from the model. https://amitchowdary122.github.io/traceback/

## More from Meta Global AI Developer Hackathon

- [The Wolf Pack](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/3948)
- [MazeFlaze](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/3949)
- [ChronoStrider](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/3950)
- [Amarizoft](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/3952)
- [No Body](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/3953)
- [ProtoMind](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/3954)

---

Markdown version of https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/3951. Site index for agents: https://cerebralvalley.ai/llms.txt · full text: https://cerebralvalley.ai/llms-full.txt