EarWitness

EarWitness: check that your podcast ads ran as bought, and ask what the shows are saying. Ask EarWitness what podcast hosts are saying about a brand. Muse Spark reads the episodes and answers, and every claim links to the passage it came from. Then plain code checks each claim against its passage and removes the ones that don't match. We tested Ask on 118 scripted questions in 76 conversations, graded by code against fixed answers: it never wrongly said nothing was found, and 3 follow-up answers were about the wrong brand, 2 of which we have since fixed. It can also leave out a brand that has ads in the collection, so an answer is a starting point, not a full count. The code check can also remove a true claim. Ask works best on one brand at a time; questions comparing two brands or asking for counts may answer only part. Brands and agencies pay for podcast ads read by the host. To check that an ad ran as bought (the right show, the right week, the right read, the right code), someone listens by hand. EarWitness searches what was said in the episodes and points to the exact passage, with a short clip where one exists. We fetch each episode once, as a listener would, so EarWitness checks ads read into the episode, mostly host reads, not ads inserted later. On 565 ad reads we labelled by hand, measured before the window and in-sample (we also used these reads to tune the rules, so the numbers are optimistic), the pipeline finds 96.6% of the ads. Plain-code rules raised the reads named with the right brand from 195 to 253, while the names shown held at about 82% right. The weak spots, plainly: many ads still show "Brand not identified", and a clip can start late, end early, or run into the next ad. Try it: start at https://getearwitness.com/access/ and sign in as judge6@getearwitness.com with the password q2Xaq-rvKwf-7XxsE-7z8xk. This judge login closes after judging. Play the real ad on your Overview, or search a brand. Open the ad's page, with its facts and sources. Ask a question and open "How this answer was made". Ask's design and model tests came before the window. Its build started the evening before kickoff and was finished, reviewed and shipped during the window. In the window we also built meaning alerts and Muse Spark's naming of ads in back-to-back ad breaks, and measured Muse Spark against our plain-code rules. Everything else existed before: the pipeline, the web app and the plain-code ad rules. The code is open source under the Apache-2.0 license: https://github.com/renoneto/earwitness