#aihallucinations #ai #AIbenchmarks Has AI benchmarking reached its limit, and what do we have to fill this gap? Sinan Ozdemir speaks to @JonKrohnLearns about the lack of transparency in training data and the necessity of human-led quality assurance to detect AI hallucinations, when and why to be skeptical of AI benchmarks, and the future of benchmarking agentic and multimodal models. This episode is brought to you by: • Trainium2, the latest AI chip from AWS: https://aws.amazon.com/ai/machine-learning/trainium/ • Dell AI Factory with NVIDIA: https://www.dell.com/superdatascience • Adverity, the conversational analytics platform: https://eu1.hubs.ly/H0jxK210 Interested in sponsoring a SuperDataScience Podcast episode? Email natalie@superdatascience.com for sponsorship information. In this episode you will learn: • (00:00:00) Introduction • (00:15:26) Sinan’s new podcast, Practically Intelligent • (00:21:07) What to know about the limits of AI benchmarking • (00:52:08) Alternatives to AI benchmarks • (01:00:00) The difficulties in getting a model to recognize its mistakes Additional materials: https://www.superdatascience.com/903