MIT spinout teaches AI to admit when it’s clueless

AI hallucinations are becoming more dangerous as models are increasingly trusted to surface information and make critical decisions.

We’ve all got that know-it-all friend that can’t admit when they don’t know something, or resorts to giving dodgy advice based on something they’ve read online. Hallucinations by AI models are like that friend, but this one could be in charge of creating your cancer treatment plan.

That’s where Themis AI enters the picture. This MIT spinout has managed to achieve something that seems straightforward in theory but is actually quite complex, teaching AI systems to say, “I’m not sure about this.”

AI systems typically display overconfidence. Themis’ Capsa platform acts as a reality check for AI, helping models recognise when they’re venturing into guesswork rather than certainty.

Founded in 2021 by MIT Professor Daniela Rus, along with former research colleagues Alexander Amini and Elaheh Ahmadi, Themis AI has developed a platform that can integrate with virtually any AI system to flag moments of uncertainty before they lead to mistakes.

Capsa essentially trains AI to detect patterns in how it processes information that might indicate it’s confused, biased, or working with incomplete data that could lead to hallucinations.

Since launching, Themis claims it has helped telecoms companies avoid costly network planning errors, assisted oil and gas firms in making sense of complex seismic data, and published research on creating chatbots that don’t confidently make things up.

Most people remain unaware of how frequently AI systems are simply taking their best guess. As these systems handle increasingly critical tasks, those guesses could have serious consequences. Themis AI’s software adds a layer of self-awareness that’s been missing.

Themis’ journey towards tackling AI hallucinations

The journey to Themis AI began years ago in Professor Rus’s MIT lab, where the team was investigating a fundamental problem: how do you make a machine aware of its own limitations?

In 2018, Toyota funded their research into reliable AI for self-driving vehicles—a sector where mistakes could be fatal. The stakes are incredibly high when autonomous vehicles must accurately identify pedestrians and other road hazards.

Their breakthrough came when they developed an algorithm that could spot racial and gender bias in facial recognition systems. Rather than just identifying the problem, their system actually fixed it by rebalancing the training data—essentially teaching the AI to correct its own prejudices.

By 2021, they’d demonstrated how this approach could revolutionise drug discovery. AI systems could evaluate potential medications but – crucially – flag when their predictions were based on solid evidence versus educated guesswork or complete hallucinations. The pharmaceutical industry recognised the potential savings in money and time by focusing only on drug candidates the AI was confident about.

Another advantage of the technology is for devices with limited computing power. Edge devices use smaller models that cannot match the accuracy of huge models run on a server, but with Themis’ technology, these devices will be far more capable of handling most tasks locally and only request help from the big servers when they encounter something challenging.

AI holds tremendous potential to improve our lives, but that potential comes with real risks. As AI systems become more deeply integrated into critical infrastructure and decisionmaking, the ability to acknowledge uncertainty leading to hallucinations may prove to be their most human – and most valuable – quality. Themis AI is making sure they learn this crucial skill.

See also: Diabetes management: IBM and Roche use AI to forecast blood sugar levels

Want to learn more about AI and big data from industry leaders? Check out AI & Big Data Expo taking place in Amsterdam, California, and London. The comprehensive event is co-located with other leading events including Intelligent Automation Conference, BlockX, Digital Transformation Week, and Cyber Security & Cloud Expo.

Explore other upcoming enterprise technology events and webinars powered by TechForge here.