A new study by researchers at MIT and elsewhere found that, while AI assistance generally improved the accuracy of non-experts and clinicians in diagnosing skin diseases, AI explainability methods had different impacts depending on the users' knowledge level. Explainable AI methods help users know when to trust a model's predictions by describing or validating the model's decision-making. For instance, a model might use a heat map to highlight image regions that were most important in its diagnosis or a large language model (LLM) to explain the prediction in plain language. They found that non-experts' diagnostic accuracy improved, but it was largely due to deference to the AI system. Non-experts trusted LLM-based explanations whether they were right or wrong, and found explanations more convincing when they were vague or generic. 'Good AI systems can improve performance in some health settings, but this has to be balanced carefully with algorithmic deference that can lead to more...
learn more
