Study finds AI rarely recommend unnecessary medical care, but still make mistakes

Study finds AI rarely recommend unnecessary medical care, but still make mistakes - Artificial Intelligence apps. (Photo: Courtesy)

The findings come as clinicians increasingly explore the use of AI tools to support medical decision-making.

Artificial intelligence (AI) models rarely recommend unnecessary medical tests and treatments, but they can still make mistakes in certain clinical situations, Israel's Ben-Gurion University of the Negev said on Tuesday.

In a study published in the Annals of Internal Medicine, researchers evaluated six commercial AI models from Google, Anthropic and OpenAI to determine whether they could distinguish situations in which screening, testing, or treatment may be overused.

Researchers assessed the models against recommendations from the Choosing Wisely initiative, which aims to reduce unnecessary medical tests, procedures and treatments while recognising that some interventions are appropriate depending on a patient's circumstances.

The findings showed that the models recommended potentially unnecessary medical action in 5.4 per cent of responses in which doctors should have exercised restraint. By comparison, they advised against an appropriate medical action in just 0.6 per cent of responses.

Researchers developed 225 clinical scenarios based on 15 Choosing Wisely recommendations. For each recommendation, 10 scenarios called for restraint and five called for action. Six commercial AI models were tested three times on each scenario, producing a total of 4,050 responses.

The researchers found that recommendations for potentially unnecessary care were concentrated in two areas: thyroid ultrasound for an incidentally discovered thyroid nodule smaller than one centimeter, and routine electrocardiography (ECG), a test that records the heart's electrical activity, in adults at low risk of cardiovascular disease.

Latest Stories

  • Trans Nzoia administrators ordered to stop saluting Governor Natembeya as standoff with Ruto deepens
  • Long distance drivers suspend DRC-bound cargo after 12 drivers reportedly killed
  • Uganda says it has no active Ebola cases as Africa CDC calls for urgent global support
  • Japan’s MMX mission aims to swipe soil samples from Mars’ moon Phobos

The findings come as clinicians increasingly explore the use of AI tools to support medical decision-making. They suggest that commercial AI models can often recognise when unnecessary care should be avoided, although their performance varies depending on the clinical situation.

The researchers cautioned that the study tested responses to single prompts rather than extended conversations and relied on standardised scenarios rather than real-world clinical encounters. Responses were initially assessed by an AI-based evaluator, with physicians reviewing and adjudicating the results.

The findings may therefore not reflect how the models perform in everyday medical practice or in consumer-facing chatbots used directly by patients.

They concluded that the AI models rarely advised against appropriate medical action in the scenarios examined, but recommendations for potentially unnecessary care remained a concern in specific clinical situations.