Pentagon completes AI chatbot pilot program for military medicine

The US Department of Defense has concluded a pilot program exploring the use of AI chatbots in military medical services.

In a January 2 announcement, the Department of Defense (DoD) revealed that the pilot focused on utilising large-language models (LLM) for clinical note summarisation and serving as medical advisors within the military.

This program is part of an ongoing effort as more AI firms offer their products to the US military, aiming to assess the potential uses of AI in defense applications.

The pilot, called the Crowdsourced AI Red-Teaming (CAIRT) Assurance Program, was a red-teaming effort led by the technology nonprofit Humane Intelligence.

Over 200 independent participants, including clinical providers and healthcare analysts, took part in the program, evaluating three leading chatbot models.

Collaboration also came from analysts at the Defense Health Agency and the Uniformed Services University of the Health Sciences, who worked alongside other participants to identify potential weaknesses in the systems.

The DoD stated that the pilot uncovered over 800 possible issues related to the use of chatbots in military medical applications, including vulnerabilities and biases.

“This exercise will result in repeatable and scalable output via the development of benchmark data sets,” the DoD explained, which could be used to assess future AI vendors and tools.

Matthew Johnson, the lead for the initiative, noted that the findings would influence future research and development in the area of Generative AI (GenAI) within the DoD.

The Chief Digital and Artificial Intelligence Office (CDAO), which was established in June 2022, oversees the integration of digital and AI technologies in US defense operations.