The Chief Digital and Artificial Intelligence Office (CDAO), under the U.S. Department of Defense (DoD) has completed a red teaming pilot, uncovering over 800 vulnerabilities in the use of large language models (LLMs) for improving military medical services.

The Defense department expects this exercise, called the Crowdsourced AI Red-Teaming (CAIRT) Assurance Program, to produce benchmark datasets for evaluating future vendors and tools, shaping its policies and best practices for responsible GenAI use, and enhancing military medical care. 

“Since applying GenAI for such purposes within the DoD is in earlier stages of piloting and experimentation, this program acts as an essential pathfinder for generating a mass of testing data, surfacing areas for consideration, and validating mitigation options that will shape future research, development, and assurance of GenAI systems that may be deployed in the future,” CDAO’s lead for this initiative, Dr. Matthew Johnson said in an update. 

This exercise was conducted by Humane Intelligence, a tech company building a community of practice around algorithmic evaluations, in collaboration with the Defense Health Agency (DHA) and the Program Executive Office, Defense Healthcare Management Systems (PEO DHMS). 

The exercise involved over 200 participants, including clinical providers and healthcare analysts from DHA, the Uniformed Services University of the Health Sciences, and the Services.