DoD to Evaluate CDAO’s Adoption of AI

The U.S. Department of Defense (DOD) is conducting a thorough evaluation aimed at determining the overall impact and "effectiveness" of the Chief Digital and Artificial Intelligence Office's (CDAO) procurement, strategy, and policy development, as well as its adoption of AI.

“We may revise the objective as the evaluation proceeds, and will consider suggestions from management for additional or revised objectives. We plan to perform this evaluation in accordance with the Council of the Inspectors General on Integrity and Efficiency ‘Quality Standards for Inspection and Evaluation’,” a memorandum issued by the Office of Inspector General under the Defense Department said.

The evaluation is a part of the watchdog's continuous series of oversight projects focused on monitoring the Defense Department's execution of AI initiatives.

The evaluation will be done at CDAO, however, over time, newer locations could be added during the process.

Last month, it was reported that the DoD’s Chief Digital and Artificial Intelligence Office (CDAO) is partnering with Scale AI, a test and evaluation (T&E) partner for AI companies, to create a comprehensive T&E framework for the responsible use of large language models (LLMs) within the DoD.

Scale AI will develop customized benchmark tests tailored to DoD use cases, integrate them into the T&E platform, and aid the CDAO's T&E strategy for LLMs.

In January 2024, CDAO CDAO) initiated the initial phase of the AI Bias Bounty program, a crowdsourced initiative designed to identify bias in AI systems.