NIST Launches ARIA Program to Evaluate AI Risk and Impact

The National Institute of Standards and Technology (NIST) is initiating a new program called Assessing Risks and Impacts of AI (ARIA), aimed at enhancing comprehension of AI capabilities and consequences.

ARIA seeks to aid organizations and individuals in assessing whether a particular AI technology will uphold validity, reliability, safety, security, privacy, and fairness post-deployment. 

The program will contribute to evaluating these risks and impacts by formulating novel methodologies and metrics to measure the effectiveness of a system in maintaining safe functionality within societal contexts.

“In order to fully understand the impacts AI is having and will have on our society, we need to test how AI functions in realistic scenarios — and that’s exactly what we’re doing with this program. With the ARIA program and other efforts to support Commerce’s responsibilities under President Biden’s Executive Order on AI, NIST and the U.S. AI Safety Institute are pulling every lever when it comes to mitigating the risks and maximizing the benefits of AI,” said U.S. Commerce Secretary Gina Raimondo.

Last month, NIST also launched NIST GenAI, a new program aimed to evaluate generative AI technologies, focusing on text and image generation. 

This pilot addresses the research question of how human content differs from synthetic content, and how the evaluation findings can guide users in differentiating between the two. The generator task creates high-quality outputs while the discriminator task detects if a target output was generated by AI models or humans.

“The NIST GenAI program will issue a series of challenge problems to evaluate and measure the capabilities and limitations of generative AI technologies. These evaluations will be used to identify strategies to promote information integrity and guide the safe and responsible use of digital content,”,” NIST said in a statement.