NIST Launches Evaluation Program for GenAI Technologies

The National Institute of Standards and Technology (NIST), under the U.S. Department of Commerce, has announced the launch of NIST GenAI, a new program aimed to evaluate generative AI technologies, focusing on text and image generation. 

This pilot addresses the research question of how human content differs from synthetic content, and how the evaluation findings can guide users in differentiating between the two. The generator task creates high-quality outputs while the discriminator task detects if a target output was generated by AI models or humans.

“The NIST GenAI program will issue a series of challenge problems to evaluate and measure the capabilities and limitations of generative AI technologies. These evaluations will be used to identify strategies to promote information integrity and guide the safe and responsible use of digital content,” a NIST media update mentioned.

NIST GenAI's objectives encompass creating benchmark datasets, advancing content authenticity detection technologies across various mediums like text, audio, image, video, and code, conducting comparative analyses using pertinent metrics, and fostering the development of tools to trace the origins of fake or misleading information.