The National Institute of Standards and Technology (NIST) released a report on November 2 that identifies gaps in how organizations prepare for and respond to incidents involving AI systems, including cyberattacks targeting those systems and harms arising from misuse, malfunction or unintended actions. 

The report draws on a May 14 workshop convened by NIST. It brought together more than 400 participants from government, industry, academia, standards organizations and civil society.

NIST examined current incident-management practices, available guidance, preparedness gaps and the roles of organizations involved in developing, deploying and using AI systems. The work was motivated by the federal government’s America’s AI Action Plan.

Participants highlighted the need for better coordination when an incident involves multiple model developers, deployers, service providers and users. 

Key areas for future work include monitoring and evidence collection, incident information sharing, and clarifying responsibilities across the AI supply chain. 

NIST also identified a need to develop additional resources to help model developers prepare for and respond to incidents.

NIST Considers Updates to Cybersecurity Incident Response Guidance

A key area for future work is updating NIST’s existing cybersecurity incident-response guidance, including Special Publication 800-61. That publication addresses cybersecurity incidents, but AI systems introduce additional challenges involving model behavior, autonomous actions and distributed responsibility across the AI value chain.

The report says traditional incident-response life cycles built for conventional software may be incomplete when applied to probabilistic, autonomous and deeply integrated AI systems.

Need for Greater Visibility into Systems Involving AI

For chief data officers (CDOs), the report highlights the importance of maintaining visibility into the data and systems involved in AI operations. When an AI system produces an unintended outcome, organizations need to establish what information it accessed, which actions it took, how those actions were authorized and where responsibility lies.

That requires incident-response processes to account for AI-specific evidence, data provenance, access permissions and system activity. It also reinforces the need to connect AI governance with broader enterprise risk and cybersecurity frameworks. 

NIST said the report will inform further stakeholder engagement and its future work on SI incident management. It does not establish a new mandatory incident-response standard.