All You Need to Know About California’s New GenAI Rules

The California state government has published new guidelines for deploying generative AI responsibly. Governor Gavin Newsom’s Executive Order stresses ethical and transparent AI use, with state entities tasked to evaluate and integrate the technology according to its missions.

The “Benefits and Risks of Generative Artificial Intelligence Report” offers interim guidance until the final procurement and training policy is established in 2025 after piloting and stakeholder engagement.

The guidelines outline steps for state leaders to assess, procure, and deploy generative AI tools responsibly, aligning with the Executive Order's directive to consider generative AI pilot projects by July 2024.

Key highlights

The California Department of Technology (CDT), the Department of General Services (DGS), and the California Department of Human Resources (CalHR) will jointly oversee the responsible adoption and procurement of generative AI by state entities. Each entity is accountable for ensuring ethical, transparent, and trustworthy usage of generative AI within their operations, including assessing its impact on the state workforce.

Several steps are required for all state entities before engaging in any generative AI purchases:

• Designate a member of the executive team, typically the Chief Information Officer (CIO), to monitor and evaluate generative AI continuously.

• Attend mandatory executive and procurement team generative AI training.

• Review annual employee training and policy to ensure staff comprehension and acceptance of generative AI tool usage guidelines.

The following additional steps are necessary for state entities intending to procure generative AI intentionally:

• Identify a specific business need and understand the implications of generative AI utilization through comprehensive discovery and market research.

• Foster a culture of engagement and open communication with state employee end users to assess the impact of generative AI technology collaboratively.

• Evaluate risks and potential impacts associated with deploying generative AI, guided by a risk assessment process.

• Invest resources in preparing data inputs and thoroughly testing models before deploying generative AI solutions into production.

• Establish a dedicated team responsible for continuously evaluating generative AI use and its operational implications, ensuring human oversight where necessary.

• All state entities must conduct and report an inventory of generative AI usage to the California Department of Technology.

The government recommends a phased approach regarding training:

• Phase 1 targets executive leadership, legal, labor, and privacy specialists to address potential risks proactively.

• Phase 2 focuses on program staff, technical experts, and cybersecurity professionals to develop necessary competencies and ensure safety during planning and piloting.

• Phase 3 provides general education and training to the broader workforce before deploying generative AI tools, ensuring alignment with business needs and effective service delivery.

Training modules will cover foundational AI concepts, generative AI functionality, legal and privacy considerations, technical aspects, and procurement processes. Additional support will be provided through workshops and learning opportunities. Procurement generative AI Training, beginning March 29, 2024, will equip purchasing officials with the necessary knowledge to identify and manage generative AI purchases effectively.

Overall, the state is committed to fostering safe, secure, and responsible generative AI implementation while preparing its workforce for the challenges and opportunities of the generative AI economy.