66098.1.
The office of the Chancellor of the California Community Colleges and the California State University shall, and the University of California is requested to, before providing a GenAI system to students, faculty, or staff,
do both of the following:(a) Convene a joint Jointly convene and participate in an intersegmental working group to develop standards for responsible GenAI training and procurement, as follows:
(1) The purpose of the working group is to develop responsible training protocols and procurement standards for the purchase, development, and use of GenAI systems for educational purposes that are consistent with current law regarding the regulation, procurement, development, and use of GenAI systems.
(2) (A) The working group
shall include, but not be limited to, privacy experts, representatives from student and faculty advocacy organizations, community-based organizations, labor organizations that represent employees of the segments, and civil society groups, and academic researchers focused on responsible GenAI procurement, design, and implementation. implementation from the three segments.
(B) The office of the Chancellor of the California Community Colleges and the Chancellor of the California State University
shall jointly convene the working group and designate a lead convener responsible for coordinating working group activities.
(C)The University of California is requested to join the working group.
(3) The working group shall do all of the following:
(A) Develop procurement standards that ensure that any procured GenAI system meets all of the following minimum protections:
(i) The system has undergone a documented risk assessment that evaluated potential harms, misuses, abuses, and bias, and has demonstrably
effective measures in place to ensure the system does not output harmful content or illegal content, including nonconsensual intimate imagery, suicide and suicidal ideation, disordered eating, and mental health therapy ordinarily administered by a licensed professional.
(ii) The system does not prioritize engagement over factual accuracy by using excessive sycophancy.
(iii) The system’s vendor has a safety monitoring protocol that detects a user’s intent to harm themselves or others, escalates to human review, and, in cases of imminent threat or grave bodily injury or death, makes appropriate referrals to authorities.
(iv) The vendor has a transparent incident reporting system to report and respond to harmful
outputs.
(v) The system is subject to written policies governing data use, retention, sharing with third parties, and deletion timelines.
(vi) The system is configured to collect only data that is strictly necessary for the stated educational purpose and is subject to regular audits.
(vii) The system’s vendor discloses all training data sources, including whether copyrighted content, personally identifiable information, or student-generated content was used.
(viii) As it can be reasonably determined, the vendor does not contract with entities that have unlawfully undermined privacy or civil liberties.
(ix) As it can be reasonably determined, the vendor does not use exploitative labor practices.
(B) Develop a training on the use of any procured GenAI system that shall include, but not be limited to, all of the following:
(i) Relevant systemwide policies of the institution concerning the use of a GenAI system, including policies related to academic integrity.
(ii) A disclosure An overview of
the privacy policies applicable to a GenAI system, including how user data and prompts may be collected, stored, or used to train the system. GenAI systems, and internet website links to the privacy policies.
(iii) Guidance on the limitations of a GenAI system, including the potential for inaccurate, incomplete, or misleading outputs.
(C) On or before January 1, 2028, present its recommendations for procurement standards and training developed pursuant to subparagraphs (A) and (B) to the respective system leaders.
(b) (1) Provide the training
developed pursuant to subparagraph (B) of paragraph (3) of subdivision (a) to students, faculty, or staff, as applicable.
(2) The office of the Chancellor of the California Community Colleges, the California State University, and the University of California may develop distinct versions of a training required pursuant to this section that are tailored to reflect the distinct uses, responsibilities, and applicable policies that are relevant to students, faculty, and staff, respectively. This article shall not be construed to require a single uniform training for all recipients.
66098.3.
The Until GenAI procurement standards are adopted, the office of the Chancellor of the California Community Colleges and the California State University shall, and the University of California is requested to, within 60 days following the execution of a systemwide contract for a GenAI system, submit a written report to the Legislature and relevant policy committees of the Legislature with jurisdiction over
higher education and privacy and consumer protection, consistent with Section 9795 of the Government Code, before executing a contract for the procurement of a GenAI system during the period beginning on January 1, 2027, and ending upon the adoption of procurement standards, as recommended by the working group established pursuant to Section 66098.1. Code. The report shall include, but not be limited to, a description of all of the following:(a) The GenAI system to be that was
procured, including the vendor name, the intended use, and the population of students, faculty, or staff who will have access to the system.
(b) The process used in evaluating and selecting the system, including any risk assessments conducted, competing systems considered, and how the procurement decision was made.
(c) The training that will be provided to students, faculty, and staff before the deployment of the system, staff, including the content of the training and the method of delivery.
(d) The data use, retention, and
privacy practices of the vendor, including how user data and prompts may be collected, stored, or used to train the system.