66098.1.
The office of the Chancellor of the California Community Colleges and the California State University shall, and the University of California is requested to,
do both of the following:(a) Jointly convene and participate in an intersegmental working group to develop standards for responsible GenAI training and procurement, as follows:
(1) The purpose of the working group is to develop responsible training protocols and procurement standards for the purchase, development, and use of GenAI systems for educational purposes that are consistent with current law regarding the regulation, procurement, development, and use of GenAI systems.
(2) (A) The working group shall include, but not be limited to, privacy experts, representatives from student advocacy organizations,
systemwide academic senates, community-based organizations, labor organizations that represent employees of the segments, and civil society groups, and academic researchers focused on responsible GenAI procurement, design, and implementation from the three segments.
(B) The office of the Chancellor of the California Community Colleges and the Chancellor of the California State University
shall jointly convene the working group and designate a lead convener responsible for coordinating working group activities.
(3) The working group shall do all of the following:
(A) Develop procurement standards that ensure that any procured GenAI system meets all of the following minimum protections:
(i) The system has undergone a documented risk assessment that evaluated potential harms, misuses, abuses, and bias, and has demonstrably effective measures in place to ensure the system does not output harmful content or illegal content, including nonconsensual intimate imagery, suicide and suicidal ideation, disordered eating, and mental health therapy ordinarily administered by a licensed
professional.
(ii) The system does not prioritize engagement over factual accuracy by using excessive sycophancy.
(iii) The system’s vendor has a safety monitoring protocol that detects a user’s intent to harm themselves or others, escalates to human review, and, in cases of imminent threat or grave bodily injury or death, makes appropriate referrals to authorities.
(iv) The vendor has a transparent incident reporting system to report and respond to harmful outputs.
(v) The system is subject to written policies governing data use, retention, sharing with third parties, and deletion timelines.
(vi) The
system is configured to collect only data that is strictly necessary for the stated educational purpose and is subject to regular audits.
(vii) The system’s vendor discloses all training data sources, including whether copyrighted content, personally identifiable information, or student-generated content was used.
(viii) As it can be reasonably determined, the vendor does not contract with entities that have unlawfully undermined privacy or civil liberties.
(ix) As it can be reasonably determined, the vendor does not use exploitative labor practices.
(B) Develop a training on the use of any procured GenAI system that shall include, but not be limited to, all
of the following:
(i) Relevant systemwide policies concerning the use of a GenAI system, including policies related to academic integrity.
(ii) An overview of the privacy policies applicable to a GenAI system, including how user data and prompts may be collected, stored, or used to train GenAI systems, and internet website links to the privacy policies.
(iii) Guidance on the limitations of a GenAI system, including the potential for inaccurate, incomplete, or misleading outputs.
(C) On or before January 1, 2028, present its recommendations for procurement standards and training developed pursuant to subparagraphs (A) and (B) to the respective system
leaders.
(b) (1) Provide the training developed pursuant to subparagraph (B) of paragraph (3) of subdivision (a) to students, faculty, or staff, as applicable.
(2) The office of the Chancellor of the California Community Colleges, the California State University, and the University of California may develop distinct versions of a training required pursuant to this section that are tailored to reflect the distinct uses, responsibilities, and applicable policies that are relevant to students, faculty, and staff, respectively. This article shall not be construed to require a single uniform training for all recipients.
66098.3.
Until GenAI procurement standards are adopted, the office of the Chancellor of the California Community Colleges and the California State University shall, and the University of California is requested to, within 60 days following the execution of a systemwide contract for a GenAI system, submit a written report to the Legislature and relevant policy committees of the Legislature with jurisdiction over higher education and privacy and consumer protection, consistent with Section 9795 of the Government Code. The report shall include, but not be limited to, a description of all of the following:(a) The GenAI system that was procured, including the vendor name, the intended
use, and the population of students, faculty, or staff who will have access to the system.
(b) The process used in evaluating and selecting the system, including any risk assessments conducted, competing systems considered, and how the procurement decision was made.
(c) The training that will be provided to students, faculty, and staff, including the content of the training and the method of delivery.
(d) The data use, retention, and privacy practices of the vendor, including how user data and prompts may be collected, stored, or used to train the system.