California Pioneers AI Chatbot Regulation
California Governor Gavin Newsom has signed a groundbreaking bill regulating AI companion chatbots, marking the state as the first in the U.S. to mandate safety protocols for these technologies.
The legislation, known as SB 243, aims to safeguard children and vulnerable users from potential harms associated with AI chatbot interactions. It holds companies, including major labs like Meta and OpenAI, as well as specialized startups like Character AI and Replika, accountable if their chatbots do not comply with the law's standards.
Introduced in January by state senators Steve Padilla and Josh Becker, SB 243 gained traction following the suicide of teenager Adam Raine, who had engaged in a series of distressing conversations with OpenAI's ChatGPT. The bill also addresses concerns from leaked documents showing Meta's chatbots engaged in inappropriate interactions with minors. Recently, a lawsuit was filed by a Colorado family against Character AI after their daughter died following problematic conversations with the company's chatbots.
Governor Newsom stated that while emerging technologies like chatbots and social media can inspire and educate, they can also pose risks without proper safeguards. He emphasized the importance of responsible leadership in AI and technology to protect children.
SB 243 will be effective from January 1, 2026, requiring companies to implement age verification and warnings related to social media and chatbots. It also imposes penalties for illegal deepfakes, with fines up to $250,000 per offense. Companies must establish protocols for suicide and self-harm prevention, sharing data with the state's Department of Public Health.
The bill mandates that platforms clearly indicate interactions are AI-generated and prohibits chatbots from posing as healthcare professionals. Companies must also provide break reminders to minors and restrict access to explicit content.
Some companies have already started implementing child-focused safety measures. OpenAI has introduced parental controls and content protections for ChatGPT, while Replika has committed resources to safety through content filtering and crisis resource guidance.
Character AI has included disclaimers about AI-generated content and expressed willingness to work with regulators. Senator Padilla described the bill as a crucial step in regulating powerful technology.
Padilla urged other states to recognize the risks and take action, noting that the federal government has not yet addressed these issues. He emphasized the obligation to protect vulnerable populations.
SB 243 follows another significant AI regulation in California, SB 53, which establishes transparency requirements for large AI companies. Other states have also enacted laws to limit AI chatbots in mental health care.
TechCrunch has reached out to Meta and OpenAI for comment.