Opinion: New York still needs more AI regulation

Concerns over AI safety have been growing amidst calls to slow down superintelligence. AI company CEOs are sounding the alarm that the pace of AI development is moving faster than the industry can manage. Anthropic released a 154-page security report that shows how bad actors have used AI systems for everything from extortion and fraud to ballistic weapons tests. The July Hugging Face incident showed how OpenAI’s agents can operate with no human input and gain access to supposedly secure systems. We cannot rely on the very same industry leaders who have allowed innovation to outpace guardrails to write the rules for themselves.

These incidents are frightening and show how urgently we need government regulation and public oversight. In this moment, slowing the pace of AI development should mean a combination of frontier model regulation while setting clear guardrails on everyday use cases and environmental impacts. Meaningfully clawing back power from AI oligarchs means reining in AI agents and addressing the source of their current success – billions in revenue from imperfect AI-powered tools that have been rushed to market with little oversight, and the proliferation of data centers.

The good news is that over the last few years, New York has taken important steps towards both. For example, the state Legislature has passed and Gov. Kathy Hochul has signed into law my bills on regulating deepfakes in elections and on government uses of AI, as well asthe RAISE Act, which was specific to frontier models. It is true that frontier models can pose catastrophic risks from nuclear weapons to cyberattacks, but alarms of unknown dangers at the frontier should not distract from the ways AI is being used right now and the steps we can take to improve safety immediately and for the future. 

As chair of the state Senate Committee on Internet & Technology, I have passed legislation designed to accomplish both of these goals by addressing today’s concerns and, in doing so, implementing the necessary guardrails to prevent creating dangerous technologies in the future. 

Automated decision-making tools powered by AI exacerbate racial and gender bias, resulting in worse outcomes for marginalized communities. Misinformation from chatbots powered by large language models have led to life-threatening medical conditions, and young people have taken their own lives. My bill, the New York AI Act (S1169B), is a comprehensive set of regulations for the private sector to address the risks associated with AI use. The bill addresses algorithmic bias by requiring audits and risk assessments, and requires human oversight of consequential decision-making. 

Anthropic CEO Dario Amodei recently announced that Anthropic would commit to embed third-party auditors with employee-level access. The New York AI Act would require all companies, whether developing frontier models or deploying AI tools as part of their business, to produce audits and risk assessments. Government oversight can make companies address safety without colluding or engaging in a race to the bottom. 

Commercially available chatbots using large language models are some of the most common consumer uses of AI, and they pose some of the biggest risks to everyday users as they can spread misinformation and lack sufficient security measures against dangerous prompt responses. Regulating chatbots is imperative to ensure future development of AI technology accounts for safe and responsible use. 

This year, I passed a bill (S9051B) banning chatbots from providing unsafe features to children. This bill ensures AI-powered chatbots are not using features that can create unhealthy emotional relationships with young users or provide responses to minors that promote self-harm and inappropriate content like drug use, eating disorders or sexual content.

You might think that tech companies worried about the future of humanity would want to protect children, but I received opposition from at least three industry groups that represent OpenAI or Anthropic. We passed the bill unanimously through both houses of the legislature anyway, and Hochul can now sign this bill to implement important safeguards that set a standard for future AI development.

One of the most important steps we can take to slow the pace of AI development is to rein in uncontrolled data centers. I passed the Responsible Data Center Development Act (S10642) and worked with the governor to establish New York as the first state in the country with a statewide data center moratorium. We need to make sure that our resources are not being directed toward AI slop, the surveillance state or exponentially more dangerous models that are being operated without human control. Taking a responsible approach to data center growth means focusing our compute resources on technology that benefits the public good. 

Workplaces are rushing to add AI tools so they don’t get left behind, and teachers are encouraged to adopt them in classrooms without adequate research on their educational effectiveness or cognitive offloading. But the urgency to adopt these tools ignores their effects on workers and students, and does not account for the serious gaps in safety that even the CEOs of these companies are now acknowledging. Before we adopt AI tools, we need to make sure that they are effective and safe. 

Acknowledging that we need to slow the pace of AI development does not deter our technological advancement – it improves it. AI could be essential for scientific breakthroughs, curing diseases and improving our economy, but not if it continues to place people at risk, enable criminal activity and spread misinformation. Regulations that place guardrails on AI and protect the public can guide development toward beneficial uses. We can start implementing these right now in New York by signing my bills into law.