Leading AI companies fail to impress at City Council AI hearing

The race for the frontier of artificial intelligence may end in human extinction. Or it may just need some friendly guardrails. 

It depends on which of 14 panels you caught at Monday’s marathon City Council hearing on AI safety.

The bulk of the hearing, still carrying on – and apparently long from over – at the time of publication, was taken up by the testimony of four leading AI companies developing what’s known as frontier AI. That includes Google, Meta, Anthropic and OpenAI. (The City Council is taking legal action to enforce a subpoena that Elon Musk’s SpaceXAI ignored.)

New York City Council Speaker Julie Menin, a former regulatory lawyer who questioned representatives of those companies like opposing counsel on Monday, suggested she learned more from what those representatives didn’t say than from anything they did. 

“I think it’s disappointing that they can’t seem to answer some of the most basic, rudimentary questions,” Menin told City & State around hour four of the hearing.

A handful of other council members said they didn’t learn anything new at all.

When Menin asked the four companies’ representatives to quantify the risk posed by AI in the “worst-case, catastrophic scenario,” none of the largely policy-oriented representatives offered an answer.

“I don’t know,” said Morgan Dwyer, head of policy development and operations at OpenAI. “I also don’t think it matters whether it’s 1% or 10% or 20% chance of something catastrophic or wrong. None of these levels is remotely acceptable.”

“To say you don’t know and it doesn't matter is flippant at best,” Menin retorted, drawing chuckles from a crowd of what seemed to be mostly AI critics and safety advocates.

When Menin asked whether the companies would release models that fail internal or third-party tests, the companies spoke about their internal reviews and commitment to safety, but didn’t answer the specific questions. “I think it would be far clearer if we could get clear yeses or no’s,” Menin said. “I am going to take what each of you said as an equivocation.”

If the companies’ answers didn’t prove illuminating, they still gave Menin and the rest of the council a chance to shine as a legislative body doing something about the ubiquity and risks of AI, as federal action remains out of reach. 

The companies’ testimony, lasting several hours, also rang in stark contrast to the first panel of the day. On that panel, former researchers-turned-whistleblowers at those leading companies warned that such catastrophic events aren’t unlikely at all, and that the companies are acting recklessly in pushing the frontier of AI forward without guardrails.

Jacob Coxon, a former Anthropic researcher whose dire warnings captured international attention last month, said AI could make huge improvements in science and make people’s lives better. “But on the current path, I think it is more likely than not that humanity loses control of these AIs and it could end in human extinction,” he said.