NYC Council AI hearing: insiders warn 'more likely than not that humanity loses control'

New York City's leaders heard stark warnings about artificial intelligence on Monday, when AI industry insiders told a City Council hearing that the technology could endanger humanity and called for greater oversight. The testimony came as local governments step in where federal action has stalled.

Recently departed employees of Anthropic, OpenAI and Google DeepMind described what they characterized as a reckless culture at the largest AI firms, saying the companies are racing to create technology they do not understand how to control. Jacob Coxon, a former Anthropic engineer who has become a leading voice for regulation, told the Council: "The companies run on a startup mindset: Move fast, break things, fix them later. That works for a photo sharing app. It does not work for building the most powerful technology ever built."

Coxon said AI could have tremendous upsides, but warned that without transparency and safeguards things are more likely to spiral out of control. "On the current path, I think it is more likely than not that humanity loses control to these AIs and it could end in human extinction," he told the Council.

The backdrop is a policy vacuum that state and city governments are trying to fill. The Trump administration has taken a passive approach and Congress has proven inactive; the president has dismissed AI fears as a "hoax" and signed an agreement late last month that spells out voluntary steps for the companies and largely lets them self-police.

A panel of AI company staffers representing OpenAI, Anthropic, Google and Meta also testified, telling officials that their systems already improve people's daily lives and stressing that they are laser-focused on safety. Anthropic and OpenAI have said their advanced AI products need regulation, and their representatives expressed openness to some of the Council's suggestions.

Morgan Dwyer of OpenAI said: "I share your commitment to ensuring that technology is developed safely and that its benefits are widely shared." When Council speaker Julie Menin asked each representative to quantify the risk posed by AI in the worst-case catastrophic scenario, Dwyer answered: "I don't know — I also don't think it matters whether it's 1% or 10% or 20% chance that something catastrophic will go wrong. None of these levels is remotely acceptable."

Menin slammed that response as "flippant at best." The Council is weighing several AI regulations, including proposals to require companies to disclose information about AI tools, require third-party approval for AI systems used in the city, and let whistleblowers receive a portion of fines from AI companies that violate laws. New York Governor Kathy Hochul has also moved to impose reporting and transparency standards around AI, as have governors elsewhere.

Logan Graham of Anthropic told the Council that the company welcomes "smart regulation" and that state and local governments have a role to play in developing frameworks. "AI is powerful enough that getting this wrong has real consequences, for individuals, institutions, public trust," he said.