Pelosi's successor in Congress wants to slow AI down and give it a kill switch

After winning the San Francisco congressional race for the seat vacated by Nancy Pelosi, California state senator Scott Wiener sat down with MIT Technology Review. The interview took place only hours after he was declared the victor, and according to the report he was already talking not about the campaign but about the risk of losing control of AI.

As a state senator, Wiener authored SB 1047, a bill meant to curb the risks of advanced AI that the report says was set to become the first real piece of frontier AI legislation in the country before Governor Gavin Newsom vetoed it in 2024. He then sponsored SB 53, a lighter-touch bill Newsom signed. Both have inspired proposed federal legislation.

One prompt for the conversation was a report released the previous day by the AI nonprofit METR, which the article says was widely interpreted as evidence that Anthropic had achieved recursive self-improvement, meaning AI that meaningfully contributes to its own advancement. Wiener called it "very concerning" and said he had hoped Congress would take decisive action before the end of the year, adding that this does not appear to be happening.

The report notes that he used the phrase "recursive self-improvement" half a dozen times in a 20-minute conversation. "We need to slow this down," he said, "because it's moving very quickly, and we could sleepwalk into a disaster."

On substance, Wiener says he would be eager to serve on a select committee on artificial intelligence if Congress forms one, something Democrats have discussed since September. He supports a strong, empowered and nimble federal regulator, citing legislation proposed by California Democrat Ro Khanna, and stresses whistleblower protections, testing and auditing requirements, and a kill switch requirement.

He is equally clear on federalism: he opposes broad federal preemption of state AI laws. Quoting Brandeis that states are laboratories, he argues that states can stay nimble in ways Congress sometimes cannot.

The report also covers his political baggage. His campaign received over $3 million from Public First Action, a nonprofit that favors AI regulation and receives substantial funding from Anthropic, which led opponents to accuse him of being too close to the industry. In a written reply after the interview, Wiener rejected that, saying the group backed him because he was the first lawmaker in the country to introduce AI regulation and the first to pass and implement it, and calling the attacks on him cynical.

Put together, the profile sketches a new kind of AI politician: someone who ran the legislative experiment at state level, arrives in Washington with a specific list of provisions, and brings the language of AI catastrophe into congressional debate. What to watch is whether Democrats actually form a select committee on AI, and whether his auditing, whistleblower and kill-switch demands survive a divided Congress as binding text.