Safeworld exits stealth with over $12M to prove generative-AI robots are safe

The defining trend in robotics is handing control to a generative AI model, but that architecture is not predictable the way traditional algorithms are, which raises an awkward question: how do you know a brand-new humanoid is safe?

Safeworld is emerging from stealth today with a seed round of more than $12 million, led by Shine Capital and a16z Speedrun, with additional investment from Box Group, Carnegie Mellon University Endowment, Innovation Endeavors and SV Angel. The company was founded by Dr. Ding Zhao, who directs the Safe AI lab at Carnegie Mellon University, alongside veteran startup executive Kyle Wong and machine learning engineer Simo Rachidi.

Zhao frames the problem in two halves: advanced generative AI probabilistic evaluations, meaning how you underwrite the risk of a probabilistic system, and the harder trust problem. Both are needed before a robot can actually be deployed, he argues.

Safeworld's specialty is evaluating a robotic control system inside simulations populated with realistic human models. It builds a digital version of a real environment, such as a blind corner in a factory, in a simulator like Genesis or MuJoCo, inserts the robot driven by its actual software, and then runs thousands of scenarios in which human models encounter it.

Wong describes the questions that matter in practice: at what speed, or with what stopping distance, will the robot avoid colliding with a person, and if that person is carrying boxes, will the robot detect them at all? Tripping and falling also gets heavy simulation coverage, because otherwise, as Wong puts it, you would have to keep tripping and falling in front of the robot yourself.

The approach resembles the tools robot builders already use internally, but the founders bet that manufacturers will still want a third party to validate their work, partly so competitors can share information about safety cases. Zhao stresses that the real risk is not a robot performing in a vacuum; it is the robot deployed at scale, alongside people who may never have operated one before.

Gritt Robotics CTO Vishal Dugar is already partnering with Safeworld on safety simulations. His robots help workers install photovoltaic panels at industrial-scale solar farms and aspire to take on more complex construction tasks, and he argues that the safety of such systems is very hard to prove formally with math and must instead be established empirically.

It is still early days for both Safeworld and generative AI in robotics, and the company is still deciding on the best model for its product, whether an external platform or a services-based approach. Zhao is nonetheless bullish, saying the team will probably be the first profitable company in this field, because anyone who wants to deploy a robot will have to pay it to handle the safety problem.