Pixar's RenderMan tutorial credits Hesai's Kosmo for real-world 3D Gaussian splatting capture

Pixar's official RenderMan tutorial on 3D Gaussian splatting now credits a Chinese company for the scanner behind its real-world captures: Kosmo, a handheld 3D data terminal built by lidar maker Hesai Technology.

RenderMan is Pixar's core rendering technology, older than the studio itself, and Steve Jobs backed it on the bet that computer graphics could be an art form. Its GPU-accelerated XPU architecture is now in production, and support for 3D Gaussian splatting (3DGS) — a scene representation built from many small 3D Gaussians rather than meshes — has been added.

According to the tutorial text, visual effects work happens at the pixel level, so richer data allows finer adjustments, and Kosmo lets modelers walk into a location and capture every spatial element, exact geometry and real lighting. Scanning the lamp and ball outside the Steve Jobs building at Pixar takes 10 to 20 minutes, and the resulting 3DGS data enters the production environment directly through the USD format, where artists can move, scale, adjust transparency and apply stylized looks.

Kosmo splits into a capture side and a cloud side. Kosmo One packs a custom lidar, an imaging system at up to 8K resolution, a one-inch CMOS sensor and a high-precision IMU; the four streams share one clock and trigger mechanism for microsecond-level synchronization and are fused in hardware, with extrinsics calibrated at the factory and maintained in use with IMU assistance, so each frame carries aligned point cloud, RGB pixels and pose trajectory.

In the cloud, the Kosmo Spatial Core initializes Gaussian positions from the lidar point cloud, uses depth as a geometric constraint during optimization and injects color through differentiable rasterization, producing a representation whose position, shape and color converge together. Each reconstruction also exports mesh and depth from the same coordinate system without cross-format registration, and AIGC hole-filling plus semantic segmentation turn a static scan into structured, trainable data. Hesai puts capture time at roughly 15 to 20 minutes for a 100-square-meter space.

Film and games are the most visible outlet. Game Science, the studio behind Black Myth: Wukong, used Hesai lidar to capture ancient architecture, one of the drivers behind Kosmo's development; surveying, robot training and virtual home tours round out the listed uses.

The deeper pitch is about data rather than hardware. Embodied-AI teams have had to buy sensors separately and assemble their own rigs — non-standard, unreliable, expensive and hard to iterate on — while Kosmo is sold as a factory-calibrated, integrated capture device feeding a cloud reconstruction service. Hesai suggests the business may eventually earn more from the tokens consumed producing data assets than from selling the hardware itself.

The commercial claim is unproven, and the write-up is openly enthusiastic about the company. But the underlying signal is concrete: as world models get better, demand for fresh, physically grounded real-world scans tends to grow rather than shrink, because generated scenes still struggle with true scale and physical constraint. The checkpoints to watch are how many embodied-AI customers standardize on Kosmo, and whether its data-asset platform attracts contributors beyond Hesai itself.