Tsinghua and Infinigence Open-Source RPent, an Embodied Agent Stack for Real Robots

RPent, an embodied-agent infrastructure project jointly launched by Tsinghua University, Infinigence AI and Zhengxing Innovation, was open-sourced today, with the code published at github.com/RLinf/RPent, according to a report from QbitAI.

The project targets the physical world. It connects the task understanding and planning ability of a general foundation model with the fine manipulation skills of specialist models such as VLAs, and adds memory, tools and robot interfaces on top, forming a full loop from perception and decision to execution and feedback-driven correction.

The timing is no accident. The report notes that as GPT-6 Astra begins to be tested on real robot operation, the paradigm of using a large model as the decision-making brain is moving out of software and into hardware. But robots cannot roll back a bad action: the environment keeps changing, the state is never fully observable, and grasping, assembly and insertion demand specialised models and control.

The system reports 92.6% task success on LIBERO-PRO along with more than 7x faster end-to-end task completion. Its hardware demonstrations include pouring steel balls into a bowl, two arms wiping a plate together, and clearing occluders to find a spoon hidden under a bowl. Crucially, the report says these tasks were not handled by a separate policy trained for each scene: the change is that the robot starts understanding the task, calling up the right capability and adapting its actions to the environment.

Efficiency comes from Flash Mode and Task Cards. After an exploration succeeds, RPent compresses the validated task flow into a Task Card that stores task stages, action primitives, key goals and check conditions rather than fixed coordinates that only work in one scene. In Flash Mode the system reuses that card, re-locating objects from vision and adjusting actions, and only falls back to the planner when a check fails or the environment drifts away from the flow.

In 200 LIBERO Object evaluations, Flash Mode cut average execution time from 283.6 seconds to 40.9 seconds, roughly a 7x speed-up for a 3.5 percentage point drop in success rate.

Memory is the other pillar. RPent organises experience into three layers with scope, type, confidence and supporting evidence, so a one-off success does not silently become a capability, and conflicting records are kept and isolated. In a LIBERO-Pro Goal position-swap experiment, adding the memory mechanism lifted success from 31.0% to 87.0%, and memory assets can be distributed to other agents.

Architecturally the project splits into user, intelligence, interface and environment layers, with an Agentic Planner paired with reusable Action Primitives that wrap VLA and WAM models plus programmatic skills into standard tools. It already covers the LIBERO-PRO, RoboCasa, RoboTwin and RoboDojo simulators and Franka, dual-arm Franka, YAM and SO101 hardware, and exposes MCP, RPC and MHS interfaces so decision-making is decoupled from physical execution.

On technical direction, end-to-end VLA policies manipulate precisely but degrade on long-horizon, perturbed tasks, while pure large-model agents still lag on sub-centimetre precision, latency and getting better with use. RPent does not split the difference; it decomposes embodied intelligence into layers, with the general model planning, specialist models acting, memory accumulating experience and the framework wiring models, tools and the physical environment together. The underlying algorithm traces back to the team's Harness VLA work published in July, and RPent is also publishing a leaderboard comparing different technical routes and base models.

Whether the community can reuse Task Cards and memory assets across different robots, turning a one-off deployment into a system that keeps improving, is the question worth watching next.