AReaL 2.0 Goes Open Source: Building RL Infrastructure for Self-Evolving AI Agents

On July 2, the open-source reinforcement learning infrastructure project AReaL officially released version 2.0. AReaL aims to bridge the gap between foundational model training and modern agent applications by providing efficient RL training support for agent scenarios. The new release targets agents already operating in real business environments, offering a system infrastructure that lets agents continuously learn while being used.

Through AReaL 2.0, the interaction processes generated by agents completing real tasks can be recorded, organized, and fed into subsequent training workflows to continuously optimize the underlying model, allowing agents to grow stronger through use while maintaining safety and control. This addresses a growing problem: agents work every day but rarely learn from that work. In real business contexts, agents generate valuable experience data — which tasks performed well, where tool calls failed, why users were dissatisfied — but this information is mostly saved as logs and rarely converted into capability improvements.

AReaL 2.0 solves the problem of how agents can continue to improve after deployment. Developers don't need to rebuild their agents; they simply route their agent's original requests through AReaL 2.0's unified inference gateway to access the online RL pipeline. This means agent improvement no longer depends solely on manually constructed data, offline training, and redeployment — real-task conversations, tool calls, execution results, and feedback signals can all become training material.

This is especially important in enterprise scenarios, where agents face real, complex, and ever-changing tasks: codebases get updated, business processes are adjusted, user needs evolve, and tools and systems change. If an agent's capabilities are essentially fixed once deployed, it cannot adapt to its real environment over the long term. Additionally, continuous learning in real business environments requires careful consideration of access control, data anonymization, isolation, and auditing, since agents may interact with code, customer information, knowledge bases, and internal systems. AReaL 2.0 introduces an agent-trajectory data proxy mechanism to manage real task data safely and controllably.

AReaL was co-founded in 2024 by teams from Ant Group, Tsinghua University, and Hong Kong University of Science and Technology. In May 2026, AReaL was spun out from Ant's InclusionAI as an independent open-source community and joined the PyTorch Foundation Ecosystem project. The community has also gained support from Huawei Cloud and MindLab, among others.

AReaL 2.0's technical report and code are now available on GitHub. Going forward, AReaL will continue to iterate on online RL, automated evaluation, and multimodal agent training, working with the community to advance the self-evolving agent ecosystem.