为什么值得关注

它针对智能体强化学习中的长轨迹、工具调用和权重同步等独特挑战，提供了部分 rollout、基于句柄的数据移动以及 rollout-训练器一致性分析等内置机制，并依托生产级推理和训练引擎。

适合谁使用

从事大语言模型后训练的研究人员和工程师
管理分布式强化学习系统的机器学习基础设施团队
构建涉及工具使用和多轮环境的智能体应用的开发者

典型使用场景

使用 GRPO 或 PPO 在 GSM8K 等数据集上训练数学推理模型
训练检索增强的多轮搜索智能体（Search-R1）
通过黑盒集成（如 OpenHands/E2B）训练复杂工具调用智能体
通过尖峰重放和差异分析调试训练稳定性问题

项目优势

基于 SGLang 提供高吞吐 rollout，基于 Megatron 实现可扩展分布式训练
支持超过 300 步的长轨迹和千亿参数级的训练
通过部分 rollout、基于句柄的数据通道、前缀树合并与 MagiAttention 提升效率
提供全面的 rollout-训练器一致性检查（差异分析、路由重放、尖峰重放）

使用前须知

与 SGLang 和 Megatron 紧密耦合，需要特定的 Docker 镜像和引擎使用经验
黑盒强化学习配方仍在开发中，配置和接口可能变化
默认配方需要具备大量 GPU 资源的集群

README 快速开始

English · 简体中文

XYZ AI Lab ·
Technical Blog

AxisRL

AxisRL is an agentic RL post-training framework built on SGLang rollout, Megatron training, and real-world agent workflows.

AxisRL connects high-throughput rollout, large-scale training, weight synchronization, data movement, resource scheduling, and reproducible debugging inside one coherent framework. SGLang and Megatron remain the core serving and training engines; AxisRL handles the system layer around agentic post-training.

✨ Highlights

Built on SGLang for high-throughput rollout and Megatron for large-scale distributed training.
Used in agent RL workflows with 300+ turn trajectories and training runs at hundreds-of-billions-parameter scale.
Provides configurable policy optimization objectives including PPO, GRPO/GRPO2, GSPO, TOPR, TIS, and related variants.
Supports both white-box agent environments and black-box harness capture through an OpenAI-compatible proxy.
Reduces rollout/training idle time with partial rollout and a lightweight control plane.
Provides handle-based data movement, context packing, routing replay, mismatch analysis, and spike replay for rollout-trainer consistency.

🧭 Why AxisRL?

LLM post-training workloads are moving beyond single-turn question answering. In agentic RL, a model may interact with a long-lived environment, call tools, observe tool results, update context, and receive a reward only after several turns.

That changes the job of a post-training framework. It has to coordinate multi-turn rollout, environment state, tool calls, verifiers, reward collection, training sample construction, and weight synchronization. It also has to make training behavior observable, because small differences in tokenization, chat templates, logprobs, routing, packing, or weight sync can appear later as loss spikes, reward instability, or rollout-trainer mismatch.

AxisRL is designed for this setting: real agent workflows, SGLang rollout, Megatron training, and the system contracts between them.

🏗️ Architecture

At a high level, an AxisRL run follows this loop:

Rollout actors execute task-specific agent workflows.
SGLang workers serve model generation.
Environments, tools, verifiers, or external harnesses produce interaction records and rewards.
Megatron workers consume training sam

项目描述

AxisRL is an agentic RL post-training framework built on SGLang rollout, Megatron training, and real-world agent workflows.

axrl