How Mooncake Enables Efficient Batched Rollout Data Transfer for the Miles Reinforcement Learning System
Blog Mooncake for Miles: From Fragmented Rollout Data to Efficient Bulk I/O Reinforcement learning for large language models combines two very different workloads: rollout generation and model training. During rollout, inference workers run the current policy on a set of pro... Mooncake community
AI BriefingWith the adoption of decoupled architectures for large-scale LLM reinforcement learning, transferring rollout data from inference to training has become a bottleneck. For the heterogeneous, fragmented rollout data in the Miles system (such as tokens, loss_masks, and rollout_log_probs in list[np.ndarray] form), Mooncake uses batched I/O optimizations to enable efficient transfer while meeting requirements for efficiency, correctness, scalability, flexibility, and predictable handoff latency.
Source: LMSYS Blog · lmsys.org