DisagMoE：用解耦 Attention 和 FFN 打通 MoE 训练的 all-to-all 瓶颈
一篇关于 DisagMoE 的中文阅读笔记：把 attention 和 FFN 分别放到独立 GPU 池，用 AF-Pipe 调度和 M2N 通讯原语把两侧拼起来，从而把 MoE 训练里的 all-to-all 瓶颈藏进计算之下。
DisagMoE: Disaggregating Attention and FFN to Beat the MoE All-to-All Bottleneck
Posted on
Edited on
In
LLM Training
,
Distributed Systems
A detailed technical review of DisagMoE, which disaggregates attention and FFN layers onto separate GPU pools and stitches them together via the AF-Pipe schedule to hide the MoE all-to-all bottleneck during training.
A detailed technical review of DisagMoE, which disaggregates attention and FFN layers onto separate GPU pools and stitches them together via the AF-Pipe schedule to hide the MoE all-to-all bottleneck during training.
DAPO：大规模开源 LLM 强化学习系统
Posted on
In
RL Training
,
LLM Training
一篇关于 DAPO 的中文阅读笔记：它把 Clip-Higher、动态采样、token-level loss 与 overlong reward shaping 组合成可复现的大规模 LLM 强化学习配方。
一篇关于 DAPO 的中文阅读笔记：它把 Clip-Higher、动态采样、token-level loss 与 overlong reward shaping 组合成可复现的大规模 LLM 强化学习配方。
DAPO: An Open-Source LLM Reinforcement Learning System at Scale
Posted on
In
RL Training
,
LLM Training
A detailed technical review of DAPO, an open-source large-scale reinforcement learning recipe for reasoning LLMs using Clip-Higher, dynamic sampling, token-level loss, and overlong reward shaping.
A detailed technical review of DAPO, an open-source large-scale reinforcement learning recipe for reasoning LLMs using Clip-Higher, dynamic sampling, token-level loss, and overlong reward shaping.
MASPO：面向 LLM 多智能体系统的联合提示词优化
Posted on
In
AI Agents
,
Prompt Optimization
一篇关于 MASPO 的中文阅读笔记：它用 local、lookahead 与 global 三类信号联合优化 LLM 多智能体系统中的角色提示词。
一篇关于 MASPO 的中文阅读笔记：它用 local、lookahead 与 global 三类信号联合优化 LLM 多智能体系统中的角色提示词。
MASPO: Joint Prompt Optimization for LLM-based Multi-Agent Systems
Posted on
In
AI Agents
,
Prompt Optimization
A detailed technical review of MASPO, a joint prompt optimization method for multi-agent LLM systems that balances local, downstream, and global rewards.
A detailed technical review of MASPO, a joint prompt optimization method for multi-agent LLM systems that balances local, downstream, and global rewards.
Tutti：让基于 SSD 的 KV Cache 真正适用于长上下文 LLM Serving
Posted on
Edited on
In
ML Systems
,
LLM Inference
一篇关于 Tutti 的中文阅读笔记：它从 GPU-native KV cache object store、GPU io_uring 与 slack-aware scheduling 出发，让 SSD-backed KV cache 更适合长上下文 LLM serving。
一篇关于 Tutti 的中文阅读笔记：它从 GPU-native KV cache object store、GPU io_uring 与 slack-aware scheduling 出发，让 SSD-backed KV cache 更适合长上下文 LLM serving。
Tutti: Making SSD-Backed KV Cache Practical for Long-Context LLM Serving
Posted on
Edited on
In
ML Systems
,
LLM Inference
A detailed technical review of Tutti, a GPU-centric SSD-backed KV cache system that makes long-context LLM serving cache reuse practical.
A detailed technical review of Tutti, a GPU-centric SSD-backed KV cache system that makes long-context LLM serving cache reuse practical.
Queueing Stability for LLM Inference with KV Cache Memory Constraints
Posted on
Edited on
In
ML Systems
,
LLM Inference
A detailed technical review of a queueing-theoretic framework for predicting LLM inference stability under KV cache memory constraints.
A detailed technical review of a queueing-theoretic framework for predicting LLM inference stability under KV cache memory constraints.
Swift-SVD: Activation-Aware Low-Rank Compression for LLM Weights and KV Cache
Posted on
Edited on
In
SVD Acceleration
,
Efficient ML
A detailed technical review of Swift-SVD, an activation-aware low-rank compression method for LLM weights and KV cache that uses output covariance eigendecomposition to avoid expensive generalized SVD.
A detailed technical review of Swift-SVD, an activation-aware low-rank compression method for LLM weights and KV cache that uses output covariance eigendecomposition to avoid expensive generalized SVD.
Piper: Efficient Large-Scale MoE Training via Resource Modeling and Pipelined Hybrid Parallelism
Posted on
Edited on
In
ML Systems
,
Training Infrastructure
A detailed technical review of Piper, a resource-model-driven system for large-scale MoE training with pipelined hybrid parallelism, HALO hierarchical all-to-all, and topology-aware expert placement.
A detailed technical review of Piper, a resource-model-driven system for large-scale MoE training with pipelined hybrid parallelism, HALO hierarchical all-to-all, and topology-aware expert placement.
Low-Rank Optimization Trajectories for LLM RLVR Acceleration: A Technical Review of NExt
Posted on
Edited on
In
LLM Training
,
Optimization
A detailed technical review of NExt, a method that models low-rank optimization trajectories to accelerate reinforcement learning with verifiable rewards for large language models.
A detailed technical review of NExt, a method that models low-rank optimization trajectories to accelerate reinforcement learning with verifiable rewards for large language models.