GrandCode: Achieving Grandmaster Level in Competitive Programming via Agentic Reinforcement Learning

The paper introduces GrandCode, a multi‑agent reinforcement‑learning system that surpasses top human competitors in live Codeforces contests. GrandCode orchestrates several specialized agents— a main solver, a hypothesis generator, a summarization model, and a test‑case generator—through a tightly coupled RL loop. Central to its training is Agentic GRPO, an extension of Group Relative Policy Optimization that provides immediate reward updates and delayed corrections, addressing off‑policy drift in long, multi‑stage rollouts. The system undergoes extensive pipeline stages: continued pre‑training on noisy competitive‑programming data, supervised fine‑tuning with verified reasoning traces, and multi‑component RL that jointly optimizes the solver, hypothesis, and summarizer. Test‑case generation combines difference‑driven and solution‑attack strategies, while online test‑time RL (with a rank‑based reward smoothing) fine‑tunes the main solver during live contests. Empirical results show GrandCode achieving first place in three recent Codeforces Rounds (1087‑1089), outperforming all human participants, including grandmasters, and achieving high accept rates and weighted scores across benchmarks. The paper also details system architecture, asynchronous training with pipelined context parallelism, dynamic difficulty‑aware batching, and comprehensive evaluations of each component.

Verification.

https://arxiv.org/html/2604.02721v2