nanoRL
One RL training loop that runs CartPole on a laptop and disaggregated async RLVR on a GPU cluster. About 1,800 lines across 7 files. No Ray, no TRL, no DeepSpeed. It's small enough to read in an afternoon, and it's meant to be forked, not imported.
About
One RL training loop that runs CartPole on a laptop and disaggregated async RLVR on a GPU cluster. About 1,800 lines across 7 files. No Ray, no TRL, no DeepSpeed. It's small enough to read in an afternoon, and it's meant to be forked, not imported.
Languages
Contributors1
No features listed.
Comments Theme
