REINFORCEMENT LEARNING FOR ROBOTICS

Reinforcement Learning for Robotics

The FCP way — from multi-armed bandits to a quadruped that learns to walk, in Rust.

Shannon Press · 2026 · 22 chapters

Foundation · Conceptual · Practical

Most reinforcement learning texts prove theorems about environments that robots do not live in. This book keeps the mathematics complete, then insists on saying which assumptions break the moment a real machine touches the ground — and what practitioners do about it.

Start readingBrowse all 22 chaptersDownload PDF

The method

Three layers, interleaved in every chapter — never one without the others.

F

Foundation

Complete mathematical formalism. Definitions, theorems, and derivations carried through to the last line — including the convergence conditions robots routinely violate, stated plainly rather than buried.

C

Conceptual

Every hard idea gets a visual you can manipulate. Drag γ and watch the horizon stretch; crank Δt until the integrator explodes; slide λ from 0 to 1 and rediscover Monte Carlo.

P

Practical

Every algorithm implemented in Rust with the best current crates — burn for learning, rapier for physics, egui for dashboards — as code that trains natively and demos in the browser.

The cast

Four robots carry every idea in the book, so abstractions always land somewhere physical.

Rusty

differential-drive mobile robot

hello-robot → gridworld → visual gridworld → lidar navigation → multi-agent

Enters in Chapter 1

Pendle

pendulum / cart-pole

classical control ↔ RL bridge

Enters in Chapter 2

Reacher

2-link planar arm

manipulation thread

Enters in Chapter 3

Ferris

quadruped

locomotion & sim-to-real thread

Enters in Chapter 15

The arc

Part IFoundations of Sequential Decision-MakingChapters 1–7Sutton & Barto's spine, retold with robots and interactive math.
Part IIScaling Up: Function Approximation & Deep RLChapters 8–12From tables to tensors — the leap robots require.
Part IIIThe Robotics SideChapters 13–17Kober's bridge, rebuilt with modern materials.
Part IVCompetencies: RL on Real RobotsChapters 18–20Tang's taxonomy as deep dives: what worked, why, and rebuilt in Rust.
Part VFrontiers & CapstoneChapters 21–22Where the field is going — and one project that uses all of it.

Built on

Four works form the spine: Sutton & Barto's Reinforcement Learning: An Introduction for the mathematics, Kober, Bagnell & Peters' 2013 survey for the robotics bridge, Tang et al.'s 2024 survey of real-world successes for the modern taxonomy, and Akinola's lectures for pedagogy. Where those stop, this book continues — PPO and SAC as workhorses, world models, offline RL, teacher–student sim-to-real, and foundation-model frontiers — always grounded in the same spine.