22.Capstone: An End-to-End Learned Robot in Rust

22.1 The specification, written before any code

Kober's survey closes with a case study, and the discipline it models is worth copying: write the specification before touching the implementation. Most robot-learning projects fail at this step rather than at training.

Task. Ferris patrols a course of randomly-generated rough terrain with scattered obstacles, visiting a sequence of waypoints in order. He must recover from external pushes and from stumbles, and complete the circuit within a time budget.

Success criteria, stated so a disinterested party could check them:

• Visits all four waypoints in order, within 3 minutes.
• Survives at least three lateral pushes of 150 N applied for 0.2 s at random times.
• No base-ground contact (a "fall") at any point.
• Mean cost of transport below 1.2.

Explicit non-goals. Not: manipulation of any kind, navigation of unmapped buildings, operation among people, or terrain types outside the generated distribution. Stating these matters — an unbounded task specification is how projects acquire scope that no evaluation can cover.

22.2 The formal problem

Now the MDP, with each choice traceable to a chapter.

Observation (48 dimensions; what the deployed student sees):

Privileged observation (teacher only, +31 dimensions): terrain heights on a 4×4 grid under the body, true friction, per-foot contact states, applied external force, and true base linear velocity. Discarded after distillation (Ch 15).

Action. Twelve joint position targets at 50 Hz, tracked by a 1 kHz PD loop with $K_p = 40$, $K_d = 1$. Mid-level, per Chapter 17 and universal practice in Chapter 18.

Reward. The Chapter 18 locomotion terms, plus task terms:

Waypoint progress is potential-based — $\gamma\Phi(s') - \Phi(s)$ with $\Phi = -\text{dist}$ — so by Chapter 14's Theorem 14.1 it accelerates learning without changing the optimal policy. The waypoint bonus is the actual objective.

Randomization (Ch 15): link masses ±20%, friction 0.4–1.2, motor $K_p$ ±30%, latency 0–30 ms, IMU noise, terrain roughness scheduled by curriculum, and random pushes of 50–200 N.

22.3 The pipeline

Five stages, each an earlier chapter cashed in.

1. Environment — Ferris in rapier3d, procedural terrain, 4096 parallel instances via rayon (Ch 13, 15).
2. Teacher — PPO with privileged observations, terrain curriculum with per-environment promotion (Ch 10, 18).
3. Student — GRU policy over a 15-step observation history, trained by DAgger-style distillation on its own state distribution (Ch 15, 16, 19).
4. Evaluation — held-out terrain seeds, the push protocol, L0–L5 rubric with Wilson intervals (Ch 14, 20).
5. Deployment — native binary plus a WASM build with live telemetry.

use clap::Parser;
use serde::{Deserialize, Serialize};
 
#[derive(Parser)]
#[command(name = "ferris-capstone")]
struct Cli {
    /// Path to the experiment config. Everything reproducible lives here.
    #[arg(short, long, default_value = "configs/patrol.toml")]
    config: String,
 
    #[arg(short, long)]
    stage: Stage,
 
    /// Overrides the config seed; recorded in the results either way.
    #[arg(long)]
    seed: Option<u64>,
}
 
#[derive(Clone, Copy, clap::ValueEnum)]
enum Stage { Teacher, Student, Evaluate, Export }
 
#[derive(Serialize, Deserialize, Clone)]
struct ExperimentConfig {
    seed: u64,
    env: EnvConfig,
    randomization: RandomizationRanges,
    curriculum: CurriculumConfig,
    reward: Vec<RewardTermConfig>,   // weights live in config, never in code
    ppo: PpoConfig,
    distillation: DistillConfig,
    evaluation: EvalConfig,
}
 
fn main() -> anyhow::Result<()> {
    let cli = Cli::parse();
    let raw = std::fs::read_to_string(&cli.config)?;
    let mut cfg: ExperimentConfig = toml::from_str(&raw)?;
    if let Some(s) = cli.seed { cfg.seed = s; }
 
    // Hash the exact config into every artifact, so a result can always be
    // traced back to the settings that produced it.
    let config_hash = blake3::hash(raw.as_bytes()).to_hex().to_string();
    tracing::info!(seed = cfg.seed, config = %config_hash, "starting");
 
    match cli.stage {
        Stage::Teacher => {
            let mut envs = FerrisVecEnv::new(&cfg.env, &cfg.randomization, cfg.seed)?;
            let mut teacher = PrivilegedPolicy::new(&cfg.ppo);
            let mut curriculum = TerrainCurriculum::new(&cfg.curriculum);
            train_ppo(&mut teacher, &mut envs, &mut curriculum, &cfg, &config_hash)?;
        }
        Stage::Student => {
            let teacher = PrivilegedPolicy::load("artifacts/teacher.mpk")?;
            let mut student = RecurrentPolicy::new(&cfg.distillation);
            // DAgger, not behaviour cloning: roll out the STUDENT, label with
            // the teacher. Chapter 16 explains why this distinction decides it.
            distill_on_policy(&teacher, &mut student, &cfg, &config_hash)?;
        }
        Stage::Evaluate => {
            let student = RecurrentPolicy::load("artifacts/student.mpk")?;
            let report = evaluate(&student, &cfg.evaluation, cfg.seed)?;
            report.print_with_wilson_intervals();
            report.write_json("artifacts/evaluation.json")?;
        }
        Stage::Export => export_wasm("artifacts/student.mpk", "dist/")?,
    }
    Ok(())
}

22.4 Evaluation, and an honest verdict

Chapter 14 set the standard; here it is applied to our own work.

Protocol. 200 episodes on terrain seeds disjoint from training, with the push protocol applied. Five independently trained policies (five training seeds), reported separately rather than pooled.

Results, as they would be reported:

The intervals are Wilson score intervals, appropriate near the boundary where the normal approximation misleads. Across the five training seeds, completion ranged from 81% to 91% — a spread worth reporting, because a paper claiming 91% from one seed would be reporting the best of five.

22.5 Failure modes, with post-mortems

The failures are more instructive than the successes, and each traces to a specific decision.

Stuck at a gap (11 of 24 failures). Ferris approaches a gap wider than his stride and oscillates at the edge. Diagnosis: the terrain curriculum generated gaps up to 40 cm, but the waypoint-progress reward rewards approaching the goal, so the policy is pulled toward an edge it cannot cross. Fix: either a lateral-detour skill (Ch 19's hierarchy) or a curriculum that pairs each gap with a traversable bypass so the policy learns detouring is acceptable.

Push recovery failure on low friction (7 of 24). Pushes at $\mu = 0.4$ cause a slide into a fall. Diagnosis: the randomization range covers $\mu \in [0.4, 1.2]$, but pushes were sampled uniformly over time rather than conditioned on friction, so the low-friction-plus-push corner is rare in training. Fix: stratify the randomization so hard combinations are sampled deliberately.

Timeout on rough terrain (4 of 24). Completes the circuit but past the budget. Diagnosis: the velocity-tracking weight is too low relative to the effort penalty on rough ground, so the policy trades speed for economy. Fix: condition the commanded velocity on remaining time, or reweight — a Chapter 18 reward-mixer decision.

Distillation gap (2 of 24). The student fails where the teacher succeeds. Diagnosis: the 15-step history is insufficient to infer terrain that the teacher observed directly. Fix: a longer history, or an explicit terrain-estimation auxiliary loss.

22.6 What you have

The repository is a template. Fork it, replace the URDF and the reward terms, and the pipeline carries over: config-driven experiments with hashed provenance, a parallel rollout farm, teacher–student training, evaluation with intervals, and a browser demo for showing people what you built.

Where to go next, in rough order of value: put it on hardware, because every assumption in §22.2 is a hypothesis until a physical robot tests it. Add a manipulator and confront Chapter 20's difficulties. Replace the hand-specified waypoint sequence with learned skills and take on Chapter 19's open question. Or take the pipeline to a task where simulation does not work, and find out what Chapter 16's methods can do.

22.7 A closing note

Twenty-two chapters ago, Rusty drove into a concave trap and stayed there, running a controller that was correct in the sense that mattered to its author and useless in the sense that mattered to the robot.

The distance from there to here is not a collection of algorithms. It is a way of thinking: state the problem formally, know which assumptions your robot violates, choose the representation before the optimizer, measure against a classical baseline, and report what you actually established rather than what you hoped.

The four curses are still there. They will be there for your project too. Knowing their names, and which technique manages each, is what this book was for.

References