20.Learning Manipulation

20.1 Why manipulation is the hard one

Chapter 18 listed four properties that made locomotion tractable. Manipulation fails on all four, and the symmetry is worth spelling out.

The dynamics do not simulate well. Manipulation is made of the contact interactions that Chapter 15 identified as simulators' weakest point — sustained sliding, friction that decides outcomes, compliance, deformation. A locomotion policy needs contact to be roughly right; a manipulation policy needs it to be right in detail.

Rewards are sparse and hard to write. "Did the assembly succeed?" is one bit, arriving after a long sequence. Dense shaping requires knowing what intermediate progress looks like, which for contact-rich tasks is often exactly what you do not know.

Failure is expensive. A dropped object may break; a mis-inserted peg can damage the workpiece or the robot; a manipulator near a person is a safety case.

Objects are diverse. A quadruped's world is terrain — endless variation over one type of thing. A manipulator's world contains objects it has never seen, whose mass, friction and compliance it cannot observe.

20.2 Grasping: the analytic criterion and its learned approximation

Grasping has the cleanest theory in manipulation, and it is worth knowing even though modern systems learn rather than compute it.

A contact exerting force $f$ at position $p$ produces a wrench — force plus torque — of $w = (f, p \times f)$. With Coulomb friction, the forces a contact can exert lie in a cone of half-angle $\arctan\mu$ about the surface normal.

A grasp has force closure when its contact wrenches positively span the wrench space: any disturbance can be opposed by some non-negative combination of contact forces. In the plane this reduces to a readable test — no half-plane may contain all the wrench generators.

Force closure is binary; quality is continuous. The Ferrari–Canny metric takes the radius of the largest wrench ball the grasp can resist — the worst-case disturbance it survives. That is the difference between a grasp that technically holds and one that survives being carried across a room.

What modern systems actually do is skip the geometry. A network takes a depth image, proposes candidate grasps, and scores each one — approximating the force-closure computation without ever reconstructing the object. This is the payoff for Chapter 9: grasp selection is exactly the setting where value-based methods belong, because the action space is a few hundred discrete candidates and the argmax is both tractable and correct.

20.3 Contact-rich manipulation and impedance actions

Insertion is the canonical contact-rich task: a peg into a hole with clearance smaller than the robot's positioning error. Pure position control fails by construction — commanding a position the peg cannot reach produces enormous contact forces and a jammed or damaged part.

The answer, from Chapter 13's operational-space material, is impedance control. Instead of commanding position, command a target position and a stiffness, making the robot behave like a programmable spring:

$F = K_p (x_d - x) + K_d (\dot x_d - \dot x).$

Low stiffness in the directions where contact is expected lets the environment guide the part; high stiffness along the insertion axis maintains the push.

Why planning through contact is intractable is worth stating precisely, because it explains why learning is attractive here. With $n$ contact points, each is separated, sticking, or sliding — $3^n$ discrete modes, each with different dynamics. A peg-in-hole with 8 potential contacts has $6561$ modes. Planning must search over mode sequences, and the search is combinatorial in a quantity that grows with the task's contact richness. Learned policies sidestep the enumeration entirely by never representing modes explicitly.

20.4 In-hand manipulation: what dexterity cost

Reorienting an object within the hand, using the fingers alone, is the hardest thing manipulators do — many contacts, all of them making and breaking, with the object's pose partially occluded by the hand doing the manipulating.

The result that defined the area used no new algorithm. It used everything from Part III, at maximum intensity: extreme domain randomization (masses, frictions, object dimensions, gravity, actuator dynamics, visual appearance), a recurrent policy performing the implicit system identification of Chapter 15, distributed PPO from Chapter 10, and enormous compute.

20.5 Where the techniques land

Assembling Part IV's map for manipulation:

The pattern in Tang's tables is consistent: manipulation successes lean on expert data far more than locomotion successes, which lean on simulation. That is exactly what the four criteria of §20.1 predict — when simulation is untrustworthy and reward design is hard, human demonstrations supply both the data and the specification.

use nalgebra::{Vector3, Matrix3};
 
/// The policy's action: where to go, and how hard to insist on getting there.
pub struct ImpedanceAction {
    pub target_pos: Vector3<f64>,
    /// Per-axis stiffness in the TASK frame — low along expected contact
    /// directions lets the environment guide the part.
    pub stiffness: Vector3<f64>,
}
 
pub struct ImpedanceWrapper {
    kp_range: (f64, f64),
    damping_ratio: f64,   // ζ; 1.0 = critically damped
}
 
impl ImpedanceWrapper {
    /// Map a normalized policy output in [−1, 1] to a physical command.
    pub fn decode(&self, raw: &[f64], current_pos: &Vector3<f64>) -> ImpedanceAction {
        let delta = Vector3::new(raw[0], raw[1], raw[2]) * 0.05;   // ±5 cm
        let stiffness = Vector3::from_iterator((3..6).map(|i| {
            let t = (raw[i] + 1.0) / 2.0;                          // → [0, 1]
            self.kp_range.0 * (self.kp_range.1 / self.kp_range.0).powf(t)  // log-spaced
        }));
        ImpedanceAction { target_pos: current_pos + delta, stiffness }
    }
 
    /// Cartesian impedance law, mapped to joint torques by Jᵀ (Chapter 13).
    /// Damping is derived from stiffness so the system stays critically damped
    /// as the policy varies Kp — otherwise a stiffness change causes ringing.
    pub fn to_joint_torques(
        &self,
        action: &ImpedanceAction,
        x: &Vector3<f64>,
        xdot: &Vector3<f64>,
        jacobian: &nalgebra::Matrix3<f64>,
        mass_est: f64,
    ) -> Vector3<f64> {
        let kp = Matrix3::from_diagonal(&action.stiffness);
        let kd = Matrix3::from_diagonal(&action.stiffness.map(|k| {
            2.0 * self.damping_ratio * (k * mass_est).sqrt()
        }));
 
        let force = kp * (action.target_pos - x) - kd * xdot;
        jacobian.transpose() * force
    }
}

20.6 Reporting results honestly

Manipulation is where evaluation goes wrong most often, so Chapter 14's methodology deserves restating with the specifics.

State the object set. "90% success" over five known objects in fixed poses is a different claim from 90% over fifty novel objects in clutter. Both may be worth publishing; conflating them is not.

Give an interval. With 20 trials and 18 successes, the point estimate is 90%. The Wilson score interval — appropriate near the boundary where the normal approximation fails — is roughly [70%, 97%]. Reporting 90% alone claims precision the experiment did not buy.

Report the failures. Which two failed, and how? A grasp that slipped is a different problem from one that never closed.

Report the human cost. How many resets, and did a person perform them? For manipulation this is usually the binding constraint on the entire experiment, and omitting it hides the real cost of the method.

20.7 Chapter bridge

Part IV is complete. Locomotion won because its dynamics simulate and its rewards shape. Navigation is unsettled between learned and classical because each fails differently. Manipulation lags because contact, diversity and cost all cut against it — and where it succeeds, it does so by leaning on human data rather than simulation.

Three competencies remain in Tang's taxonomy, and they share a property: the environment contains other agents. Chapter 21 takes on human–robot interaction, where the unmodelable part of the world is a person; multi-robot systems, where coordination is a Dec-POMDP with unpleasant complexity; and the frontier that has moved fastest since the survey was written — foundation models supplying the priors, goals and even rewards that this book has so far asked engineers to write by hand.

References