PPO Booster Landing
Reinforcement learning — rocket landing
A continuous-control PPO agent (Stable-Baselines3) that learns to land a simplified 2D rocket booster — trained under domain randomization for wind, sensor noise, and engine/mass/drag variation so the policy holds up away from its exact training conditions, not just on it.
Problem
Land a simplified 2D rocket booster on a pad using a learned policy instead of hand-tuned control — and make the resulting controller robust to the kind of variation a real landing would see: wind, sensor noise, engine and mass variation, drag.
What I built
A custom Gymnasium environment (BoosterLandingEnv) over a 7-dimensional state — position (x, z), velocity (vx, vz), orientation (tilt theta, angular velocity omega), and fuel — under a continuous action space: throttle in [0, 1] and gimbal in [-1, 1]. The reward is shaped around a smooth descent, minimal lateral error, staying upright, and landing fuel-efficiently, balancing stability, precision, and control smoothness.
RobustBoosterLandingEnv layers domain randomization on top: stochastic wind and turbulence, engine-thrust variation, mass perturbations, drag scaling, and sensor noise, all driven by a curriculum parameter d ∈ [0, 1] that widens both the initial-condition distribution and the dynamics uncertainty as it increases. That forces the policy to learn control strategies that are invariant to the exact physics, not overfit to one deterministic system.
Training
PPO (Stable-Baselines3) — clipped objective, actor-critic, vectorized parallel rollouts, trained with curriculum progression over difficulty and evaluated across multiple difficulty bands. Final policy success rate: ~93% at difficulty 0.75, ~90% at 0.85, ~73% at 1.00.
Try it
Click anywhere in the sky below to spawn the booster and watch the trained policy fly it in — throttle and gimbal recomputed every timestep. You can adjust wind, turbulence, sensor noise, engine power, mass, drag, and the initial state live, with full state and action telemetry alongside the render.
Deployment
The deployed model is a frozen policy — inference-only, no training happening server-side. A forward pass takes roughly 0.5–2 ms, so it runs comfortably on CPU with no GPU needed: on the order of 1–5% of a CPU core per active session, ~50–150 MB of memory each. Sessions are stateless, so a modest 8-core server should hold up under several dozen concurrent users.