LunarLander-v3 in 32 parameters (max-safety)

A 32-parameter single-layer neural network that solves Gymnasium LunarLander-v3: 231.2 mean return, 99.7% landings, 60/60 seeds, crashing in only ~0.25% of episodes.

logits = Linear(8 -> 4, bias=False)(obs)     # 32 parameters
action = argmax(logits)

The weights shipped here are the max-safety variant (see Choice of objective below).

Evaluation

60 evaluation seeds x 100 episodes = 6000 episodes, deterministic greedy policy:

Reference random policy on this env is about -100..-500; the "solved" threshold is 200. Reproduce with:

pip install -r requirements.txt
python verify.py --episodes 100 --seeds 60

What "landing rate" means. An episode ends with the environment's own ±100 signal: +100 when the lander comes to rest (both legs grounded, low speed, small angle) — a safe landing; -100 when the lander's body hits the ground or it flies off screen — a crash (see LunarLander._get_reward). "Landing rate" is the fraction of episodes that received the +100 verdict. Measured exactly on the 6000 episodes:

The return >= 100 proxy (99.67%) agrees with the environment's own verdict on 6000 / 6000 episodes. python landing_check.py reproduces this cross-check.

Because a safe landing is worth roughly +100..+320 (the terminal +100 plus shaping and leg contacts), the "solved" criterion (mean >= 200) is equivalent to "lands almost every time".

Choice of objective (why 231 and not 247)

In this environment average score and reliability trade off, and the shipped weights pick maximum safety. Starting from the mean-optimal teacher (247.2 mean, 5.9% crashes) the weights were refined against a crash-penalised objective (mean - 500 * crash_rate) on a separate train seed set, evaluating on held-out seeds. On the 60 canonical evaluation seeds:

The crashes are not caused by a few "bad seeds" — all 60 seeds pass, and failures are spread over episodes (2-11 crashes out of 100 per seed); they start from faster, more tilted spawns. They are also policy-addressable: the same 32-parameter class contains a policy with ~20x fewer crashes, so 5.9% is a compromise of one point, not a hard limit of the class. Reliability is bought with mean return — within the same 32 parameters.

Earlier revisions (crash-averse 239.5 / 0.6%; mean-optimal 247.2 / 5.9%) are preserved in this repository's git history.

Reproducibility (why not to over-trust the crash rate)

The max-safety weights above are start 1 of an experiment with 6 independent optimisation runs of the same objective (mean - 500 * crash_rate). The six runs land at very different points, and several barely move from the teacher at all:

All six still solve every seed (>= 230 mean, 60/60), so the regime is safe — but the crash rate is noisy: only 2 of 6 starts reached <= 2%. Reproducing a specific crash rate is therefore not guaranteed at this small optimisation budget; run more starts (or a larger budget) if you need a target reliability. Two starts Pareto-dominate the teacher on both axes (start 2: 250.1 / 2.0%; start 4: 258.1 / 3.5%), i.e. higher score and fewer crashes than the mean-optimal policy.

Out-of-sample check (fresh seeds)

Because the shipped weights were selected as the best of those 6 runs, the numbers above were re-checked on entirely fresh seeds — three disjoint blocks of 60 seeds x 100 episodes (18,000 episodes per policy) that were used neither for training nor for selection:

The shipped model reproduces its canonical numbers on fresh seeds (231.4 vs 231.2 mean; 0.17% vs 0.25% crashes), so its low crash rate is a property of the weights, not of the seed set used to pick them. The two variants kept in history also hold up out-of-sample and Pareto-dominate the mean-optimal teacher on both axes (higher score and fewer crashes).

Usage

Standalone (only torch + safetensors):

from model import TinyLunarPolicy

policy = TinyLunarPolicy.from_pretrained("Dimitrius174/lunarlander-lin32")
# or: TinyLunarPolicy.from_safetensors("model.safetensors")

action = policy.act([0.0, 1.4, 0.0, -0.6, 0.0, 0.0, 0.0, 0.0])   # int in 0..3

With transformers (custom code via auto_map):

import torch
from transformers import AutoModel

model = AutoModel.from_pretrained("Dimitrius174/lunarlander-lin32", trust_remote_code=True).eval()
logits = model(torch.tensor([0.0, 1.4, 0.0, -0.6, 0.0, 0.0, 0.0, 0.0]))   # shape (4,)
action = int(logits.argmax())

Gymnasium loop:

import gymnasium as gym
from model import TinyLunarPolicy

env = gym.make("LunarLander-v3")
policy = TinyLunarPolicy.from_pretrained("Dimitrius174/lunarlander-lin32")
obs, _ = env.reset(seed=0)
done = False
while not done:
    obs, _, term, trunc, _ = env.step(policy.act(obs))
    done = term or trunc

Observation and action space

obs (8,): x, y, vx, vy, angle, angular_velocity, leg_left_contact, leg_right_contact. action (4): do nothing, fire left engine, fire main engine, fire right engine.

Is 32 really minimal?

In a capacity study of small networks fitted to this policy (same data, optimiser and steps; verified on the same 60 seeds), the findings were:

• the linear 32-parameter model was the only one that reproduced the policy under every regime tried (activation, initialisation, training budget);
• a 1-hidden-layer tanh MLP needed ~43-56 parameters (43 borderline, 56 reliable);
• ReLU-family MLPs at the same budget were unreliable (some seeds solve, most do not), so extra non-linearity costs parameters and training fragility;
• beyond ~56 parameters nothing improves (the environment, not the network, sets the ceiling).

Buying a linear layer instead of an MLP is therefore the cheap and robust choice.

Files

License

MIT. See LICENSE.