Arena SARSA

Back up the current checkpoint, continue the existing model for a controlled run, and record its starting performance.

Copy-Item artifacts\checkpoints\sarsa_q_table.npy `
  artifacts\checkpoints\sarsa_q_table.day1.backup.npy
python scripts/train.py --episodes 100 --resume --seed 1000
python scripts/evaluate_model.py --episodes 20 --steps 300 --seed 1000

Record average reward, survival steps, and collision-free rate in accuracy.md.

Drive 20–50 careful human episodes. Save smooth, collision-free episodes and discard poor demonstrations.

python scripts/play_human.py
python scripts/train.py --episodes 200 --resume --warm-start --seed 2000

Demonstrations are saved as JSONL in data/human_demonstrations/. Warm-start gives the table a human-guided starting bias.

Expose the policy to different lane, speed, gap, and rear-traffic states without changing the state representation.

python scripts/train.py --episodes 300 --resume --vehicles-count 20 `
  --vehicles-density 1.2 --duration 400 --seed 3000

Use different seeds, traffic counts, densities, and longer episodes so the table sees more situations.

Compare learning rates around 0.05–0.10, gamma around 0.95–0.99, and exploration ending around 0.02–0.05.

$env:ARENA_STORAGE_DIR = "D:\arena-experiment-day4"
python scripts/train.py --episodes 300 --seed 4000
python scripts/evaluate_model.py --episodes 20 --steps 300 --seed 4000
Remove-Item Env:ARENA_STORAGE_DIR

Keep the configuration with the best consistent collision-free rate and survival, not just the highest single reward.

Use moderate target speeds, retain the lane-change penalty, and keep the collision penalty strong.

python scripts/train.py --episodes 300 --resume --duration 400 --seed 5000
python scripts/evaluate_model.py --episodes 30 --steps 400 --seed 5000

Do not select a model only because it drives faster. Stable survival matters more than raw speed.

Test conditions that were not used for the main training run:

• Different random seeds
• More vehicles and higher density
• Different starting lanes
• Longer episodes
• Different target speeds

Reject models that perform well only on familiar seeds or light traffic.

Run a larger greedy evaluation, compare it with Day 1, and publish only a checkpoint that improves consistently.

python scripts/evaluate_model.py --episodes 100 --steps 300 --seed 7000
python scripts/publish_model.py
git add published accuracy.md
git commit -m "Publish improved SARSA baseline"
git push

Publishing is explicit. Review the evaluation before sharing the checkpoint.