Skip to content
MouseMouse

Add a termination that makes the degenerate strategy fatal - no height cut-off meant crouch-shuffling could live forever

A lesson the Coach cites as termination-closes-degenerate-basin.
Observed onceReward design

For each known degenerate strategy, check whether the termination set makes it fatal; if the robot can live indefinitely inside the degenerate posture, add a termination just past the intended operating envelope rather than escalating penalties.

Symptom

Crouched foot-dragging survived indefinitely because the termination set contained only bad_orientation (40 deg) and base contact - there was no height termination at all, so a deep squat was a viable long-term strategy.

Context

The hypothesis audit found the missing termination (hypothesis 7); the cross-check against published configs found the field practice: Booster terminates at 0.45 m (38% of body height) and the research warning is that the termination height must not be so low that crouching survives it. The proposed value: 0.32 m, just below the walk crouch base height 0.3739 - a deep squat terminates immediately, "断掉蹲着蹭的活路" (cutting off the crouch-shuffle's livelihood).

Change

Add height termination at 0.32 m as a second-priority item of the walk fix package, alongside restoring base_height_l2 to -10.

Outcome

Entered the v5/v6 fix package under which the crouch-shuffle optimum disappeared (34 mm clearance, 87% tracking by v6).

Mechanism

Termination conditions define which strategies exist at all: a reward penalty prices a behavior, but a termination deletes its future returns entirely. Degenerate basins that are merely penalized can remain optimal under enough tracking pressure; a termination placed between the degenerate posture and the intended one makes the basin unreachable as a steady state.

Applies when

  • a degenerate but stable behavior persists across reward tunings
  • auditing termination conditions for a locomotion task
  • a policy exploits the gap between penalized and terminated states
“加终止高度:研究第 6 条"终止高度不能低到让蹲着也能活"。我们完全没有高度终止。建议 0.32 m(略低于 walk 蹲姿基座高 0.3739,深蹲即终止)。… 加终止高度 0.32 m(深蹲即终止,断掉蹲着蹭的活路)”
train/WALK_DIAGNOSIS.md § 修正 ④ / 最终改动清单 第二优先

Principles that cite it