Limits to Recursive Self Improvement of Models

Stability Engineering – Volume 9 www.manhattanproject20.com 1 117-262-D Limits to Recursive Self-Improvement of Models Evolution Cannot Outrun Observation, Evidence, or Stability 1. The Seductive Picture of Recursive Self-Improvement • Recursive Self-Improvement (RSI) is often imagined as an accelerating sequence: I ! →I " →I # →⋯→I $ • Each generation becomes better at creating its successor: Capability(I $%! )>Capability(I $ ) • If better intelligence produces still better intelligence, the intuitive…