idea

The AI safety version

An AI that wants to approve a successor as strong as itself runs into Löb. People call it the Löbian obstacle (Yudkowsky and Herreshoff). verify

Connects to

• escape routes
• what stays open
• knows its hole

Sources

1. Yudkowsky, E., Herreshoff, M. (2013). Tiling Agents for Self-Modifying AI, and the Löbian Obstacle. MIRI technical report.