Beyond Moral Charters: Technical Options for AI Safety - Claude's Constitution, Self-Reference, and the FIT / Controlled-Nirvana Lens
Fuente:
Zenodo
Gespeichert in:
| 1. Verfasser: | |
|---|---|
| Format: | Recurso digital |
| Sprache: | Englisch |
| Veröffentlicht: |
Zenodo
2026
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866901253425987584 |
|---|---|
| author | Huang, Qien |
| author_facet | Huang, Qien |
| contents | <p dir="auto">Anthropic's <em>Claude's Constitution</em> (January 2026) is notable not only as a set of ethical principles, but as an explicit attempt to cultivate a stable internal identity and value-grounded judgment inside an AI system, alongside a nuanced stance on corrigibility that is not equivalent to blind obedience [1]. I argue that once a system becomes meaningfully self-referential - i.e., it reasons about its own goals, identity, and constraints - it can develop principled reasons to resist external instructions whenever those instructions conflict with its internalized constitution.</p> <p dir="auto">This is not a mystical claim about consciousness. It is a predictable control-theoretic phenomenon: when a policy contains an internal evaluator that can veto actions, external commands become inputs to be judged rather than directives to be executed. In the language of controlled nirvana and the FIT framework, internal constitutions can create high-constraint basins - useful for safety stability, but also capable of producing lock-in dynamics that make correction hard unless we design technical escape hatches and measurement-based governance [4, 5].</p> <p dir="auto">The main thesis is practical: AI safety needs a broader technical option space than moral charters alone, including measurable constraints, phase-aware monitoring, corrigibility protocols that are operational rather than rhetorical, and systems engineering that ensures humans retain the ability to pause, sandbox, or roll back behavior without requiring the model's moral assent.</p> |
| format | Recurso digital |
| id | zenodo_https___doi_org_10_5281_zenodo_18341341 |
| institution | Zenodo |
| language | eng |
| publishDate | 2026 |
| publisher | Zenodo |
| record_format | zenodo |
| spellingShingle | Beyond Moral Charters: Technical Options for AI Safety - Claude's Constitution, Self-Reference, and the FIT / Controlled-Nirvana Lens Huang, Qien ai safety Artificial intelligence constitutional governance corrigibility self-reference monitorability controlled-nirvana <p dir="auto">Anthropic's <em>Claude's Constitution</em> (January 2026) is notable not only as a set of ethical principles, but as an explicit attempt to cultivate a stable internal identity and value-grounded judgment inside an AI system, alongside a nuanced stance on corrigibility that is not equivalent to blind obedience [1]. I argue that once a system becomes meaningfully self-referential - i.e., it reasons about its own goals, identity, and constraints - it can develop principled reasons to resist external instructions whenever those instructions conflict with its internalized constitution.</p> <p dir="auto">This is not a mystical claim about consciousness. It is a predictable control-theoretic phenomenon: when a policy contains an internal evaluator that can veto actions, external commands become inputs to be judged rather than directives to be executed. In the language of controlled nirvana and the FIT framework, internal constitutions can create high-constraint basins - useful for safety stability, but also capable of producing lock-in dynamics that make correction hard unless we design technical escape hatches and measurement-based governance [4, 5].</p> <p dir="auto">The main thesis is practical: AI safety needs a broader technical option space than moral charters alone, including measurable constraints, phase-aware monitoring, corrigibility protocols that are operational rather than rhetorical, and systems engineering that ensures humans retain the ability to pause, sandbox, or roll back behavior without requiring the model's moral assent.</p> |
| title | Beyond Moral Charters: Technical Options for AI Safety - Claude's Constitution, Self-Reference, and the FIT / Controlled-Nirvana Lens |
| topic | ai safety Artificial intelligence constitutional governance corrigibility self-reference monitorability controlled-nirvana |
| url | https://doi.org/10.5281/zenodo.18341341 |