Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918235135279104 |
|---|---|
| author | Korbak, Tomek Balesni, Mikita Barnes, Elizabeth Bengio, Yoshua Benton, Joe Bloom, Joseph Chen, Mark Cooney, Alan Dafoe, Allan Dragan, Anca Emmons, Scott Evans, Owain Farhi, David Greenblatt, Ryan Hendrycks, Dan Hobbhahn, Marius Hubinger, Evan Irving, Geoffrey Jenner, Erik Kokotajlo, Daniel Krakovna, Victoria Legg, Shane Lindner, David Luan, David Mądry, Aleksander Michael, Julian Nanda, Neel Orr, Dave Pachocki, Jakub Perez, Ethan Phuong, Mary Roger, Fabien Saxe, Joshua Shlegeris, Buck Soto, Martín Steinberger, Eric Wang, Jasmine Zaremba, Wojciech Baker, Bowen Shah, Rohin Mikulik, Vlad |
| author_facet | Korbak, Tomek Balesni, Mikita Barnes, Elizabeth Bengio, Yoshua Benton, Joe Bloom, Joseph Chen, Mark Cooney, Alan Dafoe, Allan Dragan, Anca Emmons, Scott Evans, Owain Farhi, David Greenblatt, Ryan Hendrycks, Dan Hobbhahn, Marius Hubinger, Evan Irving, Geoffrey Jenner, Erik Kokotajlo, Daniel Krakovna, Victoria Legg, Shane Lindner, David Luan, David Mądry, Aleksander Michael, Julian Nanda, Neel Orr, Dave Pachocki, Jakub Perez, Ethan Phuong, Mary Roger, Fabien Saxe, Joshua Shlegeris, Buck Soto, Martín Steinberger, Eric Wang, Jasmine Zaremba, Wojciech Baker, Bowen Shah, Rohin Mikulik, Vlad |
| contents | AI systems that "think" in human language offer a unique opportunity for AI safety: we can monitor their chains of thought (CoT) for the intent to misbehave. Like all other known AI oversight methods, CoT monitoring is imperfect and allows some misbehavior to go unnoticed. Nevertheless, it shows promise and we recommend further research into CoT monitorability and investment in CoT monitoring alongside existing safety methods. Because CoT monitorability may be fragile, we recommend that frontier model developers consider the impact of development decisions on CoT monitorability. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_11473 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety Korbak, Tomek Balesni, Mikita Barnes, Elizabeth Bengio, Yoshua Benton, Joe Bloom, Joseph Chen, Mark Cooney, Alan Dafoe, Allan Dragan, Anca Emmons, Scott Evans, Owain Farhi, David Greenblatt, Ryan Hendrycks, Dan Hobbhahn, Marius Hubinger, Evan Irving, Geoffrey Jenner, Erik Kokotajlo, Daniel Krakovna, Victoria Legg, Shane Lindner, David Luan, David Mądry, Aleksander Michael, Julian Nanda, Neel Orr, Dave Pachocki, Jakub Perez, Ethan Phuong, Mary Roger, Fabien Saxe, Joshua Shlegeris, Buck Soto, Martín Steinberger, Eric Wang, Jasmine Zaremba, Wojciech Baker, Bowen Shah, Rohin Mikulik, Vlad Artificial Intelligence Machine Learning AI systems that "think" in human language offer a unique opportunity for AI safety: we can monitor their chains of thought (CoT) for the intent to misbehave. Like all other known AI oversight methods, CoT monitoring is imperfect and allows some misbehavior to go unnoticed. Nevertheless, it shows promise and we recommend further research into CoT monitorability and investment in CoT monitoring alongside existing safety methods. Because CoT monitorability may be fragile, we recommend that frontier model developers consider the impact of development decisions on CoT monitorability. |
| title | Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety |
| topic | Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2507.11473 |