Enregistré dans:
| Auteurs principaux: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | https://arxiv.org/abs/2502.16797 |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866912243794313216 |
|---|---|
| author | Jones, Erik Tong, Meg Mu, Jesse Mahfoud, Mohammed Leike, Jan Grosse, Roger Kaplan, Jared Fithian, William Perez, Ethan Sharma, Mrinank |
| author_facet | Jones, Erik Tong, Meg Mu, Jesse Mahfoud, Mohammed Leike, Jan Grosse, Roger Kaplan, Jared Fithian, William Perez, Ethan Sharma, Mrinank |
| contents | Standard language model evaluations can fail to capture risks that emerge only at deployment scale. For example, a model may produce safe responses during a small-scale beta test, yet reveal dangerous information when processing billions of requests at deployment. To remedy this, we introduce a method to forecast potential risks across orders of magnitude more queries than we test during evaluation. We make forecasts by studying each query's elicitation probability -- the probability the query produces a target behavior -- and demonstrate that the largest observed elicitation probabilities predictably scale with the number of queries. We find that our forecasts can predict the emergence of diverse undesirable behaviors -- such as assisting users with dangerous chemical synthesis or taking power-seeking actions -- across up to three orders of magnitude of query volume. Our work enables model developers to proactively anticipate and patch rare failures before they manifest during large-scale deployments. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_16797 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Forecasting Rare Language Model Behaviors Jones, Erik Tong, Meg Mu, Jesse Mahfoud, Mohammed Leike, Jan Grosse, Roger Kaplan, Jared Fithian, William Perez, Ethan Sharma, Mrinank Machine Learning Standard language model evaluations can fail to capture risks that emerge only at deployment scale. For example, a model may produce safe responses during a small-scale beta test, yet reveal dangerous information when processing billions of requests at deployment. To remedy this, we introduce a method to forecast potential risks across orders of magnitude more queries than we test during evaluation. We make forecasts by studying each query's elicitation probability -- the probability the query produces a target behavior -- and demonstrate that the largest observed elicitation probabilities predictably scale with the number of queries. We find that our forecasts can predict the emergence of diverse undesirable behaviors -- such as assisting users with dangerous chemical synthesis or taking power-seeking actions -- across up to three orders of magnitude of query volume. Our work enables model developers to proactively anticipate and patch rare failures before they manifest during large-scale deployments. |
| title | Forecasting Rare Language Model Behaviors |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2502.16797 |