Saved in:
| Main Authors: | , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2511.01836 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917099627085824 |
|---|---|
| author | Lubana, Ekdeep Singh Rager, Can Hindupur, Sai Sumedh R. Costa, Valerie Tuckute, Greta Patel, Oam Murthy, Sonia Krishna Fel, Thomas Wurgaft, Daniel Bigelow, Eric J. Lin, Johnny Ba, Demba Wattenberg, Martin Viegas, Fernanda Weber, Melanie Mueller, Aaron |
| author_facet | Lubana, Ekdeep Singh Rager, Can Hindupur, Sai Sumedh R. Costa, Valerie Tuckute, Greta Patel, Oam Murthy, Sonia Krishna Fel, Thomas Wurgaft, Daniel Bigelow, Eric J. Lin, Johnny Ba, Demba Wattenberg, Martin Viegas, Fernanda Weber, Melanie Mueller, Aaron |
| contents | Recovering meaningful concepts from language model activations is a central aim of interpretability. While existing feature extraction methods aim to identify concepts that are independent directions, it is unclear if this assumption can capture the rich temporal structure of language. Specifically, via a Bayesian lens, we demonstrate that Sparse Autoencoders (SAEs) impose priors that assume independence of concepts across time, implying stationarity. Meanwhile, language model representations exhibit rich temporal dynamics, including systematic growth in conceptual dimensionality, context-dependent correlations, and pronounced non-stationarity, in direct conflict with the priors of SAEs. Taking inspiration from computational neuroscience, we introduce a new interpretability objective -- Temporal Feature Analysis -- which possesses a temporal inductive bias to decompose representations at a given time into two parts: a predictable component, which can be inferred from the context, and a residual component, which captures novel information unexplained by the context. Temporal Feature Analyzers correctly parse garden path sentences, identify event boundaries, and more broadly delineate abstract, slow-moving information from novel, fast-moving information, while existing SAEs show significant pitfalls in all the above tasks. Overall, our results underscore the need for inductive biases that match the data in designing robust interpretability tools. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_01836 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Priors in Time: Missing Inductive Biases for Language Model Interpretability Lubana, Ekdeep Singh Rager, Can Hindupur, Sai Sumedh R. Costa, Valerie Tuckute, Greta Patel, Oam Murthy, Sonia Krishna Fel, Thomas Wurgaft, Daniel Bigelow, Eric J. Lin, Johnny Ba, Demba Wattenberg, Martin Viegas, Fernanda Weber, Melanie Mueller, Aaron Machine Learning Recovering meaningful concepts from language model activations is a central aim of interpretability. While existing feature extraction methods aim to identify concepts that are independent directions, it is unclear if this assumption can capture the rich temporal structure of language. Specifically, via a Bayesian lens, we demonstrate that Sparse Autoencoders (SAEs) impose priors that assume independence of concepts across time, implying stationarity. Meanwhile, language model representations exhibit rich temporal dynamics, including systematic growth in conceptual dimensionality, context-dependent correlations, and pronounced non-stationarity, in direct conflict with the priors of SAEs. Taking inspiration from computational neuroscience, we introduce a new interpretability objective -- Temporal Feature Analysis -- which possesses a temporal inductive bias to decompose representations at a given time into two parts: a predictable component, which can be inferred from the context, and a residual component, which captures novel information unexplained by the context. Temporal Feature Analyzers correctly parse garden path sentences, identify event boundaries, and more broadly delineate abstract, slow-moving information from novel, fast-moving information, while existing SAEs show significant pitfalls in all the above tasks. Overall, our results underscore the need for inductive biases that match the data in designing robust interpretability tools. |
| title | Priors in Time: Missing Inductive Biases for Language Model Interpretability |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2511.01836 |