Guardado en:
| Autores principales: | , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | https://arxiv.org/abs/2503.18693 |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866912292384276480 |
|---|---|
| author | Shin, Changho Yan, Xinya Jo, Suenggwan Cho, Sungjun Chaudhuri, Shourjo Aditya Sala, Frederic |
| author_facet | Shin, Changho Yan, Xinya Jo, Suenggwan Cho, Sungjun Chaudhuri, Shourjo Aditya Sala, Frederic |
| contents | Language models often struggle with temporal misalignment, performance degradation caused by shifts in the temporal distribution of data. Continuously updating models to avoid degradation is expensive. Can models be adapted without updating model weights? We present TARDIS, an unsupervised representation editing method that addresses this challenge. TARDIS extracts steering vectors from unlabeled data and adjusts the model's representations to better align with the target time period's distribution. Our experiments reveal that TARDIS enhances downstream task performance without the need for fine-tuning, can mitigate temporal misalignment even when exact target time period data is unavailable, and remains efficient even when the temporal information of the target data points is unknown at inference time. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_18693 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | TARDIS: Mitigating Temporal Misalignment via Representation Steering Shin, Changho Yan, Xinya Jo, Suenggwan Cho, Sungjun Chaudhuri, Shourjo Aditya Sala, Frederic Machine Learning Language models often struggle with temporal misalignment, performance degradation caused by shifts in the temporal distribution of data. Continuously updating models to avoid degradation is expensive. Can models be adapted without updating model weights? We present TARDIS, an unsupervised representation editing method that addresses this challenge. TARDIS extracts steering vectors from unlabeled data and adjusts the model's representations to better align with the target time period's distribution. Our experiments reveal that TARDIS enhances downstream task performance without the need for fine-tuning, can mitigate temporal misalignment even when exact target time period data is unavailable, and remains efficient even when the temporal information of the target data points is unknown at inference time. |
| title | TARDIS: Mitigating Temporal Misalignment via Representation Steering |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2503.18693 |