Easing Optimization Paths: a Circuit Perspective
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866909448492023808 |
|---|---|
| author | Odonnat, Ambroise Bouaziz, Wassim Cabannes, Vivien |
| author_facet | Odonnat, Ambroise Bouaziz, Wassim Cabannes, Vivien |
| contents | Gradient descent is the method of choice for training large artificial intelligence systems. As these systems become larger, a better understanding of the mechanisms behind gradient training would allow us to alleviate compute costs and help steer these systems away from harmful behaviors. To that end, we suggest utilizing the circuit perspective brought forward by mechanistic interpretability. After laying out our intuition, we illustrate how it enables us to design a curriculum for efficient learning in a controlled setting. The code is available at \url{https://github.com/facebookresearch/pal}. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_02362 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Easing Optimization Paths: a Circuit Perspective Odonnat, Ambroise Bouaziz, Wassim Cabannes, Vivien Machine Learning Signal Processing Gradient descent is the method of choice for training large artificial intelligence systems. As these systems become larger, a better understanding of the mechanisms behind gradient training would allow us to alleviate compute costs and help steer these systems away from harmful behaviors. To that end, we suggest utilizing the circuit perspective brought forward by mechanistic interpretability. After laying out our intuition, we illustrate how it enables us to design a curriculum for efficient learning in a controlled setting. The code is available at \url{https://github.com/facebookresearch/pal}. |
| title | Easing Optimization Paths: a Circuit Perspective |
| topic | Machine Learning Signal Processing |
| url | https://arxiv.org/abs/2501.02362 |