Knowledge Entropy Decay during Language Model Pretraining Hinders New Knowledge Acquisition
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866912270500495360 |
|---|---|
| author | Kim, Jiyeon Lee, Hyunji Cho, Hyowon Jang, Joel Hwang, Hyeonbin Won, Seungpil Ahn, Youbin Lee, Dohaeng Seo, Minjoon |
| author_facet | Kim, Jiyeon Lee, Hyunji Cho, Hyowon Jang, Joel Hwang, Hyeonbin Won, Seungpil Ahn, Youbin Lee, Dohaeng Seo, Minjoon |
| contents | In this work, we investigate how a model's tendency to broadly integrate its parametric knowledge evolves throughout pretraining, and how this behavior affects overall performance, particularly in terms of knowledge acquisition and forgetting. We introduce the concept of knowledge entropy, which quantifies the range of memory sources the model engages with; high knowledge entropy indicates that the model utilizes a wide range of memory sources, while low knowledge entropy suggests reliance on specific sources with greater certainty. Our analysis reveals a consistent decline in knowledge entropy as pretraining advances. We also find that the decline is closely associated with a reduction in the model's ability to acquire and retain knowledge, leading us to conclude that diminishing knowledge entropy (smaller number of active memory sources) impairs the model's knowledge acquisition and retention capabilities. We find further support for this by demonstrating that increasing the activity of inactive memory sources enhances the model's capacity for knowledge acquisition and retention. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_01380 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Knowledge Entropy Decay during Language Model Pretraining Hinders New Knowledge Acquisition Kim, Jiyeon Lee, Hyunji Cho, Hyowon Jang, Joel Hwang, Hyeonbin Won, Seungpil Ahn, Youbin Lee, Dohaeng Seo, Minjoon Computation and Language Artificial Intelligence In this work, we investigate how a model's tendency to broadly integrate its parametric knowledge evolves throughout pretraining, and how this behavior affects overall performance, particularly in terms of knowledge acquisition and forgetting. We introduce the concept of knowledge entropy, which quantifies the range of memory sources the model engages with; high knowledge entropy indicates that the model utilizes a wide range of memory sources, while low knowledge entropy suggests reliance on specific sources with greater certainty. Our analysis reveals a consistent decline in knowledge entropy as pretraining advances. We also find that the decline is closely associated with a reduction in the model's ability to acquire and retain knowledge, leading us to conclude that diminishing knowledge entropy (smaller number of active memory sources) impairs the model's knowledge acquisition and retention capabilities. We find further support for this by demonstrating that increasing the activity of inactive memory sources enhances the model's capacity for knowledge acquisition and retention. |
| title | Knowledge Entropy Decay during Language Model Pretraining Hinders New Knowledge Acquisition |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2410.01380 |