HAMMER: Hamiltonian Curiosity Augmented Large Language Model Reinforcement
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866918150875906048 |
|---|---|
| author | Yang, Ming Li, Xiaofan Ma, Zhiyuan Shi, Dengliang Du, Jintao Cheng, Yu Zheng, Weiguo |
| author_facet | Yang, Ming Li, Xiaofan Ma, Zhiyuan Shi, Dengliang Du, Jintao Cheng, Yu Zheng, Weiguo |
| contents | Recent curriculum reinforcement learning for large language models (LLMs) typically rely on difficulty-based annotations for data filtering and ordering. However, such methods suffer from local optimization, where continual training on simple samples in the early steps can cause the policy to lose its exploration. We propose a novel schema, namely Hamiltonian curiosity augmented large language model reinforcement (HAMMER), that transfers diversity metrics, commonly used in dataset evaluation, into the dynamic reinforcement learning procedure, where training samples are ordered via a minimum-semantic Hamiltonian path making the initial training retrain more exploration. From a theoretical perspective of generalization bounds, diversity-driven ordering facilitates stable convergence. Empirical evaluations indicate that HAMMER stimulates model "curiosity" and consistently achieves a 3% to 4% average accuracy gain across diverse inference benchmark. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_25240 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | HAMMER: Hamiltonian Curiosity Augmented Large Language Model Reinforcement Yang, Ming Li, Xiaofan Ma, Zhiyuan Shi, Dengliang Du, Jintao Cheng, Yu Zheng, Weiguo Machine Learning Artificial Intelligence Computation and Language I.2.7 Recent curriculum reinforcement learning for large language models (LLMs) typically rely on difficulty-based annotations for data filtering and ordering. However, such methods suffer from local optimization, where continual training on simple samples in the early steps can cause the policy to lose its exploration. We propose a novel schema, namely Hamiltonian curiosity augmented large language model reinforcement (HAMMER), that transfers diversity metrics, commonly used in dataset evaluation, into the dynamic reinforcement learning procedure, where training samples are ordered via a minimum-semantic Hamiltonian path making the initial training retrain more exploration. From a theoretical perspective of generalization bounds, diversity-driven ordering facilitates stable convergence. Empirical evaluations indicate that HAMMER stimulates model "curiosity" and consistently achieves a 3% to 4% average accuracy gain across diverse inference benchmark. |
| title | HAMMER: Hamiltonian Curiosity Augmented Large Language Model Reinforcement |
| topic | Machine Learning Artificial Intelligence Computation and Language I.2.7 |
| url | https://arxiv.org/abs/2509.25240 |