Federated Nested Learning: Collaborative Training of Self-Referential Memories for Test-Time Adaptation
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2026
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866911690261528576 |
|---|---|
| author | Chen, Hong Wu, Pengcheng Lin, Yuanguo Zhao, Peilin Zhou, Xiuze Lin, Fan Yu, Han |
| author_facet | Chen, Hong Wu, Pengcheng Lin, Yuanguo Zhao, Peilin Zhou, Xiuze Lin, Fan Yu, Han |
| contents | We rethink Federated Learning (FL) from a nested learning perspective, framing the core challenge as how to collaboratively learn optimization rules, not just static models, to tackle Non-IID client data. To address this, we propose Federated Nested Learning (FedNL), a novel framework that reformulates FL as a three-level nested optimization system. FedNL embeds Titans-based linear attention into FL, enabling clients to perform lightweight, zero-shot test-time adaptation by treating a delta rule as an online gradient step. Experiments on Non-IID MMLU and long-context benchmarks show that FedNL achieves competitive performance in short-context reasoning, enhances the performance of long-context retrieval and streaming Cross-Entropy, and maintains constant inference memory. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_16350 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Federated Nested Learning: Collaborative Training of Self-Referential Memories for Test-Time Adaptation Chen, Hong Wu, Pengcheng Lin, Yuanguo Zhao, Peilin Zhou, Xiuze Lin, Fan Yu, Han Machine Learning Artificial Intelligence We rethink Federated Learning (FL) from a nested learning perspective, framing the core challenge as how to collaboratively learn optimization rules, not just static models, to tackle Non-IID client data. To address this, we propose Federated Nested Learning (FedNL), a novel framework that reformulates FL as a three-level nested optimization system. FedNL embeds Titans-based linear attention into FL, enabling clients to perform lightweight, zero-shot test-time adaptation by treating a delta rule as an online gradient step. Experiments on Non-IID MMLU and long-context benchmarks show that FedNL achieves competitive performance in short-context reasoning, enhances the performance of long-context retrieval and streaming Cross-Entropy, and maintains constant inference memory. |
| title | Federated Nested Learning: Collaborative Training of Self-Referential Memories for Test-Time Adaptation |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2605.16350 |