Chameleon: Taming Dynamic Operator Sequences for Memory-Intensive LLM Training
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866911153263738880 |
|---|---|
| author | Wang, Zibo Zhou, Yuhang Wang, Zhibin Li, Shipeng Huang, Xinjing Cai, Chendong Mu, Bingxu Sun, Yuqing Hu, Zhiheng She, Bin You, Shu Fang, Guanghuan Gu, Rong Dou, Wanchun Chen, Guihai Tian, Chen |
| author_facet | Wang, Zibo Zhou, Yuhang Wang, Zhibin Li, Shipeng Huang, Xinjing Cai, Chendong Mu, Bingxu Sun, Yuqing Hu, Zhiheng She, Bin You, Shu Fang, Guanghuan Gu, Rong Dou, Wanchun Chen, Guihai Tian, Chen |
| contents | The increasing size of large language models (LLMs) has led to a surge in memory requirements during training, often exceeding the capacity of high-bandwidth memory (HBM). Swap-based memory optimization incurs neither accuracy loss nor additional end-to-end overhead when effectively overlapped, thus being an attractive solution. However, existing swap methods assume consistent operator sequences, which is impractical in Eager Mode, where operator sequences can vary during change.
We propose Chameleon, which redesigns the end-to-end process of swap-based memory optimization and is the first work to consider varying operator sequences in Eager Mode. Chameleon (i) introduces a lightweight online profiler to enable continuous profiling for monitoring operator sequences, (ii) generates effective swap policies with limited operator information, and (iii) optimizes the policy execution module for accurate policy application and better performance. Experimental results demonstrate that Chameleon reduces profiling overhead by 84.25%, enables training models up to 4x larger than hardware memory while adapting to changes in operator sequences, improves performance by up to 38.94% compared to recomputation or high-degree parallelism. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_11076 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Chameleon: Taming Dynamic Operator Sequences for Memory-Intensive LLM Training Wang, Zibo Zhou, Yuhang Wang, Zhibin Li, Shipeng Huang, Xinjing Cai, Chendong Mu, Bingxu Sun, Yuqing Hu, Zhiheng She, Bin You, Shu Fang, Guanghuan Gu, Rong Dou, Wanchun Chen, Guihai Tian, Chen Distributed, Parallel, and Cluster Computing The increasing size of large language models (LLMs) has led to a surge in memory requirements during training, often exceeding the capacity of high-bandwidth memory (HBM). Swap-based memory optimization incurs neither accuracy loss nor additional end-to-end overhead when effectively overlapped, thus being an attractive solution. However, existing swap methods assume consistent operator sequences, which is impractical in Eager Mode, where operator sequences can vary during change. We propose Chameleon, which redesigns the end-to-end process of swap-based memory optimization and is the first work to consider varying operator sequences in Eager Mode. Chameleon (i) introduces a lightweight online profiler to enable continuous profiling for monitoring operator sequences, (ii) generates effective swap policies with limited operator information, and (iii) optimizes the policy execution module for accurate policy application and better performance. Experimental results demonstrate that Chameleon reduces profiling overhead by 84.25%, enables training models up to 4x larger than hardware memory while adapting to changes in operator sequences, improves performance by up to 38.94% compared to recomputation or high-degree parallelism. |
| title | Chameleon: Taming Dynamic Operator Sequences for Memory-Intensive LLM Training |
| topic | Distributed, Parallel, and Cluster Computing |
| url | https://arxiv.org/abs/2509.11076 |