Untied Ulysses: Memory-Efficient Context Parallelism via Headwise Chunking

Documenti analoghi

• AutoChunk: Automated Activation Chunk for Memory-Efficient Long Sequence Inference
  di: Zhao, Xuanlei, et al.
  Pubblicazione: (2024)
• DCP: Addressing Input Dynamism In Long-Context Training via Dynamic Context Parallelism
  di: Jiang, Chenyu, et al.
  Pubblicazione: (2025)
• ChunkFlow: Communication-Aware Chunked Prefetching for Layerwise Offloading in Distributed Diffusion Transformer Inference
  di: Meng, Han, et al.
  Pubblicazione: (2026)
• LoongServe: Efficiently Serving Long-Context Large Language Models with Elastic Sequence Parallelism
  di: Wu, Bingyang, et al.
  Pubblicazione: (2024)
• Memory and Bandwidth are All You Need for Fully Sharded Data Parallel
  di: Wang, Jiangtao, et al.
  Pubblicazione: (2025)