Copertina

COAT: Compressing Optimizer states and Activation for Memory-Efficient FP8 Training

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Xi, Haocheng, Cai, Han, Zhu, Ligeng, Lu, Yao, Keutzer, Kurt, Chen, Jianfei, Han, Song
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Machine Learning Artificial Intelligence
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Jet-RL: Enabling On-Policy FP8 Reinforcement Learning with Unified Training and Rollout Precision Flow
di: Xi, Haocheng, et al.
Pubblicazione: (2026)

DC-Gen: Post-Training Diffusion Acceleration with Deeply Compressed Latent Space
di: He, Wenkun, et al.
Pubblicazione: (2025)

DC-VideoGen: Efficient Video Generation with Deep Compression Video Autoencoder
di: Chen, Junyu, et al.
Pubblicazione: (2025)

Oscillation-Reduced MXFP4 Training for Vision Transformers
di: Chen, Yuxiang, et al.
Pubblicazione: (2025)

Jet-Nemotron: Efficient Language Model with Post Neural Architecture Search
di: Gu, Yuxian, et al.
Pubblicazione: (2025)

Deep Compression Autoencoder for Efficient High-Resolution Diffusion Models
di: Chen, Junyu, et al.
Pubblicazione: (2024)

MOSS: Efficient and Accurate FP8 LLM Training with Microscaling and Automatic Scaling
di: Zhang, Yu, et al.
Pubblicazione: (2025)

SageAttention3: Microscaling FP4 Attention for Inference and An Exploration of 8-Bit Training
di: Zhang, Jintao, et al.
Pubblicazione: (2025)

OckBench: Measuring the Efficiency of LLM Reasoning
di: Du, Zheng, et al.
Pubblicazione: (2025)

DC-AR: Efficient Masked Autoregressive Image Generation with Deep Compression Hybrid Tokenizer
di: Wu, Yecheng, et al.
Pubblicazione: (2025)

SpargeAttention: Accurate and Training-free Sparse Attention Accelerating Any Model Inference
di: Zhang, Jintao, et al.
Pubblicazione: (2025)

Memory-Efficient Fine-Tuning via Low-Rank Activation Compression
di: Shi, Jiang-Xin, et al.
Pubblicazione: (2025)

Lightning OPD: Efficient Post-Training for Large Reasoning Models with Offline On-Policy Distillation
di: Wu, Yecheng, et al.
Pubblicazione: (2026)

FP4 Explore, BF16 Train: Diffusion Reinforcement Learning via Efficient Rollout Scaling
di: Li, Yitong, et al.
Pubblicazione: (2026)

Arbitrage: Efficient Reasoning via Advantage-Aware Speculation
di: Maheswaran, Monishwaran, et al.
Pubblicazione: (2025)

Tiny Machine Learning: Progress and Futures
di: Lin, Ji, et al.
Pubblicazione: (2024)

Angles Don't Lie: Unlocking Training-Efficient RL Through the Model's Own Signals
di: Wang, Qinsi, et al.
Pubblicazione: (2025)

Efficient Post-training Quantization with FP8 Formats
di: Shen, Haihao, et al.
Pubblicazione: (2023)

TinySAM 2: Extreme Memory Compression for Efficient Track Anything Model
di: Ding, Zhaoyuan, et al.
Pubblicazione: (2026)

InfiR2: A Comprehensive FP8 Training Recipe for Reasoning-Enhanced Language Models
di: Wang, Wenjun, et al.
Pubblicazione: (2025)

JetViT: Efficient High-Resolution Vision Transformer with Post-Training Attention Search
di: Zou, Dongyun, et al.
Pubblicazione: (2026)

Radial Attention: $O(n\log n)$ Sparse Attention with Energy Decay for Long Video Generation
di: Li, Xingyang, et al.
Pubblicazione: (2025)

QuantSpec: Self-Speculative Decoding with Hierarchical Quantized KV Cache
di: Tiwari, Rishabh, et al.
Pubblicazione: (2025)

FP6-LLM: Efficiently Serving Large Language Models Through FP6-Centric Algorithm-System Co-Design
di: Xia, Haojun, et al.
Pubblicazione: (2024)

To FP8 and Back Again: Quantifying Reduced Precision Effects on LLM Training Stability
di: Lee, Joonhyung, et al.
Pubblicazione: (2024)

FlashOptim: Optimizers for Memory-Efficient Training
di: Ortiz, Jose Javier Gonzalez, et al.
Pubblicazione: (2026)

FP8-Flow-MoE: A Casting-Free FP8 Recipe without Double Quantization Error
di: Wang, Fengjuan, et al.
Pubblicazione: (2025)

The Curse and Blessing of Mean Bias in FP4-Quantized LLM Training
di: Cao, Hengjie, et al.
Pubblicazione: (2026)

Training-Free Activation Sparsity in Large Language Models
di: Liu, James, et al.
Pubblicazione: (2024)

How to Compress KV Cache in RL Post-Training? Shadow Mask Distillation for Memory-Efficient Alignment
di: Zhu, Rui, et al.
Pubblicazione: (2026)

NeuZip: Memory-Efficient Training and Inference with Dynamic Compression of Neural Networks
di: Hao, Yongchang, et al.
Pubblicazione: (2024)

R-Capsule: Compressing High-Level Plans for Efficient Large Language Model Reasoning
di: Shan, Hongyu, et al.
Pubblicazione: (2025)

EfficientTrain++: Generalized Curriculum Learning for Efficient Visual Backbone Training
di: Wang, Yulin, et al.
Pubblicazione: (2024)

Scaling FP8 training to trillion-token LLMs
di: Fishman, Maxim, et al.
Pubblicazione: (2024)

Ents: An Efficient Three-party Training Framework for Decision Trees by Communication Optimization
di: Lin, Guopeng, et al.
Pubblicazione: (2024)

E2Former-V2: On-the-Fly Equivariant Attention with Linear Activation Memory
di: Huang, Lin, et al.
Pubblicazione: (2026)

Cross-Modal Memory Compression for Efficient Multi-Agent Debate
di: Wu, Jing, et al.
Pubblicazione: (2026)

OmniRefine: Alignment-Aware Cooperative Compression for Efficient Omnimodal Large Language Models
di: Deng, Yuchen, et al.
Pubblicazione: (2026)

SimpleMem: Efficient Lifelong Memory for LLM Agents
di: Liu, Jiaqi, et al.
Pubblicazione: (2026)

Lattice: Learning to Efficiently Compress the Memory
di: Karami, Mahdi, et al.
Pubblicazione: (2025)