Cover Image

A Multi-Level Framework for Accelerating Training Transformer Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zou, Longwei, Zhang, Han, Deng, Yangdong
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

CQIL: Inference Latency Optimization with Concurrent Computation of Quasi-Independent Layers
by: Zou, Longwei, et al.
Published: (2024)

BiLLM: Pushing the Limit of Post-Training Quantization for LLMs
by: Huang, Wei, et al.
Published: (2024)

An Adaptive Placement and Parallelism Framework for Accelerating RLHF Training
by: Xiao, Youshao, et al.
Published: (2023)

MONA: Muon Optimizer with Nesterov Acceleration for Scalable Language Model Training
by: Li, Jiacheng, et al.
Published: (2026)

InstCache: A Predictive Cache for LLM Serving
by: Zou, Longwei, et al.
Published: (2024)

A Human-Like Reasoning Framework for Multi-Phases Planning Task with Large Language Models
by: Xie, Chengxing, et al.
Published: (2024)

Sequence-Level Leakage Risk of Training Data in Large Language Models
by: Tiwari, Trishita, et al.
Published: (2024)

Early Transformers: A study on Efficient Training of Transformer Models through Early-Bird Lottery Tickets
by: Cheekati, Shravan
Published: (2024)

SliM-LLM: Salience-Driven Mixed-Precision Quantization for Large Language Models
by: Huang, Wei, et al.
Published: (2024)

Development of Pre-Trained Transformer-based Models for the Nepali Language
by: Thapa, Prajwal, et al.
Published: (2024)

BlockFFN: Towards End-Side Acceleration-Friendly Mixture-of-Experts with Chunk-Level Activation Sparsity
by: Song, Chenyang, et al.
Published: (2025)

DataFlex: A Unified Framework for Data-Centric Dynamic Training of Large Language Models
by: Liang, Hao, et al.
Published: (2026)

Two is better than one: A Collapse-free Multi-Reward RLIF Training Framework
by: Joarder, Shourov, et al.
Published: (2026)

How does Multi-Task Training Affect Transformer In-Context Capabilities? Investigations with Function Classes
by: Bhasin, Harmon, et al.
Published: (2024)

A Comparative Analysis of LLM Memorization at Statistical and Internal Levels: Cross-Model Commonalities and Model-Specific Signatures
by: Chen, Bowen, et al.
Published: (2026)

Understanding and Accelerating the Training of Masked Diffusion Language Models
by: Hong, Chunsan, et al.
Published: (2026)

PolarQuant: Leveraging Polar Transformation for Efficient Key Cache Quantization and Decoding Acceleration
by: Wu, Songhao, et al.
Published: (2025)

MultiGPrompt for Multi-Task Pre-Training and Prompting on Graphs
by: Yu, Xingtong, et al.
Published: (2023)

Breaking Symmetry When Training Transformers
by: Zuo, Chunsheng, et al.
Published: (2024)

Toward Automated Regulatory Decision-Making: Trustworthy Medical Device Risk Classification with Multimodal Transformers and Self-Training
by: Han, Yu, et al.
Published: (2025)

Neural Attention Search Linear: Towards Adaptive Token-Level Hybrid Attention Models
by: Deng, Difan, et al.
Published: (2026)

Orchestrating Dual-Boundaries: An Arithmetic Intensity Inspired Acceleration Framework for Diffusion Language Models
by: Wei, Linye, et al.
Published: (2025)

Thinking into the Future: Latent Lookahead Training for Transformers
by: Noci, Lorenzo, et al.
Published: (2026)

Learning and Forgetting Unsafe Examples in Large Language Models
by: Zhao, Jiachen, et al.
Published: (2023)

Accelerating RL Post-Training Rollouts via System-Integrated Speculative Decoding
by: Iso, Hayate, et al.
Published: (2026)

Retrieval Backward Attention without Additional Training: Enhance Embeddings of Large Language Models via Repetition
by: Duan, Yifei, et al.
Published: (2025)

Document-Level In-Context Few-Shot Relation Extraction via Pre-Trained Language Models
by: Ozyurt, Yilmazcan, et al.
Published: (2023)

On Mesa-Optimization in Autoregressively Trained Transformers: Emergence and Capability
by: Zheng, Chenyu, et al.
Published: (2024)

Gated Linear Attention Transformers with Hardware-Efficient Training
by: Yang, Songlin, et al.
Published: (2023)

HyperAdaLoRA: Accelerating LoRA Rank Allocation During Training via Hypernetworks without Sacrificing Performance
by: Zhang, Hao, et al.
Published: (2025)

dLLM-Cache: Accelerating Diffusion Large Language Models with Adaptive Caching
by: Liu, Zhiyuan, et al.
Published: (2025)

Pre-Trained Policy Discriminators are General Reward Models
by: Dou, Shihan, et al.
Published: (2025)

MAMUT: A Novel Framework for Modifying Mathematical Formulas for the Generation of Specialized Datasets for Language Model Training
by: Drechsel, Jonathan, et al.
Published: (2025)

Accelerating Training Speed of Tiny Recursive Models with Curriculum Guided Adaptive Recursion
by: Qasim, Kaleem Ullah, et al.
Published: (2025)

Training Nonlinear Transformers for Chain-of-Thought Inference: A Theoretical Generalization Analysis
by: Li, Hongkang, et al.
Published: (2024)

Escaping the Mode Lottery: Multi-Response Training Improves Language Model Generalization
by: Amin, Hasan, et al.
Published: (2026)

Ladder: A Model-Agnostic Framework Boosting LLM-based Machine Translation to the Next Level
by: Feng, Zhaopeng, et al.
Published: (2024)

Interpretable Preferences via Multi-Objective Reward Modeling and Mixture-of-Experts
by: Wang, Haoxiang, et al.
Published: (2024)

360-LLaMA-Factory: Plug & Play Sequence Parallelism for Long Post-Training
by: Zou, Haosheng, et al.
Published: (2025)

A Symbolic Framework for Evaluating Mathematical Reasoning and Generalisation with Transformers
by: Meadows, Jordan, et al.
Published: (2023)