Cover Image

Stacking Your Transformers: A Closer Look at Model Growth for Efficient LLM Pre-Training

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Du, Wenyu, Luo, Tongxu, Qiu, Zihan, Huang, Zeyu, Shen, Yikang, Cheng, Reynold, Guo, Yike, Fu, Jie
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Unlocking Continual Learning Abilities in Language Models
by: Du, Wenyu, et al.
Published: (2024)

A Closer Look into Mixture-of-Experts in Large Language Models
by: Lo, Ka Man, et al.
Published: (2024)

Cut Your Losses! Learning to Prune Paths Early for Efficient Parallel Reasoning
by: Bi, Jiaxi, et al.
Published: (2026)

Efficient Continual Pre-training by Mitigating the Stability Gap
by: Guo, Yiduo, et al.
Published: (2024)

A Closer Look at the Robustness of Contrastive Language-Image Pre-Training (CLIP)
by: Tu, Weijie, et al.
Published: (2024)

Empirical Study on Updating Key-Value Memories in Transformer Feed-forward Layers
by: Qiu, Zihan, et al.
Published: (2024)

A Closer Look at the Limitations of Instruction Tuning
by: Ghosh, Sreyan, et al.
Published: (2024)

Unlocking Emergent Modularity in Large Language Models
by: Qiu, Zihan, et al.
Published: (2023)

A Closer Look at Knowledge Distillation in Spiking Neural Network Training
by: Liu, Xu, et al.
Published: (2025)

Look-Closer-Then-Diagnose: Confidence-Aware Ultrasound VQA via Active Zooming
by: Zhou, Yue, et al.
Published: (2026)

Force-EvT: A Closer Look at Robotic Gripper Force Measurement with Event-based Vision Transformer
by: Guo, Qianyu, et al.
Published: (2024)

Gated Linear Attention Transformers with Hardware-Efficient Training
by: Yang, Songlin, et al.
Published: (2023)

A Closer Look at the Explainability of Contrastive Language-Image Pre-training
by: Li, Yi, et al.
Published: (2023)

A Closer Look at Benchmarking Self-Supervised Pre-training with Image Classification
by: Marks, Markus, et al.
Published: (2024)

A Closer Look at Machine Unlearning for Large Language Models
by: Yuan, Xiaojian, et al.
Published: (2024)

A Closer Look at Claim Decomposition
by: Wanner, Miriam, et al.
Published: (2024)

A Closer Look at the Russell Paradox
by: Sheridan, Flash
Published: (2021)

A Closer Look at Constrained Instantons
by: Aoki, Takafumi, et al.
Published: (2026)

Bookmobiles: A Somewhat Closer Look
by: Healy, Eugene
Published: (1971)

Closer Look at Efficient Inference Methods: A Survey of Speculative Decoding
by: Ryu, Hyun, et al.
Published: (2024)

A Closer Look into Transformer-Based Code Intelligence Through Code Transformation: Challenges and Opportunities
by: Li, Yaoxian, et al.
Published: (2022)

Towards Understanding Fine-Tuning Mechanisms of LLMs via Circuit Analysis
by: Wang, Xu, et al.
Published: (2025)

Photon: Federated LLM Pre-Training
by: Sani, Lorenzo, et al.
Published: (2024)

Modeling Finite Deformations in Alloying Electrodes -- A Closer Look at Cracks and Pores During Phase Transformation
by: Zhang, Delin, et al.
Published: (2025)

A Closer Look at GAN Priors: Exploiting Intermediate Features for Enhanced Model Inversion Attacks
by: Qiu, Yixiang, et al.
Published: (2024)

A Closer Look at Multimodal Representation Collapse
by: Chaudhuri, Abhra, et al.
Published: (2025)

A Closer Look into LLMs for Table Understanding
by: Wang, Jia, et al.
Published: (2026)

A Closer Look at System Prompt Robustness
by: Mu, Norman, et al.
Published: (2025)

Taking a Closer Look at the "Grit" Narratives
by: Socol, Ira
Published: (2014)

Multicultural/Nonsexist Collections: A Closer Look.
by: Harmon, Charles
Published: (1987)

A Closer Look at Conditional Prompt Tuning for Vision-Language Models
by: Zhang, Ji, et al.
Published: (2025)

Learning Coarse-to-Fine Osteoarthritis Representations under Noisy Hierarchical Labels
by: Zhang, Tongxu
Published: (2026)

Rethinking Data Input for Point Cloud Upsampling
by: Zhang, Tongxu
Published: (2024)

TopoFR: A Closer Look at Topology Alignment on Face Recognition
by: Dan, Jun, et al.
Published: (2024)

A Closer Look on Gender Stereotypes in Movie Recommender Systems and Their Implications with Privacy
by: Roy, Falguni, et al.
Published: (2025)

A Closer Look at Time Steps is Worthy of Triple Speed-Up for Diffusion Model Training
by: Wang, Kai, et al.
Published: (2024)

Source-Free Domain Adaptation Guided by Vision and Vision-Language Pre-Training
by: Zhang, Wenyu, et al.
Published: (2024)

A Closer Look at an Unusual Ultra-Diffuse Galaxy
by: Khim, Donghyeon J., et al.
Published: (2025)

A Closer Look at Chapoton's q-Ehrhart Polynomials
by: Beck, Matthias, et al.
Published: (2025)

A Closer Look: Who Are Cancer Caregivers in 2025?
by: Leah E. Walsh, et al.
Published: (2025)