Cover Image

Shortened LLaMA: Depth Pruning for Large Language Models with Comparison of Retraining Methods

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Kim, Bo-Kyeong, Kim, Geonmin, Kim, Tae-Ho, Castells, Thibault, Choi, Shinkook, Shin, Junho, Song, Hyoung-Kyu
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

LD-Pruner: Efficient Pruning of Latent Diffusion Models using Task-Agnostic Insights
by: Castells, Thibault, et al.
Published: (2024)

BK-SDM: A Lightweight, Fast, and Cheap Version of Stable Diffusion
by: Kim, Bo-Kyeong, et al.
Published: (2023)

EdgeFusion: On-Device Text-to-Image Generation
by: Castells, Thibault, et al.
Published: (2024)

Efficient LLaMA-3.2-Vision by Trimming Cross-attended Visual Features
by: Lee, Jewon, et al.
Published: (2025)

Tailored-LLaMA: Optimizing Few-Shot Learning in Pruned LLaMA Models with Task-Specific Prompts
by: Aftab, Danyal, et al.
Published: (2024)

LLaMA Pro: Progressive LLaMA with Block Expansion
by: Wu, Chengyue, et al.
Published: (2024)

Assessing the Answerability of Queries in Retrieval-Augmented Code Generation
by: Kim, Geonmin, et al.
Published: (2024)

ECHO-LLaMA: Efficient Caching for High-Performance LLaMA Training
by: Dialameh, Maryam, et al.
Published: (2025)

Mol-LLaMA: Towards General Understanding of Molecules in Large Molecular Language Model
by: Kim, Dongki, et al.
Published: (2025)

VisionLLaMA: A Unified LLaMA Backbone for Vision Tasks
by: Chu, Xiangxiang, et al.
Published: (2024)

LLaMA-Reg: Using LLaMA 2 for Unsupervised Medical Image Registration
by: Ma, Mingrui, et al.
Published: (2024)

LLaMA-MoE: Building Mixture-of-Experts from LLaMA with Continual Pre-training
by: Zhu, Tong, et al.
Published: (2024)

LogLLaMA: Transformer-based log anomaly detection with LLaMA
by: Yang, Zhuoyi, et al.
Published: (2025)

Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruning
by: Xia, Mengzhou, et al.
Published: (2023)

Adapting LLaMA Decoder to Vision Transformer
by: Wang, Jiahao, et al.
Published: (2024)

BanglaLlama: LLaMA for Bangla Language
by: Zehady, Abdullah Khan, et al.
Published: (2024)

LLaMA-XR: A Novel Framework for Radiology Report Generation using LLaMA and QLoRA Fine Tuning
by: Jahangir, Md. Zihad Bin, et al.
Published: (2025)

LLaMA-MoE v2: Exploring Sparsity of LLaMA from Perspective of Mixture-of-Experts with Post-Training
by: Qu, Xiaoye, et al.
Published: (2024)

Me LLaMA: Foundation Large Language Models for Medical Applications
by: Xie, Qianqian, et al.
Published: (2024)

LLaMA-Omni: Seamless Speech Interaction with Large Language Models
by: Fang, Qingkai, et al.
Published: (2024)

LLaMAs Have Feelings Too: Unveiling Sentiment and Emotion Representations in LLaMA Models Through Probing
by: Di Palma, Dario, et al.
Published: (2025)

How Vocabulary Sharing Facilitates Multilingualism in LLaMA?
by: Yuan, Fei, et al.
Published: (2023)

Multimodal Medical Disease Classification with LLaMA II
by: Gapp, Christian, et al.
Published: (2024)

Amharic LLaMA and LLaVA: Multimodal LLMs for Low Resource Languages
by: Andersland, Michael
Published: (2024)

Deformable Dynamic Convolution for Accurate yet Efficient Spatio-Temporal Traffic Prediction
by: Jin, Hyeonseok, et al.
Published: (2025)

LLaMA-Excitor: General Instruction Tuning via Indirect Feature Interaction
by: Zou, Bo, et al.
Published: (2024)

LLaMA-NAS: Efficient Neural Architecture Search for Large Language Models
by: Sarah, Anthony, et al.
Published: (2024)

VoCo-LLaMA: Towards Vision Compression with Large Language Models
by: Ye, Xubing, et al.
Published: (2024)

Efficient and Effective Text Encoding for Chinese LLaMA and Alpaca
by: Cui, Yiming, et al.
Published: (2023)

Dynamic Activation Pitfalls in LLaMA Models: An Empirical Study
by: Ma, Chi, et al.
Published: (2024)

Faster Speech-LLaMA Inference with Multi-token Prediction
by: Raj, Desh, et al.
Published: (2024)

Parameter-Efficient Fine-Tuning of LLaMA for the Clinical Domain
by: Gema, Aryo Pradipta, et al.
Published: (2023)

Evaluating LLaMA 3.2 for Software Vulnerability Detection
by: Gonçalves, José, et al.
Published: (2025)

LLaMA-Based Models for Aspect-Based Sentiment Analysis
by: Šmíd, Jakub, et al.
Published: (2025)

SNP: Structured Neuron-level Pruning to Preserve Attention Scores
by: Shim, Kyunghwan, et al.
Published: (2024)

Dia-LLaMA: Towards Large Language Model-driven CT Report Generation
by: Chen, Zhixuan, et al.
Published: (2024)

CAPA: Contribution-Aware Pruning and FFN Approximation for Efficient Large Vision-Language Models
by: Jha, Samyak, et al.
Published: (2026)

LLaSE-G1: Incentivizing Generalization Capability for LLaMA-based Speech Enhancement
by: Kang, Boyi, et al.
Published: (2025)

LLaMA Beyond English: An Empirical Study on Language Capability Transfer
by: Zhao, Jun, et al.
Published: (2024)

What If We Recaption Billions of Web Images with LLaMA-3?
by: Li, Xianhang, et al.
Published: (2024)