Cover Image

Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruning

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Xia, Mengzhou, Gao, Tianyu, Zeng, Zhiyuan, Chen, Danqi
Format:	Preprint
Published:	2023
Subjects:	Computation and Language Artificial Intelligence Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

BanglaLlama: LLaMA for Bangla Language
by: Zehady, Abdullah Khan, et al.
Published: (2024)

Me LLaMA: Foundation Large Language Models for Medical Applications
by: Xie, Qianqian, et al.
Published: (2024)

LLaMA Beyond English: An Empirical Study on Language Capability Transfer
by: Zhao, Jun, et al.
Published: (2024)

LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attention
by: Zhang, Renrui, et al.
Published: (2023)

LLaMAs Have Feelings Too: Unveiling Sentiment and Emotion Representations in LLaMA Models Through Probing
by: Di Palma, Dario, et al.
Published: (2025)

Tailored-LLaMA: Optimizing Few-Shot Learning in Pruned LLaMA Models with Task-Specific Prompts
by: Aftab, Danyal, et al.
Published: (2024)

Lory: Fully Differentiable Mixture-of-Experts for Autoregressive Language Model Pre-training
by: Zhong, Zexuan, et al.
Published: (2024)

How Vocabulary Sharing Facilitates Multilingualism in LLaMA?
by: Yuan, Fei, et al.
Published: (2023)

ECHO-LLaMA: Efficient Caching for High-Performance LLaMA Training
by: Dialameh, Maryam, et al.
Published: (2025)

LitSearch: A Retrieval Benchmark for Scientific Literature Search
by: Ajith, Anirudh, et al.
Published: (2024)

LLaMA-Mesh: Unifying 3D Mesh Generation with Language Models
by: Wang, Zhengyi, et al.
Published: (2024)

LESS: Selecting Influential Data for Targeted Instruction Tuning
by: Xia, Mengzhou, et al.
Published: (2024)

Zero-Shot End-to-End Relation Extraction in Chinese: A Comparative Study of Gemini, LLaMA and ChatGPT
by: Du, Shaoshuai, et al.
Published: (2025)

Metamorphic Testing for Fairness Evaluation in Large Language Models: Identifying Intersectional Bias in LLaMA and GPT
by: Reddy, Harishwar, et al.
Published: (2025)

Shortened LLaMA: Depth Pruning for Large Language Models with Comparison of Retraining Methods
by: Kim, Bo-Kyeong, et al.
Published: (2024)

LLaMA-MoE: Building Mixture-of-Experts from LLaMA with Continual Pre-training
by: Zhu, Tong, et al.
Published: (2024)

MusiScene: Leveraging MU-LLaMA for Scene Imagination and Enhanced Video Background Music Generation
by: Izzati, Fathinah, et al.
Published: (2025)

LLaMA-Omni: Seamless Speech Interaction with Large Language Models
by: Fang, Qingkai, et al.
Published: (2024)

ChatGPT vs Gemini vs LLaMA on Multilingual Sentiment Analysis
by: Buscemi, Alessio, et al.
Published: (2024)

Empowering Smaller Models: Tuning LLaMA and Gemma with Chain-of-Thought for Ukrainian Exam Tasks
by: Syromiatnikov, Mykyta, et al.
Published: (2025)

The impact of fine tuning in LLaMA on hallucinations for named entity extraction in legal documentation
by: Vargas, Francisco, et al.
Published: (2025)

LLaMA-Excitor: General Instruction Tuning via Indirect Feature Interaction
by: Zou, Bo, et al.
Published: (2024)

LLaSE-G1: Incentivizing Generalization Capability for LLaMA-based Speech Enhancement
by: Kang, Boyi, et al.
Published: (2025)

LLaMA-Berry: Pairwise Optimization for O1-like Olympiad-Level Mathematical Reasoning
by: Zhang, Di, et al.
Published: (2024)

The Uniqueness of LLaMA3-70B Series with Per-Channel Quantization
by: Qin, Minghai
Published: (2024)

LogLLaMA: Transformer-based log anomaly detection with LLaMA
by: Yang, Zhuoyi, et al.
Published: (2025)

LLaMA-E: Empowering E-commerce Authoring with Object-Interleaved Instruction Following
by: Shi, Kaize, et al.
Published: (2023)

Generating Knowledge Graphs from Large Language Models: A Comparative Study of GPT-4, LLaMA 2, and BERT
by: Bhatt, Ahan, et al.
Published: (2024)

LLaSA: Large Language and Structured Data Assistant
by: Xu, Yao, et al.
Published: (2024)

Evaluating LLaMA 3.2 for Software Vulnerability Detection
by: Gonçalves, José, et al.
Published: (2025)

LLaMA Pro: Progressive LLaMA with Block Expansion
by: Wu, Chengyue, et al.
Published: (2024)

FedSEA-LLaMA: A Secure, Efficient and Adaptive Federated Splitting Framework for Large Language Models
by: Zhang, Zishuai, et al.
Published: (2025)

Bryndza at ClimateActivism 2024: Stance, Target and Hate Event Detection via Retrieval-Augmented GPT-4 and LLaMA
by: Šuppa, Marek, et al.
Published: (2024)

Principled Instructions Are All You Need for Questioning LLaMA-1/2, GPT-3.5/4
by: Bsharat, Sondos Mahmoud, et al.
Published: (2023)

Resource-Efficient Fine-Tuning of LLaMA-3.2-3B for Medical Chain-of-Thought Reasoning
by: Mansha, Imran
Published: (2025)

Does Tone Change the Answer? Evaluating Prompt Politeness Effects on Modern LLMs: GPT, Gemini, and LLaMA
by: Cai, Hanyu, et al.
Published: (2025)

Assessing the Brittleness of Safety Alignment via Pruning and Low-Rank Modifications
by: Wei, Boyi, et al.
Published: (2024)

StatLLaMA: Multi-Stage training for domain-optimized statistical large language models
by: Zeng, Jing-Yi, et al.
Published: (2025)

Scaling Smart: Accelerating Large Language Model Pre-training with Small Model Initialization
by: Samragh, Mohammad, et al.
Published: (2024)

LLaMA-NAS: Efficient Neural Architecture Search for Large Language Models
by: Sarah, Anthony, et al.
Published: (2024)