Cover Image

ShortGPT: Layers in Large Language Models are More Redundant Than You Expect

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Men, Xin, Xu, Mingyu, Zhang, Qingyu, Wang, Bingning, Lin, Hongyu, Lu, Yaojie, Han, Xianpei, Chen, Weipeng
Format:	Preprint
Published:	2024
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Base of RoPE Bounds Context Length
by: Men, Xin, et al.
Published: (2024)

ShortV: Efficient Multimodal Large Language Models by Freezing Visual Tokens in Ineffective Layers
by: Yuan, Qianhao, et al.
Published: (2025)

Executing Natural Language-Described Algorithms with Large Language Models: An Investigation
by: Zheng, Xin, et al.
Published: (2024)

ChatGPT is a Knowledgeable but Inexperienced Solver: An Investigation of Commonsense Problem in Large Language Models
by: Bian, Ning, et al.
Published: (2023)

Rule or Story, Which is a Better Commonsense Expression for Talking with Large Language Models?
by: Bian, Ning, et al.
Published: (2024)

AI-Salesman: Towards Reliable Large Language Model Driven Telemarketing
by: Zhang, Qingyu, et al.
Published: (2025)

KV Shifting Attention Enhances Language Modeling
by: Xu, Mingyu, et al.
Published: (2024)

Meta-Cognitive Analysis: Evaluating Declarative and Procedural Knowledge in Datasets and Large Language Models
by: Li, Zhuoqun, et al.
Published: (2024)

MetaGPT: Merging Large Language Models Using Model Exclusive Task Arithmetic
by: Zhou, Yuyan, et al.
Published: (2024)

LongReD: Mitigating Short-Text Degradation of Long-Context Large Language Models via Restoration Distillation
by: Dong, Zican, et al.
Published: (2025)

SAISA: Towards Multimodal Large Language Models with Both Training and Inference Efficiency
by: Yuan, Qianhao, et al.
Published: (2025)

The Rise and Down of Babel Tower: Investigating the Evolution Process of Multilingual Code Large Language Model
by: Chen, Jiawei, et al.
Published: (2024)

REInstruct: Building Instruction Data from Unlabeled Corpus
by: Chen, Shu, et al.
Published: (2024)

Open Grounded Planning: Challenges and Benchmark Construction
by: Guo, Shiguang, et al.
Published: (2024)

DeepRAG: Thinking to Retrieve Step by Step for Large Language Models
by: Guan, Xinyan, et al.
Published: (2025)

Self-Steering Optimization: Autonomous Preference Optimization for Large Language Models
by: Xiang, Hao, et al.
Published: (2024)

Beyond Correctness: Benchmarking Multi-dimensional Code Generation for Large Language Models
by: Zheng, Jiasheng, et al.
Published: (2024)

LiteCoder-Terminal: Scaling Long-Horizon Terminal Environments for Learning Language Agents
by: Peng, Xiaoxuan, et al.
Published: (2026)

A Unified View of Delta Parameter Editing in Post-Trained Large-Scale Models
by: Tang, Qiaoyu, et al.
Published: (2024)

Full-ECE: A Metric For Token-level Calibration on Large Language Models
by: Liu, Han, et al.
Published: (2024)

Few-shot Named Entity Recognition via Superposition Concept Discrimination
by: Chen, Jiawei, et al.
Published: (2024)

Multi-Facet Counterfactual Learning for Content Quality Evaluation
by: Zheng, Jiasheng, et al.
Published: (2024)

Auto-RT: Automatic Jailbreak Strategy Exploration for Red-Teaming Large Language Models
by: Liu, Yanjiang, et al.
Published: (2025)

ScaleBox: Enabling High-Fidelity and Scalable Code Verification for Large Language Models
by: Zheng, Jiasheng, et al.
Published: (2026)

ConsistentChat: Building Skeleton-Guided Consistent Multi-Turn Dialogues for Large Language Models from Scratch
by: Chen, Jiawei, et al.
Published: (2025)

Your Teacher Can't Help You Here: Combating Supervision Fidelity Decay in On-Policy Distillation
by: Liu, Yanjiang, et al.
Published: (2026)

SoFA: Shielded On-the-fly Alignment via Priority Rule Following
by: Lu, Xinyu, et al.
Published: (2024)

Expanding the Boundaries of Vision Prior Knowledge in Multi-modal Large Language Models
by: Liang, Qiao, et al.
Published: (2025)

The Life Cycle of Knowledge in Big Language Models: A Survey
by: Cao, Boxi, et al.
Published: (2023)

Coupled Variational Reinforcement Learning for Language Model General Reasoning
by: Wen, Xueru, et al.
Published: (2025)

When Models Outthink Their Safety: Unveiling and Mitigating Self-Jailbreak in Large Reasoning Models
by: Mao, Yingzhi, et al.
Published: (2025)

Social Simulations with Large Language Model Risk Utopian Illusion
by: Bian, Ning, et al.
Published: (2025)

Rethinking Layer Redundancy: Calibration Matters More Than Search in LLM Depth Pruning
by: Kim, Minkyu, et al.
Published: (2026)

Unveiling the Flaws: Exploring Imperfections in Synthetic Data and Mitigation Strategies for Large Language Models
by: Chen, Jie, et al.
Published: (2024)

Vision-OPD: Learning to See Fine Details for Multimodal LLMs via On-Policy Self-Distillation
by: Yuan, Qianhao, et al.
Published: (2026)

Large Language Models Often Say One Thing and Do Another
by: Xu, Ruoxi, et al.
Published: (2025)

Seg2Act: Global Context-aware Action Generation for Document Logical Structuring
by: Li, Zichao, et al.
Published: (2024)

PaperRegister: Boosting Flexible-grained Paper Search via Hierarchical Register Indexing
by: Li, Zhuoqun, et al.
Published: (2025)

RefCritic: Training Long Chain-of-Thought Critic Models with Refinement Feedback
by: Tang, Qiaoyu, et al.
Published: (2025)

LLMs Struggle with Abstract Meaning Comprehension More Than Expected
by: Alhazmi, Hamoud, et al.
Published: (2026)