Copertina

A deeper look at depth pruning of LLMs

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Siddiqui, Shoaib Ahmed, Dong, Xin, Heinrich, Greg, Breuel, Thomas, Kautz, Jan, Krueger, David, Molchanov, Pavlo
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Machine Learning Artificial Intelligence
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Minifinetuning: Low-Data Generation Domain Adaptation through Corrective Self-Distillation
di: Belcak, Peter, et al.
Pubblicazione: (2025)

Exploring the design space of deep-learning-based weather forecasting systems
di: Siddiqui, Shoaib Ahmed, et al.
Pubblicazione: (2024)

Position: Capability Control Should be a Separate Goal From Alignment
di: Siddiqui, Shoaib Ahmed, et al.
Pubblicazione: (2026)

MaskLLM: Learnable Semi-Structured Sparsity for Large Language Models
di: Fang, Gongfan, et al.
Pubblicazione: (2024)

FasterViT: Fast Vision Transformers with Hierarchical Attention
di: Hatamizadeh, Ali, et al.
Pubblicazione: (2023)

Blockwise Self-Supervised Learning at Scale
di: Siddiqui, Shoaib Ahmed, et al.
Pubblicazione: (2023)

On Evaluating LLMs' Capabilities as Functional Approximators: A Bayesian Perspective
di: Siddiqui, Shoaib Ahmed, et al.
Pubblicazione: (2024)

PHI-S: Distribution Balancing for Label-Free Multi-Teacher Distillation
di: Ranzinger, Mike, et al.
Pubblicazione: (2024)

LaCache: Ladder-Shaped KV Caching for Efficient Long-Context Modeling of Large Language Models
di: Shi, Dachuan, et al.
Pubblicazione: (2025)

The Topological Trouble With Transformers
di: Mozer, Michael C., et al.
Pubblicazione: (2026)

Permissive Information-Flow Analysis for Large Language Models
di: Siddiqui, Shoaib Ahmed, et al.
Pubblicazione: (2024)

ProfBench: Multi-Domain Rubrics requiring Professional Knowledge to Answer and Judge
di: Wang, Zhilin, et al.
Pubblicazione: (2025)

From Dormant to Deleted: Tamper-Resistant Unlearning Through Weight-Space Regularization
di: Siddiqui, Shoaib Ahmed, et al.
Pubblicazione: (2025)

Compact Language Models via Pruning and Knowledge Distillation
di: Muralidharan, Saurav, et al.
Pubblicazione: (2024)

Nemotron-Flash: Towards Latency-Optimal Hybrid Small Language Models
di: Fu, Yonggan, et al.
Pubblicazione: (2025)

Flextron: Many-in-One Flexible Large Language Model
di: Cai, Ruisi, et al.
Pubblicazione: (2024)

Scaling Up RL: Unlocking Diverse Reasoning in LLMs via Prolonged Training
di: Liu, Mingjie, et al.
Pubblicazione: (2025)

Hymba: A Hybrid-head Architecture for Small Language Models
di: Dong, Xin, et al.
Pubblicazione: (2024)

DLER: Doing Length pEnalty Right - Incentivizing More Intelligence per Token via Reinforcement Learning
di: Liu, Shih-Yang, et al.
Pubblicazione: (2025)

Efficient-DLM: From Autoregressive to Diffusion Language Models, and Beyond in Speed
di: Fu, Yonggan, et al.
Pubblicazione: (2025)

GDPO: Group reward-Decoupled Normalization Policy Optimization for Multi-reward RL Optimization
di: Liu, Shih-Yang, et al.
Pubblicazione: (2026)

RADIOv2.5: Improved Baselines for Agglomerative Vision Foundation Models
di: Heinrich, Greg, et al.
Pubblicazione: (2024)

Protecting against simultaneous data poisoning attacks
di: Alex, Neel, et al.
Pubblicazione: (2024)

Elimination-compensation pruning for fully-connected neural networks
di: Ballini, Enrico, et al.
Pubblicazione: (2026)

ToolOrchestra: Elevating Intelligence via Efficient Model and Tool Orchestration
di: Su, Hongjin, et al.
Pubblicazione: (2025)

PrunedLoRA: Robust Gradient-Based structured pruning for Low-rank Adaptation in Fine-tuning
di: Yu, Xin, et al.
Pubblicazione: (2025)

How Do LLMs Persuade? Linear Probes Can Uncover Persuasion Dynamics in Multi-Turn Conversations
di: Jaipersaud, Brandon, et al.
Pubblicazione: (2025)

Compact Bayesian Neural Networks via pruned MCMC sampling
di: Deo, Ratneel, et al.
Pubblicazione: (2025)

CARE: Turning LLMs Into Causal Reasoning Expert
di: Dong, Juncheng, et al.
Pubblicazione: (2025)

PruneGCRN: Minimizing and explaining spatio-temporal problems through node pruning
di: García-Sigüenza, Javier, et al.
Pubblicazione: (2025)

AM-RADIO: Agglomerative Vision Foundation Model -- Reduce All Domains Into One
di: Ranzinger, Mike, et al.
Pubblicazione: (2023)

Small Language Models are the Future of Agentic AI
di: Belcak, Peter, et al.
Pubblicazione: (2025)

In value-based deep reinforcement learning, a pruned network is a good network
di: Obando-Ceron, Johan, et al.
Pubblicazione: (2024)

Interpreting Emergent Planning in Model-Free Reinforcement Learning
di: Bush, Thomas, et al.
Pubblicazione: (2025)

Universal Deep Research: Bring Your Own Model and Strategy
di: Belcak, Peter, et al.
Pubblicazione: (2025)

$R^2$-dLLM: Accelerating Diffusion Large Language Models via Spatio-Temporal Redundancy Reduction
di: Du, Zhenbang, et al.
Pubblicazione: (2026)

GSPN-2: Efficient Parallel Sequence Modeling
di: Wang, Hongjun, et al.
Pubblicazione: (2025)

Implicit meta-learning may lead language models to trust more reliable sources
di: Krasheninnikov, Dmitrii, et al.
Pubblicazione: (2023)

PropMEND: Hypernetworks for Knowledge Propagation in LLMs
di: Liu, Zeyu Leo, et al.
Pubblicazione: (2025)

Fresh in memory: Training-order recency is linearly encoded in language model activations
di: Krasheninnikov, Dmitrii, et al.
Pubblicazione: (2025)