Cover Image

SAP: Syntactic Attention Pruning for Transformer-based Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Lee, Tzu-Yun, Hong, Ding-Yong, Wu, Jan-Jan
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Automatic Pruning of Fine-tuning Datasets for Transformer-based Language Models
by: Tayaranian, Mohammadreza, et al.
Published: (2024)

Syntactic Control of Language Models by Posterior Inference
by: Xefteri, Vicky, et al.
Published: (2025)

Rethinking Pruning Large Language Models: Benefits and Pitfalls of Reconstruction Error Minimization
by: Shin, Sungbin, et al.
Published: (2024)

PIP: Perturbation-based Iterative Pruning for Large Language Models
by: Cao, Yi, et al.
Published: (2025)

Compact Language Models via Pruning and Knowledge Distillation
by: Muralidharan, Saurav, et al.
Published: (2024)

High-Layer Attention Pruning with Rescaling
by: Liu, Songtao, et al.
Published: (2025)

A2SF: Accumulative Attention Scoring with Forgetting Factor for Token Pruning in Transformer Decoder
by: Jo, Hyun-rae, et al.
Published: (2024)

Pruning and Distilling Mixture-of-Experts into Dense Language Models
by: Kim, Junhyuck, et al.
Published: (2026)

A Training-free Sub-quadratic Cost Transformer Model Serving Framework With Hierarchically Pruned Attention
by: Lee, Heejun, et al.
Published: (2024)

$K$-MSHC: Unmasking Minimally Sufficient Head Circuits in Large Language Models with Experiments on Syntactic Classification Tasks
by: Chowdhary, Pratim, et al.
Published: (2025)

Think Before You Prune: Self-Reflective Structured Pruning for Reasoning Language Models
by: Wang, Ziyan, et al.
Published: (2025)

Seeing Syntax: Uncovering Syntactic Learning Limitations in Vision-Language Models
by: Dumpala, Sri Harsha, et al.
Published: (2024)

Syntactic and Semantic Control of Large Language Models via Sequential Monte Carlo
by: Loula, João, et al.
Published: (2025)

Deterministic Differentiable Structured Pruning for Large Language Models
by: Huang, Weiyu, et al.
Published: (2026)

Twilight: Adaptive Attention Sparsity with Hierarchical Top-$p$ Pruning
by: Lin, Chaofan, et al.
Published: (2025)

Large Language Model Pruning
by: Huang, Hanjuan, et al.
Published: (2024)

Dual Encoder: Exploiting the Potential of Syntactic and Semantic for Aspect Sentiment Triplet Extraction
by: Zhao, Xiaowei, et al.
Published: (2024)

Bypass Back-propagation: Optimization-based Structural Pruning for Large Language Models via Policy Gradient
by: Gao, Yuan, et al.
Published: (2024)

DarwinLM: Evolutionary Structured Pruning of Large Language Models
by: Tang, Shengkun, et al.
Published: (2025)

Frustratingly Easy Task-aware Pruning for Large Language Models
by: Tian, Yuanhe, et al.
Published: (2025)

Detecting and Pruning Prominent but Detrimental Neurons in Large Language Models
by: Ali, Ameen, et al.
Published: (2025)

Fast and Effective Weight Update for Pruned Large Language Models
by: Boža, Vladimír
Published: (2024)

Dissecting Language Models: Machine Unlearning via Selective Pruning
by: Pochinkov, Nicholas, et al.
Published: (2024)

Extracting Rule-based Descriptions of Attention Features in Transformers
by: Friedman, Dan, et al.
Published: (2025)

Interpretable Syntactic Representations Enable Hierarchical Word Vectors
by: Silwal, Biraj
Published: (2024)

Dynamic Context Pruning for Efficient and Interpretable Autoregressive Transformers
by: Anagnostidis, Sotiris, et al.
Published: (2023)

Transferring Textual Preferences to Vision-Language Understanding through Model Merging
by: Li, Chen-An, et al.
Published: (2025)

Latent Concept Disentanglement in Transformer-based Language Models
by: Hong, Guan Zhe, et al.
Published: (2025)

STUN: Structured-Then-Unstructured Pruning for Scalable MoE Pruning
by: Lee, Jaeseong, et al.
Published: (2024)

On Importance of Layer Pruning for Smaller BERT Models and Low Resource Languages
by: Shirke, Mayur, et al.
Published: (2025)

Olica: Efficient Structured Pruning of Large Language Models without Retraining
by: He, Jiujun, et al.
Published: (2025)

Self-Data Distillation for Recovering Quality in Pruned Large Language Models
by: Thangarasa, Vithursan, et al.
Published: (2024)

Language Model-Driven Data Pruning Enables Efficient Active Learning
by: Azeemi, Abdul Hameed, et al.
Published: (2024)

Efficient Post-Training Pruning of Large Language Models with Statistical Correction
by: Yu, Peiqi, et al.
Published: (2026)

DISP-LLM: Dimension-Independent Structural Pruning for Large Language Models
by: Gao, Shangqian, et al.
Published: (2024)

IDEA Prune: An Integrated Enlarge-and-Prune Pipeline in Generative Language Model Pretraining
by: Li, Yixiao, et al.
Published: (2025)

Probe Pruning: Accelerating LLMs through Dynamic Pruning via Model-Probing
by: Le, Qi, et al.
Published: (2025)

Demystifying When Pruning Works via Representation Hierarchies
by: He, Shwai, et al.
Published: (2026)

APT: Adaptive Pruning and Tuning Pretrained Language Models for Efficient Training and Inference
by: Zhao, Bowen, et al.
Published: (2024)

Structural Pruning of Pre-trained Language Models via Neural Architecture Search
by: Klein, Aaron, et al.
Published: (2024)