Cover Image

CATS: Contextually-Aware Thresholding for Sparsity in Large Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Lee, Donghyun, Lee, Je-Yong, Zhang, Genghan, Tiwari, Mo, Mirhoseini, Azalia
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

On the Role of Temperature Sampling in Test-Time Scaling
by: Wu, Yuheng, et al.
Published: (2025)

Mixture-of-Mamba: Enhancing Multi-Modal State-Space Models with Modality-Aware Sparsity
by: Liang, Weixin, et al.
Published: (2025)

ShadowLLM: Predictor-based Contextual Sparsity for Large Language Models
by: Akhauri, Yash, et al.
Published: (2024)

Synthetic Data Generation & Multi-Step RL for Reasoning & Tool Use
by: Goldie, Anna, et al.
Published: (2025)

CAST: Continuous and Differentiable Semi-Structured Sparsity-Aware Training for Large Language Models
by: Huang, Weiyu, et al.
Published: (2025)

Advancing Event Forecasting through Massive Training of Large Language Models: Challenges, Solutions, and Broader Impacts
by: Lee, Sang-Woo, et al.
Published: (2025)

Federation of Experts: Communication Efficient Distributed Inference for Large Language Models
by: Abdurrahman, Muhammad Shahir, et al.
Published: (2026)

Dependency-Aware Semi-Structured Sparsity of GLU Variants in Large Language Models
by: Guo, Zhiyu, et al.
Published: (2024)

ActTail: Global Activation Sparsity in Large Language Models
by: Hou, Wenwen, et al.
Published: (2026)

SPP: Sparsity-Preserved Parameter-Efficient Fine-Tuning for Large Language Models
by: Lu, Xudong, et al.
Published: (2024)

LLM-Barber: Block-Aware Rebuilder for Sparsity Mask in One-Shot for Large Language Models
by: Su, Yupeng, et al.
Published: (2024)

Sequence-Level Leakage Risk of Training Data in Large Language Models
by: Tiwari, Trishita, et al.
Published: (2024)

Learn To be Efficient: Build Structured Sparsity in Large Language Models
by: Zheng, Haizhong, et al.
Published: (2024)

AMGPT: a Large Language Model for Contextual Querying in Additive Manufacturing
by: Chandrasekhar, Achuth, et al.
Published: (2024)

BESA: Pruning Large Language Models with Blockwise Parameter-Efficient Sparsity Allocation
by: Xu, Peng, et al.
Published: (2024)

Rethinking Pruning Large Language Models: Benefits and Pitfalls of Reconstruction Error Minimization
by: Shin, Sungbin, et al.
Published: (2024)

Pointer-Guided Pre-Training: Infusing Large Language Models with Paragraph-Level Contextual Awareness
by: Hillebrand, Lars, et al.
Published: (2024)

LoSA: Locality Aware Sparse Attention for Block-Wise Diffusion Language Models
by: Xi, Haocheng, et al.
Published: (2026)

MaskLLM: Learnable Semi-Structured Sparsity for Large Language Models
by: Fang, Gongfan, et al.
Published: (2024)

AutoGuide: Automated Generation and Selection of Context-Aware Guidelines for Large Language Model Agents
by: Fu, Yao, et al.
Published: (2024)

Adaptive Task Vectors for Large Language Models
by: Kang, Joonseong, et al.
Published: (2025)

CHESS: Contextual Harnessing for Efficient SQL Synthesis
by: Talaei, Shayan, et al.
Published: (2024)

Merging Methods for Multilingual Knowledge Editing for Large Language Models: An Empirical Odyssey
by: Lee, Kunil, et al.
Published: (2026)

Hansel: Output Length Controlling Framework for Large Language Models
by: Song, Seoha, et al.
Published: (2024)

Cartridges: Lightweight and general-purpose long context representations via self-study
by: Eyuboglu, Sabri, et al.
Published: (2025)

Investigating Symbolic Capabilities of Large Language Models
by: Dave, Neisarg, et al.
Published: (2024)

That Chip Has Sailed: A Critique of Unfounded Skepticism Around AI for Chip Design
by: Goldie, Anna, et al.
Published: (2024)

Think, Prune, Train, Improve: Scaling Reasoning without Scaling Models
by: Costello, Caia, et al.
Published: (2025)

Energy Considerations of Large Language Model Inference and Efficiency Optimizations
by: Fernandez, Jared, et al.
Published: (2025)

Sparsing Law: Towards Large Language Models with Greater Activation Sparsity
by: Luo, Yuqi, et al.
Published: (2024)

Estimating Privacy Leakage of Augmented Contextual Knowledge in Language Models
by: Flemings, James, et al.
Published: (2024)

C$^2$-Cite: Contextual-Aware Citation Generation for Attributed Large Language Models
by: Yu, Yue, et al.
Published: (2025)

E-Sparse: Boosting the Large Language Model Inference through Entropy-based N:M Sparsity
by: Li, Yun, et al.
Published: (2023)

Contextual Text Denoising with Masked Language Models
by: Sun, Yifu, et al.
Published: (2019)

Learning and Transferring Sparse Contextual Bigrams with Linear Transformers
by: Ren, Yunwei, et al.
Published: (2024)

Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inference
by: Tang, Jiaming, et al.
Published: (2024)

Astra: A Multi-Agent System for GPU Kernel Performance Optimization
by: Wei, Anjiang, et al.
Published: (2025)

Self-Training Large Language Models with Confident Reasoning
by: Jang, Hyosoon, et al.
Published: (2025)

SAP: Syntactic Attention Pruning for Transformer-based Language Models
by: Lee, Tzu-Yun, et al.
Published: (2025)

DebateBench: A Challenging Long Context Reasoning Benchmark For Large Language Models
by: Tiwari, Utkarsh, et al.
Published: (2025)