Cover Image

Sparsing Law: Towards Large Language Models with Greater Activation Sparsity

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Luo, Yuqi, Song, Chenyang, Han, Xu, Chen, Yingfa, Xiao, Chaojun, Meng, Xiaojun, Deng, Liqun, Wei, Jiansheng, Liu, Zhiyuan, Sun, Maosong
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Computation and Language I.2.7
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

ProSparse: Introducing and Enhancing Intrinsic Activation Sparsity within Large Language Models
by: Song, Chenyang, et al.
Published: (2024)

ConPET: Continual Parameter-Efficient Tuning for Large Language Models
by: Song, Chenyang, et al.
Published: (2023)

Towards Greater Leverage: Scaling Laws for Efficient Mixture-of-Experts Language Models
by: Tian, Changxin, et al.
Published: (2025)

CRISP: Persistent Concept Unlearning via Sparse Autoencoders
by: Ashuach, Tomer, et al.
Published: (2025)

Low-Resource Court Judgment Summarization for Common Law Systems
by: Liu, Shuaiqi, et al.
Published: (2024)

Towards Alignment-Centric Paradigm: A Survey of Instruction Tuning in Large Language Models
by: Han, Xudong, et al.
Published: (2025)

Learning to Generate Structured Output with Schema Reinforcement Learning
by: Lu, Yaxi, et al.
Published: (2025)

RUQuant: Towards Refining Uniform Quantization for Large Language Models
by: Liu, Han, et al.
Published: (2026)

Toward Architecture-Aware Evaluation Metrics for LLM Agents
by: Souza, Débora, et al.
Published: (2026)

LoRS: Efficient Low-Rank Adaptation for Sparse Large Language Model
by: Hu, Yuxuan, et al.
Published: (2025)

Text-Based Approaches to Item Difficulty Modeling in Large-Scale Assessments: A Systematic Review
by: Peters, Sydney, et al.
Published: (2025)

Towards Fundamental Language Models: Does Linguistic Competence Scale with Model Size?
by: Collado-Montañez, Jaime, et al.
Published: (2025)

Preventing Safety Drift in Large Language Models via Coupled Weight and Activation Constraints
by: Peng, Songping, et al.
Published: (2026)

Merge-Bench: Resolve Merge Conflicts with Large Language Models
by: Schesch, Benedikt, et al.
Published: (2026)

Automated Bug Triaging using Instruction-Tuned Large Language Models
by: Kiashemshaki, Kiana, et al.
Published: (2025)

SecEmb: Sparsity-Aware Secure Federated Learning of On-Device Recommender System with Large Embedding
by: Mai, Peihua, et al.
Published: (2025)

Towards Human Understanding of Paraphrase Types in Large Language Models
by: Meier, Dominik, et al.
Published: (2024)

AI-MASLD Metabolic Dysfunction and Information Steatosis of Large Language Models in Unstructured Clinical Narratives
by: Shen, Yuan, et al.
Published: (2025)

PowLU: An Activation Function for Stable Pre-Training of LLMs
by: Jiang, Peijie, et al.
Published: (2026)

Aligning Large Language Models for Faithful Integrity Against Opposing Argument
by: Zhao, Yong, et al.
Published: (2025)

RomanLens: The Role Of Latent Romanization In Multilinguality In LLMs
by: Saji, Alan, et al.
Published: (2025)

Pre-trained Models Perform the Best When Token Distributions Follow Zipf's Law
by: He, Yanjin, et al.
Published: (2025)

The Foundational Capabilities of Large Language Models in Predicting Postoperative Risks Using Clinical Notes
by: Alba, Charles, et al.
Published: (2024)

Hardware Co-Design Scaling Laws via Roofline Modelling for On-Device LLMs
by: Sun, Luoyang, et al.
Published: (2026)

POISONCRAFT: Practical Poisoning of Retrieval-Augmented Generation for Large Language Models
by: Shao, Yangguang, et al.
Published: (2025)

EnDive: A Cross-Dialect Benchmark for Fairness and Performance in Large Language Models
by: Gupta, Abhay, et al.
Published: (2025)

Scaling Laws for Forgetting When Fine-Tuning Large Language Models
by: Kalajdzievski, Damjan
Published: (2024)

AsyncTLS: Efficient Generative LLM Inference with Asynchronous Two-level Sparse Attention
by: Hu, Yuxuan, et al.
Published: (2026)

Learning From Failure: Integrating Negative Examples when Fine-tuning Large Language Models as Agents
by: Wang, Renxi, et al.
Published: (2024)

Proactive Agent: Shifting LLM Agents from Reactive Responses to Active Assistance
by: Lu, Yaxi, et al.
Published: (2024)

SEPTQ: A Simple and Effective Post-Training Quantization Paradigm for Large Language Models
by: Liu, Han, et al.
Published: (2026)

The Unified Cognitive Consciousness Theory for Language Models: Anchoring Semantics, Thresholds of Activation, and Emergent Reasoning
by: Chang, Edward Y., et al.
Published: (2025)

Large Language Model (LLM) Bias Index -- LLMBI
by: Oketunji, Abiodun Finbarrs, et al.
Published: (2023)

Engineering A Large Language Model From Scratch
by: Oketunji, Abiodun Finbarrs
Published: (2024)

Entropy-Based Measurement of Value Drift and Alignment Work in Large Language Models
by: Fadli, Samih
Published: (2025)

An Investigation of Neuron Activation as a Unified Lens to Explain Chain-of-Thought Eliciting Arithmetic Reasoning of LLMs
by: Rai, Daking, et al.
Published: (2024)

Towards ethical multimodal systems
by: Roger, Alexis, et al.
Published: (2023)

SeLeRoSa: Sentence-Level Romanian Satire Detection Dataset
by: Smădu, Răzvan-Alexandru, et al.
Published: (2025)

Auditing Meta-Cognitive Hallucinations in Reasoning Large Language Models
by: Lu, Haolang, et al.
Published: (2025)

Towards Massive Multilingual Holistic Bias
by: Tan, Xiaoqing Ellen, et al.
Published: (2024)