Image de couverture de livre

Achieving Sparse Activation in Small Language Models

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Song, Jifeng, Huang, Kai, Yin, Xiangyu, Yang, Boyuan, Gao, Wei
Format:	Preprint
Publié:	2024
Sujets:	Computation and Language Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Never Start from Scratch: Expediting On-Device LLM Personalization via Explainable Model Selection
par: Wang, Haoming, et autres
Publié: (2025)

FALCON: Fine-grained Activation Manipulation by Contrastive Orthogonal Unalignment for Large Language Model
par: Hu, Jinwei, et autres
Publié: (2025)

ProtSAE: Disentangling and Interpreting Protein Language Models via Semantically-Guided Sparse Autoencoders
par: Liu, Xiangyu, et autres
Publié: (2025)

HintMR: Eliciting Stronger Mathematical Reasoning in Small Language Models
par: Hossain, Jawad, et autres
Publié: (2026)

Can Small Language Models Help Large Language Models Reason Better?: LM-Guided Chain-of-Thought
par: Lee, Jooyoung, et autres
Publié: (2024)

FedMKT: Federated Mutual Knowledge Transfer for Large and Small Language Models
par: Fan, Tao, et autres
Publié: (2024)

Federated Co-tuning Framework for Large and Small Language Models
par: Fan, Tao, et autres
Publié: (2024)

Ensemble Bayesian Inference: Leveraging Small Language Models to Achieve LLM-level Accuracy in Profile Matching Tasks
par: Sato, Haru-Tada, et autres
Publié: (2025)

Small Language Model as Data Prospector for Large Language Model
par: Ni, Shiwen, et autres
Publié: (2024)

InfiR : Crafting Effective Small Language Models and Multimodal Small Language Models in Reasoning
par: Xie, Congkai, et autres
Publié: (2025)

Small Models, Big Results: Achieving Superior Intent Extraction through Decomposition
par: Cohen, Danielle, et autres
Publié: (2025)

Small Language Model Helps Resolve Semantic Ambiguity of LLM Prompt
par: Huang, Zhenzhen, et autres
Publié: (2026)

SASQ: Static Activation Scaling for Quantization-Aware Training in Large Language Models
par: Mao, Shizhuo, et autres
Publié: (2025)

LightReasoner: Can Small Language Models Teach Large Language Models Reasoning?
par: Wang, Jingyuan, et autres
Publié: (2025)

Sparse Shift Autoencoders for Identifying Concepts from Large Language Model Activations
par: Joshi, Shruti, et autres
Publié: (2025)

TinyLlama: An Open-Source Small Language Model
par: Zhang, Peiyuan, et autres
Publié: (2024)

$μ$KE: Matryoshka Unstructured Knowledge Editing of Large Language Models
par: Su, Zian, et autres
Publié: (2025)

Refining Salience-Aware Sparse Fine-Tuning Strategies for Language Models
par: Liu, Xinxin, et autres
Publié: (2024)

Trojan Activation Attack: Red-Teaming Large Language Models using Activation Steering for Safety-Alignment
par: Wang, Haoran, et autres
Publié: (2023)

Igniting Creative Writing in Small Language Models: LLM-as-a-Judge versus Multi-Agent Refined Rewards
par: Wei, Xiaolong, et autres
Publié: (2025)

GraphGPT: Graph Instruction Tuning for Large Language Models
par: Tang, Jiabin, et autres
Publié: (2023)

Pruning Large Language Models with Semi-Structural Adaptive Sparse Training
par: Huang, Weiyu, et autres
Publié: (2024)

PhyT2V: LLM-Guided Iterative Self-Refinement for Physics-Grounded Text-to-Video Generation
par: Xue, Qiyao, et autres
Publié: (2024)

An Empirical Study of SFT-DPO Interaction and Parameterization in Small Language Models
par: Feng, Yuming, et autres
Publié: (2026)

ProSparse: Introducing and Enhancing Intrinsic Activation Sparsity within Large Language Models
par: Song, Chenyang, et autres
Publié: (2024)

SoLA: Leveraging Soft Activation Sparsity and Low-Rank Decomposition for Large Language Model Compression
par: Huang, Xinhao, et autres
Publié: (2026)

Denoising Concept Vectors with Sparse Autoencoders for Improved Language Model Steering
par: Zhao, Haiyan, et autres
Publié: (2025)

ASRU: Activation Steering Meets Reinforcement Unlearning for Multimodal Large Language Models
par: Guang, Jiahui, et autres
Publié: (2026)

SOD: Step-wise On-policy Distillation for Small Language Model Agents
par: Zhong, Qiyong, et autres
Publié: (2026)

Towards Adaptive Mechanism Activation in Language Agent
par: Huang, Ziyang, et autres
Publié: (2024)

From General to Specific: Tailoring Large Language Models for Personalized Healthcare
par: Shi, Ruize, et autres
Publié: (2024)

Achieving Peak Performance for Large Language Models: A Systematic Review
par: Rostam, Zhyar Rzgar K, et autres
Publié: (2024)

SparseDoctor: Towards Efficient Chat Doctor with Mixture of Experts Enhanced Large Language Models
par: Zhang, Jianbin, et autres
Publié: (2025)

Capturing Nuanced Preferences: Preference-Aligned Distillation for Small Language Models
par: Gu, Yanggan, et autres
Publié: (2025)

Hallucination Detection with Small Language Models
par: Cheung, Ming
Publié: (2025)

HySparse: A Hybrid Sparse Attention Architecture with Oracle Token Selection and KV Cache Sharing
par: Gao, Yizhao, et autres
Publié: (2026)

The Spike, the Sparse and the Sink: Anatomy of Massive Activations and Attention Sinks
par: Sun, Shangwen, et autres
Publié: (2026)

A New Pipeline For Generating Instruction Dataset via RAG and Self Fine-Tuning
par: Song, Chih-Wei, et autres
Publié: (2024)

ECLM: Entity Level Language Model for Spoken Language Understanding with Chain of Intent
par: Yin, Shangjian, et autres
Publié: (2024)

Activation Scaling for Steering and Interpreting Language Models
par: Stoehr, Niklas, et autres
Publié: (2024)