Copertina

Achieving Sparse Activation in Small Language Models

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Song, Jifeng, Huang, Kai, Yin, Xiangyu, Yang, Boyuan, Gao, Wei
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Computation and Language Artificial Intelligence
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Never Start from Scratch: Expediting On-Device LLM Personalization via Explainable Model Selection
di: Wang, Haoming, et al.
Pubblicazione: (2025)

FALCON: Fine-grained Activation Manipulation by Contrastive Orthogonal Unalignment for Large Language Model
di: Hu, Jinwei, et al.
Pubblicazione: (2025)

ProtSAE: Disentangling and Interpreting Protein Language Models via Semantically-Guided Sparse Autoencoders
di: Liu, Xiangyu, et al.
Pubblicazione: (2025)

HintMR: Eliciting Stronger Mathematical Reasoning in Small Language Models
di: Hossain, Jawad, et al.
Pubblicazione: (2026)

Can Small Language Models Help Large Language Models Reason Better?: LM-Guided Chain-of-Thought
di: Lee, Jooyoung, et al.
Pubblicazione: (2024)

FedMKT: Federated Mutual Knowledge Transfer for Large and Small Language Models
di: Fan, Tao, et al.
Pubblicazione: (2024)

Federated Co-tuning Framework for Large and Small Language Models
di: Fan, Tao, et al.
Pubblicazione: (2024)

Ensemble Bayesian Inference: Leveraging Small Language Models to Achieve LLM-level Accuracy in Profile Matching Tasks
di: Sato, Haru-Tada, et al.
Pubblicazione: (2025)

Small Language Model as Data Prospector for Large Language Model
di: Ni, Shiwen, et al.
Pubblicazione: (2024)

InfiR : Crafting Effective Small Language Models and Multimodal Small Language Models in Reasoning
di: Xie, Congkai, et al.
Pubblicazione: (2025)

Small Models, Big Results: Achieving Superior Intent Extraction through Decomposition
di: Cohen, Danielle, et al.
Pubblicazione: (2025)

Small Language Model Helps Resolve Semantic Ambiguity of LLM Prompt
di: Huang, Zhenzhen, et al.
Pubblicazione: (2026)

SASQ: Static Activation Scaling for Quantization-Aware Training in Large Language Models
di: Mao, Shizhuo, et al.
Pubblicazione: (2025)

LightReasoner: Can Small Language Models Teach Large Language Models Reasoning?
di: Wang, Jingyuan, et al.
Pubblicazione: (2025)

Sparse Shift Autoencoders for Identifying Concepts from Large Language Model Activations
di: Joshi, Shruti, et al.
Pubblicazione: (2025)

TinyLlama: An Open-Source Small Language Model
di: Zhang, Peiyuan, et al.
Pubblicazione: (2024)

$μ$KE: Matryoshka Unstructured Knowledge Editing of Large Language Models
di: Su, Zian, et al.
Pubblicazione: (2025)

Refining Salience-Aware Sparse Fine-Tuning Strategies for Language Models
di: Liu, Xinxin, et al.
Pubblicazione: (2024)

Trojan Activation Attack: Red-Teaming Large Language Models using Activation Steering for Safety-Alignment
di: Wang, Haoran, et al.
Pubblicazione: (2023)

Igniting Creative Writing in Small Language Models: LLM-as-a-Judge versus Multi-Agent Refined Rewards
di: Wei, Xiaolong, et al.
Pubblicazione: (2025)

GraphGPT: Graph Instruction Tuning for Large Language Models
di: Tang, Jiabin, et al.
Pubblicazione: (2023)

Pruning Large Language Models with Semi-Structural Adaptive Sparse Training
di: Huang, Weiyu, et al.
Pubblicazione: (2024)

PhyT2V: LLM-Guided Iterative Self-Refinement for Physics-Grounded Text-to-Video Generation
di: Xue, Qiyao, et al.
Pubblicazione: (2024)

An Empirical Study of SFT-DPO Interaction and Parameterization in Small Language Models
di: Feng, Yuming, et al.
Pubblicazione: (2026)

ProSparse: Introducing and Enhancing Intrinsic Activation Sparsity within Large Language Models
di: Song, Chenyang, et al.
Pubblicazione: (2024)

SoLA: Leveraging Soft Activation Sparsity and Low-Rank Decomposition for Large Language Model Compression
di: Huang, Xinhao, et al.
Pubblicazione: (2026)

Denoising Concept Vectors with Sparse Autoencoders for Improved Language Model Steering
di: Zhao, Haiyan, et al.
Pubblicazione: (2025)

ASRU: Activation Steering Meets Reinforcement Unlearning for Multimodal Large Language Models
di: Guang, Jiahui, et al.
Pubblicazione: (2026)

SOD: Step-wise On-policy Distillation for Small Language Model Agents
di: Zhong, Qiyong, et al.
Pubblicazione: (2026)

Towards Adaptive Mechanism Activation in Language Agent
di: Huang, Ziyang, et al.
Pubblicazione: (2024)

From General to Specific: Tailoring Large Language Models for Personalized Healthcare
di: Shi, Ruize, et al.
Pubblicazione: (2024)

Achieving Peak Performance for Large Language Models: A Systematic Review
di: Rostam, Zhyar Rzgar K, et al.
Pubblicazione: (2024)

SparseDoctor: Towards Efficient Chat Doctor with Mixture of Experts Enhanced Large Language Models
di: Zhang, Jianbin, et al.
Pubblicazione: (2025)

Capturing Nuanced Preferences: Preference-Aligned Distillation for Small Language Models
di: Gu, Yanggan, et al.
Pubblicazione: (2025)

Hallucination Detection with Small Language Models
di: Cheung, Ming
Pubblicazione: (2025)

HySparse: A Hybrid Sparse Attention Architecture with Oracle Token Selection and KV Cache Sharing
di: Gao, Yizhao, et al.
Pubblicazione: (2026)

The Spike, the Sparse and the Sink: Anatomy of Massive Activations and Attention Sinks
di: Sun, Shangwen, et al.
Pubblicazione: (2026)

A New Pipeline For Generating Instruction Dataset via RAG and Self Fine-Tuning
di: Song, Chih-Wei, et al.
Pubblicazione: (2024)

ECLM: Entity Level Language Model for Spoken Language Understanding with Chain of Intent
di: Yin, Shangjian, et al.
Pubblicazione: (2024)

Activation Scaling for Steering and Interpreting Language Models
di: Stoehr, Niklas, et al.
Pubblicazione: (2024)