Cover Image

Exploring Activation Patterns of Parameters in Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Wang, Yudong, Dai, Damai, Sui, Zhifang
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Language Models Encode the Value of Numbers Linearly
by: Zhu, Fangwei, et al.
Published: (2024)

Not All Demonstration Examples are Equally Beneficial: Reweighting Demonstration Examples for In-Context Learning
by: Yang, Zhe, et al.
Published: (2023)

Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotations
by: Wang, Peiyi, et al.
Published: (2023)

Let the Expert Stick to His Last: Expert-Specialized Fine-Tuning for Sparse Architectural Large Language Models
by: Wang, Zihan, et al.
Published: (2024)

Auxiliary-Loss-Free Load Balancing Strategy for Mixture-of-Experts
by: Wang, Lean, et al.
Published: (2024)

Stabilizing MoE Reinforcement Learning by Aligning Training and Inference Routers
by: Ma, Wenhan, et al.
Published: (2025)

Large Language Models Struggle with Unreasonability in Math Problems
by: Ma, Jingyuan, et al.
Published: (2024)

First Activations Matter: Training-Free Methods for Dynamic Activation in Large Language Models
by: Ma, Chi, et al.
Published: (2024)

Massive Activations in Large Language Models
by: Sun, Mingjie, et al.
Published: (2024)

Steering Language Models With Activation Engineering
by: Turner, Alexander Matt, et al.
Published: (2023)

Efficient Post-Training Pruning of Large Language Models with Statistical Correction
by: Yu, Peiqi, et al.
Published: (2026)

Q-Sparse: All Large Language Models can be Fully Sparsely-Activated
by: Wang, Hongyu, et al.
Published: (2024)

Mitigating the Impact of Outlier Channels for Language Model Quantization with Activation Regularization
by: Nrusimha, Aniruddha, et al.
Published: (2024)

Exploring Parameter-Efficient Fine-Tuning Techniques for Code Generation with Large Language Models
by: Weyssow, Martin, et al.
Published: (2023)

Towards Better RL Training Data Utilization via Second-Order Rollout
by: Yang, Zhe, et al.
Published: (2026)

Shuttle Between the Instructions and the Parameters of Large Language Models
by: Sun, Wangtao, et al.
Published: (2025)

Graph-Based Spectral Decomposition for Parameter Coordination in Language Model Fine-Tuning
by: Zhang, Hanlu, et al.
Published: (2025)

Turbo Sparse: Achieving LLM SOTA Performance with Minimal Activated Parameters
by: Song, Yixin, et al.
Published: (2024)

Towards Stable and Effective Reinforcement Learning for Mixture-of-Experts
by: Zhang, Di, et al.
Published: (2025)

Basis Sharing: Cross-Layer Parameter Sharing for Large Language Model Compression
by: Wang, Jingcun, et al.
Published: (2024)

Spherical Steering: Geometry-Aware Activation Rotation for Language Models
by: You, Zejia, et al.
Published: (2026)

ActTail: Global Activation Sparsity in Large Language Models
by: Hou, Wenwen, et al.
Published: (2026)

Mechanistic Exploration of Backdoored Large Language Model Attention Patterns
by: Baker, Mohammed Abu, et al.
Published: (2025)

Internal Activation Revision: Safeguarding Vision Language Models Without Parameter Update
by: Li, Qing, et al.
Published: (2025)

LongAct: Harnessing Intrinsic Activation Patterns for Long-Context Reinforcement Learning
by: Ping, Bowen, et al.
Published: (2026)

Exploring and Benchmarking the Planning Capabilities of Large Language Models
by: Bohnet, Bernd, et al.
Published: (2024)

CoreInfer: Accelerating Large Language Model Inference with Semantics-Inspired Adaptive Sparse Activation
by: Wang, Qinsi, et al.
Published: (2024)

LangFlow: Continuous Diffusion Rivals Discrete in Language Modeling
by: Chen, Yuxin, et al.
Published: (2026)

Keyframe-oriented Vision Token Pruning: Enhancing Efficiency of Large Vision Language Models on Long-Form Video Processing
by: Liu, Yudong, et al.
Published: (2025)

Parameter-Efficient Tuning Large Language Models for Graph Representation Learning
by: Zhu, Qi, et al.
Published: (2024)

MoLAE: Mixture of Latent Experts for Parameter-Efficient Language Models
by: Liu, Zehua, et al.
Published: (2025)

Adapting Large Language Models for Parameter-Efficient Log Anomaly Detection
by: Lim, Ying Fu, et al.
Published: (2025)

Can Small Language Models Learn, Unlearn, and Retain Noise Patterns?
by: Scaria, Nicy, et al.
Published: (2024)

ExpertWeaver: Unlocking the Inherent MoE in Dense LLMs with GLU Activation Patterns
by: Zhao, Ziyu, et al.
Published: (2026)

Universal Activation Verbalizer: A Unified Framework for Cross-Model Activation Explanation
by: Zhao, Haiyan, et al.
Published: (2026)

A Sparsity Predicting Approach for Large Language Models via Activation Pattern Clustering
by: Dhar, Nobel, et al.
Published: (2025)

Activation Control for Efficiently Eliciting Long Chain-of-thought Ability of Language Models
by: Zhao, Zekai, et al.
Published: (2025)

Distributional Surgery for Language Model Activations
by: Nguyen, Bao, et al.
Published: (2025)

SPP: Sparsity-Preserved Parameter-Efficient Fine-Tuning for Large Language Models
by: Lu, Xudong, et al.
Published: (2024)

Exploring Gender Bias in Large Language Models: An In-depth Dive into the German Language
by: Gnadt, Kristin, et al.
Published: (2025)