Cover Image

First Activations Matter: Training-Free Methods for Dynamic Activation in Large Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Ma, Chi, Huang, Mincong, Zhang, Ying, Wang, Chao, Wang, Yujie, Yu, Lei, Liu, Chuan, Lin, Wei
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Dynamic Activation Pitfalls in LLaMA Models: An Empirical Study
by: Ma, Chi, et al.
Published: (2024)

Sparse Brains are Also Adaptive Brains: Cognitive-Load-Aware Dynamic Activation for LLMs
by: Yang, Yiheng, et al.
Published: (2025)

Training-Free Activation Sparsity in Large Language Models
by: Liu, James, et al.
Published: (2024)

QQQ: Quality Quattuor-Bit Quantization for Large Language Models
by: Zhang, Ying, et al.
Published: (2024)

Re-evaluating the Memory-balanced Pipeline Parallelism: BPipe
by: Huang, Mincong, et al.
Published: (2024)

MOYU: A Theoretical Study on Massive Over-activation Yielded Uplifts in LLMs
by: Ma, Chi, et al.
Published: (2024)

Q-Sparse: All Large Language Models can be Fully Sparsely-Activated
by: Wang, Hongyu, et al.
Published: (2024)

Polynomial Composition Activations: Unleashing the Dynamics of Large Language Models
by: Zhuo, Zhijian, et al.
Published: (2024)

Pruning Large Language Models to Intra-module Low-rank Architecture with Transitional Activations
by: Shen, Bowen, et al.
Published: (2024)

Uncovering Safety Risks of Large Language Models through Concept Activation Vector
by: Xu, Zhihao, et al.
Published: (2024)

Massive Activations in Large Language Models
by: Sun, Mingjie, et al.
Published: (2024)

Measuring Maximum Activations in Open Large Language Models
by: Chen, Luxuan, et al.
Published: (2026)

Direct Simultaneous Translation Activation for Large Audio-Language Models
by: Zhang, Pei, et al.
Published: (2025)

Activation-Guided Consensus Merging for Large Language Models
by: Yao, Yuxuan, et al.
Published: (2025)

Actial: Activate Spatial Reasoning Ability of Multimodal Large Language Models
by: Zhan, Xiaoyu, et al.
Published: (2025)

Contextualized Automatic Speech Recognition with Dynamic Vocabulary Prediction and Activation
by: Lin, Zhennan, et al.
Published: (2025)

From Attention to Activation: Unravelling the Enigmas of Large Language Models
by: Kaul, Prannay, et al.
Published: (2024)

SEAP: Training-free Sparse Expert Activation Pruning Unlock the Brainpower of Large Language Models
by: Liang, Xun, et al.
Published: (2025)

Controlling Large Language Models Through Concept Activation Vectors
by: Zhang, Hanyu, et al.
Published: (2025)

Trojan Activation Attack: Red-Teaming Large Language Models using Activation Steering for Safety-Alignment
by: Wang, Haoran, et al.
Published: (2023)

Training-Free Multimodal Large Language Model Orchestration
by: Xie, Tianyu, et al.
Published: (2025)

Astra: Activation-Space Tail-Eigenvector Low-Rank Adaptation of Large Language Models
by: Liu, Kainan, et al.
Published: (2026)

SASQ: Static Activation Scaling for Quantization-Aware Training in Large Language Models
by: Mao, Shizhuo, et al.
Published: (2025)

Stable and Explainable Personality Trait Evaluation in Large Language Models with Internal Activations
by: Ma, Xiaoxu, et al.
Published: (2026)

Mitigating Over-Refusal in Aligned Large Language Models via Inference-Time Activation Energy
by: Jiang, Eric Hanchen, et al.
Published: (2025)

Achieving Sparse Activation in Small Language Models
by: Song, Jifeng, et al.
Published: (2024)

Activations as Features: Probing LLMs for Generalizable Essay Scoring Representations
by: Chi, Jinwei, et al.
Published: (2025)

LF-Steering: Latent Feature Activation Steering for Enhancing Semantic Consistency in Large Language Models
by: Yang, Jingyuan, et al.
Published: (2025)

Linear Dynamics in the RLVR Training of Large Language Models
by: Wang, Tianle, et al.
Published: (2026)

Activation-Informed Merging of Large Language Models
by: Nobari, Amin Heyrani, et al.
Published: (2025)

ActTail: Global Activation Sparsity in Large Language Models
by: Hou, Wenwen, et al.
Published: (2026)

Multi-property Steering of Large Language Models with Dynamic Activation Composition
by: Scalena, Daniel, et al.
Published: (2024)

Premise Order Matters in Reasoning with Large Language Models
by: Chen, Xinyun, et al.
Published: (2024)

A Single Layer to Explain Them All:Understanding Massive Activations in Large Language Models
by: Shi, Zeru, et al.
Published: (2026)

Adaptive Activation Steering: A Tuning-Free LLM Truthfulness Improvement Method for Diverse Hallucinations Categories
by: Wang, Tianlong, et al.
Published: (2024)

Reasoning-preserved Efficient Distillation of Large Language Models via Activation-aware Initialization
by: He, Junlin, et al.
Published: (2026)

Self-Enhanced Reasoning Training: Activating Latent Reasoning in Small Models for Enhanced Reasoning Distillation
by: Zhang, Yong, et al.
Published: (2025)

Exploring Activation Patterns of Parameters in Language Models
by: Wang, Yudong, et al.
Published: (2024)

A Systematic Study of Training-Free Methods for Trustworthy Large Language Models
by: Si, Wai Man, et al.
Published: (2026)

Towards Faithful Natural Language Explanations: A Study Using Activation Patching in Large Language Models
by: Yeo, Wei Jie, et al.
Published: (2024)