Cover Image

Massive Activations in Large Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Sun, Mingjie, Chen, Xinlei, Kolter, J. Zico, Liu, Zhuang
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

A Simple and Effective Pruning Approach for Large Language Models
by: Sun, Mingjie, et al.
Published: (2023)

FUSE-ing Language Models: Zero-Shot Adapter Discovery for Prompt Optimization Across Tokenizers
by: Williams, Joshua Nathaniel, et al.
Published: (2024)

Idiosyncrasies in Large Language Models
by: Sun, Mingjie, et al.
Published: (2025)

Forcing Diffuse Distributions out of Language Models
by: Zhang, Yiming, et al.
Published: (2024)

Predicting the Performance of Black-box LLMs through Follow-up Queries
by: Sam, Dylan, et al.
Published: (2025)

Compressed Sensing for Capability Localization in Large Language Models
by: Bair, Anna, et al.
Published: (2026)

Context-Parametric Inversion: Why Instruction Finetuning Can Worsen Context Reliance
by: Goyal, Sachin, et al.
Published: (2024)

Mimetic Initialization Helps State Space Models Learn to Recall
by: Trockman, Asher, et al.
Published: (2024)

Existing Large Language Model Unlearning Evaluations Are Inconclusive
by: Feng, Zhili, et al.
Published: (2025)

Looking beyond the next token
by: Thankaraj, Abitha, et al.
Published: (2025)

TOFU: A Task of Fictitious Unlearning for LLMs
by: Maini, Pratyush, et al.
Published: (2024)

Rethinking LLM Memorization through the Lens of Adversarial Compression
by: Schwarzschild, Avi, et al.
Published: (2024)

Base Models Look Human To AI Detectors
by: Xu, Yixuan Even, et al.
Published: (2026)

Not All Rollouts are Useful: Down-Sampling Rollouts in LLM Reinforcement Learning
by: Xu, Yixuan Even, et al.
Published: (2025)

T-MARS: Improving Visual Representations by Circumventing Text Feature Learning
by: Maini, Pratyush, et al.
Published: (2023)

AcceleratedLiNGAM: Learning Causal DAGs at the speed of GPUs
by: Akinwande, Victor, et al.
Published: (2024)

Mimetic Initialization of MLPs
by: Trockman, Asher, et al.
Published: (2026)

One-Step Diffusion Distillation via Deep Equilibrium Models
by: Geng, Zhengyang, et al.
Published: (2023)

ActTail: Global Activation Sparsity in Large Language Models
by: Hou, Wenwen, et al.
Published: (2026)

Beyond Size: How Gradients Shape Pruning Decisions in Large Language Models
by: Das, Rocktim Jyoti, et al.
Published: (2023)

Massive Editing for Large Language Models via Meta Learning
by: Tan, Chenmien, et al.
Published: (2023)

First Activations Matter: Training-Free Methods for Dynamic Activation in Large Language Models
by: Ma, Chi, et al.
Published: (2024)

Training a Generally Curious Agent
by: Tajwar, Fahim, et al.
Published: (2025)

Reasoning-preserved Efficient Distillation of Large Language Models via Activation-aware Initialization
by: He, Junlin, et al.
Published: (2026)

Antidistillation Fingerprinting
by: Xu, Yixuan Even, et al.
Published: (2026)

Diffusing Differentiable Representations
by: Savani, Yash, et al.
Published: (2024)

Robust and Scalable Model Editing for Large Language Models
by: Chen, Yingfa, et al.
Published: (2024)

Exploring Scaling Laws for Local SGD in Large Language Model Training
by: He, Qiaozhi, et al.
Published: (2024)

Test-Time Adaptation Induces Stronger Accuracy and Agreement-on-the-Line
by: Kim, Eungyeup, et al.
Published: (2023)

Sparsing Law: Towards Large Language Models with Greater Activation Sparsity
by: Luo, Yuqi, et al.
Published: (2024)

Hierarchical Orthogonal Residual Spread for Precise Massive Editing in Large Language Models
by: Gu, Xiaojie, et al.
Published: (2026)

FacLens: Transferable Probe for Foreseeing Non-Factuality in Fact-Seeking Question Answering of Large Language Models
by: Wang, Yanling, et al.
Published: (2024)

CoreInfer: Accelerating Large Language Model Inference with Semantics-Inspired Adaptive Sparse Activation
by: Wang, Qinsi, et al.
Published: (2024)

On the Thinking-Language Modeling Gap in Large Language Models
by: Liu, Chenxi, et al.
Published: (2025)

LOLA -- An Open-Source Massively Multilingual Large Language Model
by: Srivastava, Nikit, et al.
Published: (2024)

Large Language Models as Agents in Two-Player Games
by: Liu, Yang, et al.
Published: (2024)

Steering Language Models With Activation Engineering
by: Turner, Alexander Matt, et al.
Published: (2023)

Q-Sparse: All Large Language Models can be Fully Sparsely-Activated
by: Wang, Hongyu, et al.
Published: (2024)

Jailbreak Attacks and Defenses Against Large Language Models: A Survey
by: Yi, Sibo, et al.
Published: (2024)

Stronger Normalization-Free Transformers
by: Chen, Mingzhi, et al.
Published: (2025)