Cover Image

The SuperActivator Mechanism: Transformers Concentrate Reliable Concept Signals in the Tail

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Goldberg, Cassandra, Kim, Chaehyeon, Stein, Adam, Wong, Eric
Format:	Preprint
Published:	2025
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Towards Compositionality in Concept Learning
by: Stein, Adam, et al.
Published: (2024)

In-Place Feedback: Reliable Refinement for Multi-Turn Expert-LLM Collaboration
by: Choi, Youngbin, et al.
Published: (2025)

Sum-of-Checks: Structured Reasoning for Surgical Safety with Large Vision-Language Models
by: You, Weiqiu, et al.
Published: (2026)

The Road to Generalizable Neuro-Symbolic Learning Should be Paved with Foundation Models
by: Stein, Adam, et al.
Published: (2025)

Camera Agnostic Two-Head Network for Ego-Lane Inference
by: Song, Chaehyeon, et al.
Published: (2024)

Once Upon an Input: Reasoning via Per-Instance Program Synthesis
by: Stein, Adam, et al.
Published: (2025)

Instruction Following by Principled Boosting Attention of Large Language Models
by: Guardieiro, Vitoria, et al.
Published: (2025)

Kernelized Concept Erasure
by: Ravfogel, Shauli, et al.
Published: (2022)

Where's the Bug? Attention Probing for Scalable Fault Localization
by: Stein, Adam, et al.
Published: (2025)

Linear Adversarial Concept Erasure
by: Ravfogel, Shauli, et al.
Published: (2022)

On The Variability of Concept Activation Vectors
by: Wenkmann, Julia, et al.
Published: (2025)

TorchQL: A Programming Framework for Integrity Constraints in Machine Learning
by: Naik, Aaditya, et al.
Published: (2023)

Canonical Tail Dependence for Soft Extremal Clustering of Multichannel Brain Signals
by: Talento, Mara Sherlin, et al.
Published: (2025)

Causally Reliable Concept Bottleneck Models
by: De Felice, Giovanni, et al.
Published: (2025)

Provably Transformers Harness Multi-Concept Word Semantics for Efficient In-Context Learning
by: Bu, Dake, et al.
Published: (2024)

Concepts Whisper While Syntax Shouts: Spectral Anti-Concentration and the Dual Geometry of Transformer Representations
by: Acharya, Pratyush, et al.
Published: (2026)

ActTail: Global Activation Sparsity in Large Language Models
by: Hou, Wenwen, et al.
Published: (2026)

Generalized Policy Gradient with History-Aware Decision Transformer for Reliable Routing over Graph Signals
by: Wei, Xing, et al.
Published: (2025)

The FIX Benchmark: Extracting Features Interpretable to eXperts
by: Jin, Helen, et al.
Published: (2024)

Beyond Gaussian Initializations: Signal Preserving Weight Initialization for Odd-Sigmoid Activations
by: Lee, Hyunwoo, et al.
Published: (2025)

Concentration of General Stochastic Approximation Under Heavy-Tailed Markovian Noise
by: Agrawal, Shubhada, et al.
Published: (2026)

Sharp Concentration Inequalities: Phase Transition and Mixing of Orlicz Tails with Variance
by: Shen, Yinan, et al.
Published: (2026)

Interpretability for Multimodal Emotion Recognition using Concept Activation Vectors
by: Asokan, Ashish Ramayee, et al.
Published: (2022)

Tail Annealing for Heavy-Tailed Flow Matching
by: Pachebat, Jean
Published: (2026)

Ring-A-Bell! How Reliable are Concept Removal Methods for Diffusion Models?
by: Tsai, Yu-Lin, et al.
Published: (2023)

Unsupervised Concept Drift Detection based on Parallel Activations of Neural Network
by: Komorniczak, Joanna, et al.
Published: (2024)

Signals, Concepts, and Laws: Toward Universal, Explainable Time-Series Forecasting
by: Ma, Hongwei, et al.
Published: (2025)

Frequency-Integrated Transformer for Arbitrary-Scale Super-Resolution
by: Wang, Xufei, et al.
Published: (2025)

Efficient Differentiable Causal Discovery via Reliable Super-Structure Learning
by: Ma, Pingchuan, et al.
Published: (2026)

End-to-End Verifiable Decentralized Federated Learning
by: Lee, Chaehyeon, et al.
Published: (2024)

RingFormer: Rethinking Recurrent Transformer with Adaptive Level Signals
by: Heo, Jaemu, et al.
Published: (2025)

Best-of-Tails: Bridging Optimism and Pessimism in Inference-Time Alignment
by: Hsu, Hsiang, et al.
Published: (2026)

How Does Fourier Analysis Network Work? A Mechanism Analysis and a New Dual-Activation Layer Proposal
by: Jeong, Sam, et al.
Published: (2025)

Optimizing Multidimensional Scaling in Gini Metric Spaces
by: Mussard, Cassandra, et al.
Published: (2026)

T-MLP: Tailed Multi-Layer Perceptron for Level-of-Detail Signal Representation
by: Yang, Chuanxiang, et al.
Published: (2025)

The Radio-Frequency Transformer for Signal Separation
by: Lifar, Egor, et al.
Published: (2026)

Concept Unlearning via Cross-Attention Activation Projection for Diffusion Models
by: Moon, Saemi, et al.
Published: (2026)

BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models
by: Ben-Zaken, Elad, et al.
Published: (2021)

Sparsified-Learning for High-Dimensional Heavy-Tailed Locally Stationary Time Series, Concentration and Oracle Inequalities
by: Wang, Yingjie, et al.
Published: (2025)

Data-Driven Sequential Sampling for Tail Risk Mitigation
by: Ahn, Dohyun, et al.
Published: (2025)