Cover Image

Tracing Computation Density in LLMs

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Kervadec, Corentin, Lysova, Iuliia, Macocco, Iuri, Baroni, Marco, Boleda, Gemma
Format:	Preprint
Published:	2026
Subjects:	Computation and Language Artificial Intelligence Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Sparse or Dense? A Mechanistic Estimation of Computation Density in Transformer-based LLMs
by: Kervadec, Corentin, et al.
Published: (2026)

Not a nuisance but a useful heuristic: Outlier dimensions favor frequent tokens in language models
by: Macocco, Iuri, et al.
Published: (2025)

Evil twins are not that evil: Qualitative insights into machine-generated prompts
by: Rakotonirina, Nathanaël Carraz, et al.
Published: (2024)

Prediction hubs are context-informed frequent tokens in LLMs
by: Nielsen, Beatrix M. G., et al.
Published: (2025)

Emergence of a High-Dimensional Abstraction Phase in Language Transformers
by: Cheng, Emily, et al.
Published: (2024)

MemoryPrompt: A Light Wrapper to Improve Context Tracking in Pre-trained Language Models
by: Rakotonirina, Nathanaël Carraz, et al.
Published: (2024)

The Grammar of Transformers: A Systematic Review of Interpretability Research on Syntactic Knowledge in Language Models
by: Graichen, Nora, et al.
Published: (2026)

LLMs as a synthesis between symbolic and distributed approaches to language
by: Boleda, Gemma
Published: (2025)

How do LLMs Compute Verbal Confidence
by: Kumaran, Dharshan, et al.
Published: (2026)

From Growing to Looping: A Unified View of Iterative Computation in LLMs
by: Kapl, Ferdinand, et al.
Published: (2026)

HCAttention: Extreme KV Cache Compression via Heterogeneous Attention Computing for LLMs
by: Yang, Dongquan, et al.
Published: (2025)

MemTrace: Tracing and Attributing Errors in Large Language Model Memory Systems
by: Deng, Xinle, et al.
Published: (2026)

AQUA: Attention via QUery mAgnitudes for Memory and Compute Efficient Inference in LLMs
by: S, Santhosh G, et al.
Published: (2025)

Time Travel in LLMs: Tracing Data Contamination in Large Language Models
by: Golchin, Shahriar, et al.
Published: (2023)

Low-rank finetuning for LLMs: A fairness perspective
by: Das, Saswat, et al.
Published: (2024)

Tracing Uncertainty in Language Model "Reasoning"
by: Grünefeld, Nils, et al.
Published: (2026)

Are You Being Tracked? Discover the Power of Zero-Shot Trajectory Tracing with LLMs!
by: Yang, Huanqi, et al.
Published: (2024)

Adaptive Inference-Time Compute: LLMs Can Predict if They Can Do Better, Even Mid-Generation
by: Manvi, Rohin, et al.
Published: (2024)

Sparse Attention Decomposition Applied to Circuit Tracing
by: Franco, Gabriel, et al.
Published: (2024)

Anatomy of an Idiom: Tracing Non-Compositionality in Language Models
by: Gomes, Andrew
Published: (2025)

Transcoders Trace Visual Grounding and Hallucinations in Vision-Language Models
by: Damianos, Dimitrios, et al.
Published: (2026)

Mechanistic Data Attribution: Tracing the Training Origins of Interpretable LLM Units
by: Chen, Jianhui, et al.
Published: (2026)

Attention Flows: Tracing LLM Conceptual Engagement via Story Summaries
by: Hicke, Rebecca M. M., et al.
Published: (2026)

Tracing the Representation Geometry of Language Models from Pretraining to Post-training
by: Li, Melody Zixuan, et al.
Published: (2025)

The Map of Misbelief: Tracing Intrinsic and Extrinsic Hallucinations Through Attention Patterns
by: Hajji, Elyes, et al.
Published: (2025)

Matryoshka Pilot: Learning to Drive Black-Box LLMs with LLMs
by: Li, Changhao, et al.
Published: (2024)

Can LLMs Help Uncover Insights about LLMs? A Large-Scale, Evolving Literature Analysis of Frontier LLMs
by: Park, Jungsoo, et al.
Published: (2025)

Beyond Behavioural Trade-Offs: Mechanistic Tracing of Pain-Pleasure Decisions in an LLM
by: Bianco, Francesca, et al.
Published: (2026)

ACAR: Adaptive Complexity Routing for Multi-Model Ensembles with Auditable Decision Traces
by: Kumaresan, Ramchand
Published: (2026)

Generalization v.s. Memorization: Tracing Language Models' Capabilities Back to Pretraining Data
by: Wang, Xinyi, et al.
Published: (2024)

EngTrace: A Symbolic Benchmark for Verifiable Process Supervision of Engineering Reasoning
by: Gull, Ayesha, et al.
Published: (2025)

Language over Content: Tracing Cultural Understanding in Multilingual Large Language Models
by: Cho, Seungho, et al.
Published: (2025)

Scaling DPPs for RAG: Density Meets Diversity
by: Sun, Xun, et al.
Published: (2026)

Explicit Word Density Estimation for Language Modelling
by: Andonov, Jovan, et al.
Published: (2024)

LLM-Powered Ensemble Learning for Paper Source Tracing: A GPU-Free Approach
by: Chen, Kunlong, et al.
Published: (2024)

Beyond the Last Answer: Your Reasoning Trace Uncovers More than You Think
by: Hammoud, Hasan Abed Al Kader, et al.
Published: (2025)

LLMs Are Not Intelligent Thinkers: Introducing Mathematical Topic Tree Benchmark for Comprehensive Evaluation of LLMs
by: Davoodi, Arash Gholami, et al.
Published: (2024)

A Strategic Coordination Framework of Small LLMs Matches Large LLMs in Data Synthesis
by: Gao, Xin, et al.
Published: (2025)

Forget What You Know about LLMs Evaluations -- LLMs are Like a Chameleon
by: Cohen-Inger, Nurit, et al.
Published: (2025)

Interpreting the Effects of Quantization on LLMs
by: Singh, Manpreet, et al.
Published: (2025)