Cover Image

Sparser, Faster, Lighter Transformer Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Cetin, Edoardo, Peluchetti, Stefano, Castillo, Emilio, Naruse, Akira, Murakami, Mana, Jones, Llion
Format:	Preprint
Published:	2026
Subjects:	Machine Learning Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Sparser is Faster and Less is More: Efficient Sparse Attention for Long-Range Transformers
by: Lou, Chao, et al.
Published: (2024)

Sudoku-Bench: Evaluating creative reasoning with Sudoku variants
by: Seely, Jeffrey, et al.
Published: (2025)

Faster and Lighter LLMs: A Survey on Current Challenges and Way Forward
by: Chavan, Arnav, et al.
Published: (2024)

Large Language Models to Diffusion Finetuning
by: Cetin, Edoardo, et al.
Published: (2025)

Transformer-Squared: Self-adaptive LLMs
by: Sun, Qi, et al.
Published: (2025)

KamonBench: A Grammar-Based Dataset for Evaluating Compositional Factor Recovery in Vision-Language Models
by: Sproat, Richard, et al.
Published: (2026)

An Evolved Universal Transformer Memory
by: Cetin, Edoardo, et al.
Published: (2024)

BM$^2$: Coupled Schrödinger Bridge Matching
by: Peluchetti, Stefano
Published: (2024)

ShinkaEvolve: Towards Open-Ended And Sample-Efficient Program Evolution
by: Lange, Robert Tjarko, et al.
Published: (2025)

AdaSplash-2: Faster Differentiable Sparse Attention
by: Gonçalves, Nuno, et al.
Published: (2026)

Reinforcement Learning Teachers of Test Time Scaling
by: Cetin, Edoardo, et al.
Published: (2025)

CDLM: Consistency Diffusion Language Models For Faster Sampling
by: Kim, Minseo, et al.
Published: (2025)

FlashDecoding++: Faster Large Language Model Inference on GPUs
by: Hong, Ke, et al.
Published: (2023)

EAGLE-2: Faster Inference of Language Models with Dynamic Draft Trees
by: Li, Yuhui, et al.
Published: (2024)

CORAL: Learning Consistent Representations across Multi-step Training with Lighter Speculative Drafter
by: Weng, Yepeng, et al.
Published: (2025)

Faster Transformer Decoding: N-gram Masked Self-Attention
by: Chelba, Ciprian, et al.
Published: (2020)

Not All Denoising Steps Are Equal: Model Scheduling for Faster Masked Diffusion Language Models
by: Sedykh, Ivan, et al.
Published: (2026)

Even Sparser Graph Transformers
by: Shirzad, Hamed, et al.
Published: (2024)

Sig-DEG for Distillation: Making Diffusion Models Faster and Lighter
by: Jiang, Lei, et al.
Published: (2025)

Faster MoE LLM Inference for Extremely Large Models
by: Yang, Haoqi, et al.
Published: (2025)

Asynchronous RLHF: Faster and More Efficient Off-Policy RL for Language Models
by: Noukhovitch, Michael, et al.
Published: (2024)

Anatomical Heterogeneity in Transformer Language Models
by: Wietrzykowski, Tomasz
Published: (2026)

Measuring Non-Adversarial Reproduction of Training Data in Large Language Models
by: Aerni, Michael, et al.
Published: (2024)

Function-Space MCMC for Bayesian Wide Neural Networks
by: Pezzetti, Lucia, et al.
Published: (2024)

Judge Decoding: Faster Speculative Sampling Requires Going Beyond Model Alignment
by: Bachmann, Gregor, et al.
Published: (2025)

Simultaneous Interpretation Corpus Construction by Large Language Models in Distant Language Pair
by: Sakai, Yusuke, et al.
Published: (2024)

Sparser, Better, Faster, Stronger: Sparsity Detection for Efficient Automatic Differentiation
by: Hill, Adrian, et al.
Published: (2025)

Selective Neuron Amplification in Transformer Language Models
by: Akhtar, Ryyan, et al.
Published: (2026)

Towards Logically Consistent Language Models via Probabilistic Reasoning
by: Calanzone, Diego, et al.
Published: (2024)

Sparse-Autoencoder-Guided Internal Representation Unlearning for Large Language Models
by: Yamashita, Tomoya, et al.
Published: (2025)

C2: Scalable Rubric-Augmented Reward Modeling from Binary Preferences
by: Kawabata, Akira, et al.
Published: (2026)

Run LoRA Run: Faster and Lighter LoRA Implementations
by: Cherniuk, Daria, et al.
Published: (2023)

The Sparse Frontier: Sparse Attention Trade-offs in Transformer LLMs
by: Nawrot, Piotr, et al.
Published: (2025)

Fast-weight Product Key Memory
by: Zhao, Tianyu, et al.
Published: (2026)

Jamba: A Hybrid Transformer-Mamba Language Model
by: Lieber, Opher, et al.
Published: (2024)

Transformer Layers as Painters
by: Sun, Qi, et al.
Published: (2024)

Faster LLM Inference via Sequential Monte Carlo
by: Emara, Yahya, et al.
Published: (2026)

Cascade Speculative Drafting for Even Faster LLM Inference
by: Chen, Ziyi, et al.
Published: (2023)

Learning to Focus: Focal Attention for Selective and Scalable Transformers
by: Ram, Dhananjay, et al.
Published: (2025)

Can Large Language Models Transform Computational Social Science?
by: Ziems, Caleb, et al.
Published: (2023)