Cover Image

You Need Better Attention Priors

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Litman, Elon, Guo, Gabe
Format:	Preprint
Published:	2026
Subjects:	Machine Learning Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

A Theory of Generalization in Deep Learning
by: Litman, Elon, et al.
Published: (2026)

Scaled-Dot-Product Attention as One-Sided Entropic Optimal Transport
by: Litman, Elon
Published: (2025)

Equilibrium Propagation Without Limits
by: Litman, Elon
Published: (2025)

The Origin of Edge of Stability
by: Litman, Elon
Published: (2026)

Attention Is All You Need But You Don't Need All Of It For Inference of Large Language Models
by: Tyukin, Georgy, et al.
Published: (2024)

Reviving Any-Subset Autoregressive Models with Principled Parallel Sampling and Speculative Decoding
by: Guo, Gabe, et al.
Published: (2025)

Attention Score is not All You Need for Token Importance Indicator in KV Cache Reduction: Value Also Matters
by: Guo, Zhiyu, et al.
Published: (2024)

Attention Smoothing Is All You Need For Unlearning
by: Zade, Saleh Zare, et al.
Published: (2026)

Tensor Product Attention Is All You Need
by: Zhang, Yifan, et al.
Published: (2025)

Attention Is Not All You Need: The Importance of Feedforward Networks in Transformer Models
by: Gerber, Isaac
Published: (2025)

More Compute Is What You Need
by: Guo, Zhen
Published: (2024)

ABC: Any-Subset Autoregression via Non-Markovian Diffusion Bridges in Continuous Time and Space
by: Guo, Gabe, et al.
Published: (2026)

Attention Is All You Need for KV Cache in Diffusion LLMs
by: Nguyen-Tri, Quan, et al.
Published: (2025)

Reinforcement Learning is all You Need
by: Lian, Yongsheng
Published: (2025)

Synthetic Data RL: Task Definition Is All You Need
by: Guo, Yiduo, et al.
Published: (2025)

TRA: Better Length Generalisation with Threshold Relative Attention
by: Opper, Mattia, et al.
Published: (2025)

Block Rotation is All You Need for MXFP4 Quantization
by: Shao, Yuantian, et al.
Published: (2025)

More Agents Is All You Need
by: Li, Junyou, et al.
Published: (2024)

Forget Attention: Importance-Aware Attention Is All You Need
by: Shin, Soohyeong, et al.
Published: (2026)

Confidence Is All You Need: Few-Shot RL Fine-Tuning of Language Models
by: Li, Pengyi, et al.
Published: (2025)

RecurFormer: Not All Transformer Heads Need Self-Attention
by: Yan, Ruiqing, et al.
Published: (2024)

You Only Need Minimal RLVR Training: Extrapolating LLMs via Rank-1 Trajectories
by: Wei, Zhepei, et al.
Published: (2026)

Attention Needs to Focus: A Unified Perspective on Attention Allocation
by: Fu, Zichuan, et al.
Published: (2026)

Does RoBERTa Perform Better than BERT in Continual Learning: An Attention Sink Perspective
by: Bai, Xueying, et al.
Published: (2024)

Memory Is All You Need: Testing How Model Memory Affects LLM Performance in Annotation Tasks
by: Timoneda, Joan C., et al.
Published: (2025)

Hierarchical Attention Generates Better Proofs
by: Chen, Jianlong, et al.
Published: (2025)

Are You Sure? Rank Them Again: Repeated Ranking For Better Preference Datasets
by: Devine, Peter
Published: (2024)

You Don't Need Prompt Engineering Anymore: The Prompting Inversion
by: Khan, Imran
Published: (2025)

One Model is All You Need: ByT5-Sanskrit, a Unified Model for Sanskrit NLP Tasks
by: Nehrdich, Sebastian, et al.
Published: (2024)

What Matters in Transformers? Not All Attention is Needed
by: He, Shwai, et al.
Published: (2024)

Think When You Need: Self-Adaptive Chain-of-Thought Learning
by: Yang, Junjie, et al.
Published: (2025)

Reformulation is All You Need: Addressing Malicious Text Features in DNNs
by: Jiang, Yi, et al.
Published: (2025)

An Extra RMSNorm is All You Need for Fine Tuning to 1.58 Bits
by: Steinmetz, Cody, et al.
Published: (2025)

Guidance is All You Need: Temperature-Guided Reasoning in Large Language Models
by: Gomaa, Eyad, et al.
Published: (2024)

All You Need is One: Capsule Prompt Tuning with a Single Vector
by: Liu, Yiyang, et al.
Published: (2025)

Why Any-Order Autoregressive Models Need Two-Stream Attention: A Structural-Semantic Tradeoff
by: Pynadath, Patrick, et al.
Published: (2026)

Evidence Is All You Need: Ordering Imaging Studies via Language Model Alignment with the ACR Appropriateness Criteria
by: Yao, Michael S., et al.
Published: (2024)

Intrinsic Fingerprint of LLMs: Continue Training is NOT All You Need to Steal A Model!
by: Yoon, Do-hyeon, et al.
Published: (2025)

Similarity is Not All You Need: Endowing Retrieval Augmented Generation with Multi Layered Thoughts
by: Gan, Chunjing, et al.
Published: (2024)

SecEncoder: Logs are All You Need in Security
by: Bulut, Muhammed Fatih, et al.
Published: (2024)