Image de couverture de livre

Interleaved Head Attention

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Duvvuri, Sai Surya, Ekbote, Chanakya, Bansal, Rachit, Tiwari, Rishabh, Khatri, Devvrit, Brandfonbrener, David, Liang, Paul, Dhillon, Inderjit, Zaheer, Manzil
Format:	Preprint
Publié:	2026
Sujets:	Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

The Art of Scaling Reinforcement Learning Compute for LLMs
par: Khatri, Devvrit, et autres
Publié: (2025)

LASER: Attention with Exponential Transformation
par: Duvvuri, Sai Surya, et autres
Publié: (2024)

Let's (not) just put things in Context: Test-Time Training for Long-Context LLMs
par: Bansal, Rachit, et autres
Publié: (2025)

LUCID: Attention with Preconditioned Representations
par: Duvvuri, Sai Surya, et autres
Publié: (2026)

Dual-Encoders for Extreme Multi-Label Classification
par: Gupta, Nilesh, et autres
Publié: (2023)

Fast and Simplex: 2-Simplicial Attention in Triton
par: Roy, Aurko, et autres
Publié: (2025)

Learning, Fast and Slow: Towards LLMs That Adapt Continually
par: Tiwari, Rishabh, et autres
Publié: (2026)

Understanding the Emergence of Multimodal Representation Alignment
par: Tjandrasuwita, Megan, et autres
Publié: (2025)

QoQ-Med: Building Multimodal Clinical Foundation Models with Domain-Aware GRPO Training
par: Dai, Wei, et autres
Publié: (2025)

LoRA Done RITE: Robust Invariant Transformation Equilibration for LoRA Optimization
par: Yen, Jui-Nan, et autres
Publié: (2024)

What One Cannot, Two Can: Two-Layer Transformers Provably Represent Induction Heads on Any-Order Markov Chains
par: Ekbote, Chanakya, et autres
Publié: (2025)

SCATR: Simple Calibrated Test-Time Ranking
par: Shyamal, Divya, et autres
Publié: (2026)

Efficient Distributed Optimization under Heavy-Tailed Noise
par: Lee, Su Hyeong, et autres
Publié: (2025)

Differentially Private Model Merging
par: Yin, Qichuan, et autres
Publié: (2026)

Compressing Many-Shots in In-Context Learning
par: Khatri, Devvrit, et autres
Publié: (2025)

ODRPO: Ordinal Decompositions of Discrete Rewards for Robust Policy Optimization
par: Patel, Nirmal, et autres
Publié: (2026)

Geometric Median (GM) Matching for Robust Data Pruning
par: Acharya, Anish, et autres
Publié: (2024)

A Statistical Framework for Data-dependent Retrieval-Augmented Models
par: Basu, Soumya, et autres
Publié: (2024)

Federation over Text: Insight Sharing for Multi-Agent Reasoning
par: Yao, Dixi, et autres
Publié: (2026)

Geometric Median Matching for Robust k-Subset Selection from Noisy Data
par: Acharya, Anish, et autres
Publié: (2025)

Deep Reinforcement Learning for Sequential Combinatorial Auctions
par: Ravindranath, Sai Srivatsa, et autres
Publié: (2024)

Local to Global: Learning Dynamics and Effect of Initialization for Transformers
par: Makkuva, Ashok Vardhan, et autres
Publié: (2024)

MURPHY: Feedback-Aware GRPO with Retrospective Credit Assignment for Multi-Turn Code Generation
par: Ekbote, Chanakya, et autres
Publié: (2025)

Asynchronous Heavy-Tailed Optimization
par: Sun, Junfei, et autres
Publié: (2026)

Towards Quantifying the Preconditioning Effect of Adam
par: Das, Rudrajit, et autres
Publié: (2024)

MatFormer: Nested Transformer for Elastic Inference
par: Devvrit, et autres
Publié: (2023)

A Fresh Take on Stale Embeddings: Improving Dense Retriever Training with Corrector Networks
par: Monath, Nicholas, et autres
Publié: (2024)

Measures of Information Reflect Memorization Patterns
par: Bansal, Rachit, et autres
Publié: (2022)

Efficient Adaptive Federated Optimization
par: Lee, Su Hyeong, et autres
Publié: (2024)

Adaptive Retrieval and Scalable Indexing for k-NN Search with Cross-Encoders
par: Yadav, Nishant, et autres
Publié: (2024)

Cite Pretrain: Retrieval-Free Knowledge Attribution for Large Language Models
par: Huang, Yukun, et autres
Publié: (2025)

Understanding Outer Optimizers in Local SGD: Learning Rates, Momentum, and Acceleration
par: Khaled, Ahmed, et autres
Publié: (2025)

ExeDec: Execution Decomposition for Compositional Generalization in Neural Program Synthesis
par: Shi, Kensen, et autres
Publié: (2023)

OmniSapiens: A Foundation Model for Social Behavior Processing via Heterogeneity-Aware Relative Policy Optimization
par: Ong, Keane, et autres
Publié: (2026)

GQ-VAE: A gated quantized VAE for learning variable length tokens
par: Datta, Theo, et autres
Publié: (2025)

LLM-guided Hierarchical Search for End-to-end Reasoning Intensive Retrieval
par: Gupta, Nilesh, et autres
Publié: (2025)

EHI: End-to-end Learning of Hierarchical Index for Efficient Dense Retrieval
par: Kumar, Ramnath, et autres
Publié: (2023)

Scalable In-context Ranking with Generative Models
par: Gupta, Nilesh, et autres
Publié: (2025)

Adam or Gauss-Newton? A Comparative Study In Terms of Basis Alignment and SGD Noise
par: Liu, Bingbin, et autres
Publié: (2025)

Universal Length Generalization with Turing Programs
par: Hou, Kaiying, et autres
Publié: (2024)