Image de couverture de livre

Multi-Token Attention

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Golovneva, Olga, Wang, Tianlu, Weston, Jason, Sukhbaatar, Sainbayar
Format:	Preprint
Publié:	2025
Sujets:	Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Contextual Position Encoding: Learning to Count What's Important
par: Golovneva, Olga, et autres
Publié: (2024)

Reverse Training to Nurse the Reversal Curse
par: Golovneva, Olga, et autres
Publié: (2024)

CoT-Self-Instruct: Building high-quality synthetic prompts for reasoning and non-reasoning tasks
par: Yu, Ping, et autres
Publié: (2025)

StepWiser: Stepwise Generative Judges for Wiser Reasoning
par: Xiong, Wei, et autres
Publié: (2025)

R.I.P.: Better Models by Survival of the Fittest Prompts
par: Yu, Ping, et autres
Publié: (2025)

Meta-Rewarding Language Models: Self-Improving Alignment with LLM-as-a-Meta-Judge
par: Wu, Tianhao, et autres
Publié: (2024)

Some things are more CRINGE than others: Iterative Preference Optimization with the Pairwise Cringe Loss
par: Xu, Jing, et autres
Publié: (2023)

Self-Challenging Language Model Agents
par: Zhou, Yifei, et autres
Publié: (2025)

Thinking LLMs: General Instruction Following with Thought Generation
par: Wu, Tianhao, et autres
Publié: (2024)

Branch-Train-MiX: Mixing Expert LLMs into a Mixture-of-Experts LLM
par: Sukhbaatar, Sainbayar, et autres
Publié: (2024)

Diverse Preference Optimization
par: Lanchantin, Jack, et autres
Publié: (2025)

Iterative Reasoning Preference Optimization
par: Pang, Richard Yuanzhe, et autres
Publié: (2024)

Following Length Constraints in Instructions
par: Yuan, Weizhe, et autres
Publié: (2024)

Adaptive Decoding via Latent Preference Optimization
par: Dhuliawala, Shehzaad, et autres
Publié: (2024)

Training Large Language Models to Reason in a Continuous Latent Space
par: Hao, Shibo, et autres
Publié: (2024)

Self-Improving Pretraining: using post-trained models to pretrain better models
par: Tan, Ellen Xiaoqing, et autres
Publié: (2026)

Self-Rewarding Language Models
par: Yuan, Weizhe, et autres
Publié: (2024)

Bridging Offline and Online Reinforcement Learning for LLMs
par: Lanchantin, Jack, et autres
Publié: (2025)

Self-Taught Evaluators
par: Wang, Tianlu, et autres
Publié: (2024)

SPICE: Self-Play In Corpus Environments Improves Reasoning
par: Liu, Bo, et autres
Publié: (2025)

Learning to Plan & Reason for Evaluation with Thinking-LLM-as-a-Judge
par: Saha, Swarnadeep, et autres
Publié: (2025)

Self-Consistency Preference Optimization
par: Prasad, Archiki, et autres
Publié: (2024)

The Era of Real-World Human Interaction: RL from User Conversations
par: Jin, Chuanyang, et autres
Publié: (2025)

Efficient Tool Use with Chain-of-Abstraction Reasoning
par: Gao, Silin, et autres
Publié: (2024)

J1: Incentivizing Thinking in LLM-as-a-Judge via Reinforcement Learning
par: Whitehouse, Chenxi, et autres
Publié: (2025)

Jointly Reinforcing Diversity and Quality in Language Model Generations
par: Li, Tianjian, et autres
Publié: (2025)

Hybrid Reinforcement: When Reward Is Sparse, It's Better to Be Dense
par: Tao, Leitian, et autres
Publié: (2025)

SWEET-RL: Training Multi-Turn LLM Agents on Collaborative Reasoning Tasks
par: Zhou, Yifei, et autres
Publié: (2025)

System-Level Natural Language Feedback
par: Yuan, Weizhe, et autres
Publié: (2023)

Reasoning over mathematical objects: on-policy reward modeling and test time aggregation
par: Aggarwal, Pranjal, et autres
Publié: (2026)

Recycling the Web: A Method to Enhance Pre-training Data Quality and Quantity for Language Models
par: Nguyen, Thao, et autres
Publié: (2025)

Byte Latent Transformer: Patches Scale Better Than Tokens
par: Pagnoni, Artidoro, et autres
Publié: (2024)

Distilling System 2 into System 1
par: Yu, Ping, et autres
Publié: (2024)

Token-weighted Direct Preference Optimization with Attention
par: Huang, Chengyu, et autres
Publié: (2026)

Token Distillation: Attention-aware Input Embeddings For New Tokens
par: Dobler, Konstantin, et autres
Publié: (2025)

Open-Domain Text Evaluation via Contrastive Distribution Methods
par: Lu, Sidi, et autres
Publié: (2023)

Efficient Sparse Attention needs Adaptive Token Release
par: Zhang, Chaoran, et autres
Publié: (2024)

Leveraging Implicit Feedback from Deployment Data in Dialogue
par: Pang, Richard Yuanzhe, et autres
Publié: (2023)

Positional Encoding via Token-Aware Phase Attention
par: Wang, Yu, et autres
Publié: (2025)

The Majority is not always right: RL training for solution aggregation
par: Zhao, Wenting, et autres
Publié: (2025)