Cover Image

Optimizing Attention with Mirror Descent: Generalized Max-Margin Token Selection

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Julistiono, Addison Kristanto, Tarzanagh, Davoud Ataee, Azizan, Navid
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Artificial Intelligence Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Transformers as Support Vector Machines
by: Tarzanagh, Davoud Ataee, et al.
Published: (2023)

Gating is Weighting: Understanding Gated Linear Attention through In-context Learning
by: Li, Yingcong, et al.
Published: (2025)

MarginSel : Max-Margin Demonstration Selection for LLMs
by: Ambati, Rajeev Bhatt, et al.
Published: (2025)

Truncated Matrix Completion - An Empirical Study
by: Naik, Rishhabh, et al.
Published: (2025)

SEFD: Semantic-Enhanced Framework for Detecting LLM-Generated Text
by: He, Weiqing, et al.
Published: (2024)

Improving LLM General Preference Alignment via Optimistic Online Mirror Descent
by: Zhang, Yuheng, et al.
Published: (2025)

HardNet: Hard-Constrained Neural Networks with Universal Approximation Guarantees
by: Min, Youngjae, et al.
Published: (2024)

Selective Preference Optimization via Token-Level Reward Function Estimation
by: Yang, Kailai, et al.
Published: (2024)

Larger or Smaller Reward Margins to Select Preferences for Alignment?
by: Huang, Kexin, et al.
Published: (2025)

ConfPO: Exploiting Policy Model Confidence for Critical Token Selection in Preference Optimization
by: Yoon, Hee Suk, et al.
Published: (2025)

Training-Trajectory-Aware Token Selection
by: Shen, Zhanming, et al.
Published: (2026)

Fair Community Detection and Structure Learning in Heterogeneous Graphical Models
by: Tarzanagh, Davoud Ataee, et al.
Published: (2021)

OrthoRank: Token Selection via Sink Token Orthogonality for Efficient LLM inference
by: Shin, Seungjun, et al.
Published: (2025)

TokenSelect: Efficient Long-Context Inference and Length Extrapolation for LLMs via Dynamic Token-Level KV Cache Selection
by: Wu, Wei, et al.
Published: (2024)

A Penalty-Based Method for Communication-Efficient Decentralized Bilevel Programming
by: Nazari, Parvin, et al.
Published: (2022)

AlphaDPO: Adaptive Reward Margin for Direct Preference Optimization
by: Wu, Junkang, et al.
Published: (2024)

Optimize Weight Rounding via Signed Gradient Descent for the Quantization of LLMs
by: Cheng, Wenhua, et al.
Published: (2023)

Selective Attention Improves Transformer
by: Leviathan, Yaniv, et al.
Published: (2024)

Dynamics of Spontaneous Topic Changes in Next Token Prediction with Self-Attention
by: Jia, Mumin, et al.
Published: (2025)

Sessa: Selective State Space Attention
by: Horbatko, Liubomyr
Published: (2026)

On the Role of Transformer Feed-Forward Layers in Nonlinear In-Context Learning
by: Sun, Haoyuan, et al.
Published: (2025)

T-REG: Preference Optimization with Token-Level Reward Regularization
by: Zhou, Wenxuan, et al.
Published: (2024)

DPO Meets PPO: Reinforced Token Optimization for RLHF
by: Zhong, Han, et al.
Published: (2024)

Dynamic Thinking-Token Selection for Efficient Reasoning in Large Reasoning Models
by: Guo, Zhenyuan, et al.
Published: (2026)

Adaptive Layer Selection for Layer-Wise Token Pruning in LLM Inference
by: Taniguchi, Rei, et al.
Published: (2026)

Forget What Matters, Keep the Rest: Selective Unlearning of Informative Tokens
by: Koh, Seunghee, et al.
Published: (2026)

Counterfactual Token Generation in Large Language Models
by: Chatzi, Ivi, et al.
Published: (2024)

The MiniMax-M2 Series: Mini Activations Unleashing Max Real-World Intelligence
by: MiniMax, et al.
Published: (2026)

Mechanics of Next Token Prediction with Self-Attention
by: Li, Yingcong, et al.
Published: (2024)

Simplex-Optimized Hybrid Ensemble for Large Language Model Text Detection Under Generative Distribution Drif
by: Kristanto, Sepyan Purnama, et al.
Published: (2025)

Optimal Transport-Based Token Weighting scheme for Enhanced Preference Optimization
by: Li, Meng, et al.
Published: (2025)

COMPACT: Common-token Optimized Model Pruning Across Channels and Tokens
by: Kwek, Eugene, et al.
Published: (2025)

Specialised or Generic? Tokenization Choices for Radiology Language Models
by: Warr, Hermione, et al.
Published: (2025)

Explain in Your Own Words: Improving Reasoning via Token-Selective Dual Knowledge Distillation
by: Kim, Minsang, et al.
Published: (2026)

TokenButler: Token Importance is Predictable
by: Akhauri, Yash, et al.
Published: (2025)

TGDPO: Harnessing Token-Level Reward Guidance for Enhancing Direct Preference Optimization
by: Zhu, Mingkang, et al.
Published: (2025)

Taipan: Efficient and Expressive State Space Language Models with Selective Attention
by: Van Nguyen, Chien, et al.
Published: (2024)

Grounded Token Initialization for New Vocabulary in LMs for Generative Recommendation
by: Chen, Daiwei, et al.
Published: (2026)

Evaluation of Large Language Models via Coupled Token Generation
by: Benz, Nina Corvelo, et al.
Published: (2025)

Efficient Multi-Adapter LLM Serving via Cross-Model KV-Cache Reuse with Activated LoRA
by: Li, Allison, et al.
Published: (2025)