Cover Image

SEAL: Scaling to Emphasize Attention for Long-Context Retrieval

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Lee, Changhun, Seok, Minsang, Jin, Jun-gyu, Cho, Younghyun, Park, Eunhyeok
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Artificial Intelligence Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

QEFT: Quantization for Efficient Fine-Tuning of LLMs
by: Lee, Changhun, et al.
Published: (2024)

AMQ: Enabling AutoML for Mixed-precision Weight-Only Quantization of Large Language Models
by: Lee, Sangjun, et al.
Published: (2025)

PTQ4VM: Post-Training Quantization for Visual Mamba
by: Cho, Younghyun, et al.
Published: (2024)

PruneCD: Contrasting Pruned Self Model to Improve Decoding Factuality
by: Yu, Byeongho, et al.
Published: (2025)

Explain in Your Own Words: Improving Reasoning via Token-Selective Dual Knowledge Distillation
by: Kim, Minsang, et al.
Published: (2026)

SEAL: Systematic Error Analysis for Value ALignment
by: Revel, Manon, et al.
Published: (2024)

Every Attention Matters: An Efficient Hybrid Architecture for Long-Context Reasoning
by: Ling Team, et al.
Published: (2025)

Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilization
by: Hsieh, Cheng-Yu, et al.
Published: (2024)

Cost-Optimal Grouped-Query Attention for Long-Context Modeling
by: Chen, Yingfa, et al.
Published: (2025)

MoBA: Mixture of Block Attention for Long-Context LLMs
by: Lu, Enzhe, et al.
Published: (2025)

HiCI: Hierarchical Construction-Integration for Long-Context Attention
by: Zeng, Xiangyu, et al.
Published: (2026)

VPO: Leveraging the Number of Votes in Preference Optimization
by: Cho, Jae Hyeon, et al.
Published: (2024)

Reversed in Time: A Novel Temporal-Emphasized Benchmark for Cross-Modal Video-Text Retrieval
by: Du, Yang, et al.
Published: (2024)

100-LongBench: Are de facto Long-Context Benchmarks Literally Evaluating Long-Context Ability?
by: Yang, Wang, et al.
Published: (2025)

SampleAttention: Near-Lossless Acceleration of Long Context LLM Inference with Adaptive Structured Sparse Attention
by: Zhu, Qianchao, et al.
Published: (2024)

Tactic: Adaptive Sparse Attention with Clustering and Distribution Fitting for Long-Context LLMs
by: Zhu, Kan, et al.
Published: (2025)

Revisiting In-Context Learning with Long Context Language Models
by: Baek, Jinheon, et al.
Published: (2024)

SEAL: Safety-enhanced Aligned LLM Fine-tuning via Bilevel Data Selection
by: Shen, Han, et al.
Published: (2024)

Merge-Friendly Post-Training Quantization for Multi-Target Domain Adaptation
by: Shin, Juncheol, et al.
Published: (2025)

Scaling Attention to Very Long Sequences in Linear Time with Wavelet-Enhanced Random Spectral Attention (WERSA)
by: Dentamaro, Vincenzo
Published: (2025)

Retrieval Augmented Generation or Long-Context LLMs? A Comprehensive Study and Hybrid Approach
by: Li, Zhuowan, et al.
Published: (2024)

MiniCPM-SALA: Hybridizing Sparse and Linear Attention for Efficient Long-Context Modeling
by: MiniCPM Team, et al.
Published: (2026)

Long-Context LLMs Meet RAG: Overcoming Challenges for Long Inputs in RAG
by: Jin, Bowen, et al.
Published: (2024)

Longer Context, Deeper Thinking: Uncovering the Role of Long-Context Ability in Reasoning
by: Yang, Wang, et al.
Published: (2025)

LazyLLM: Dynamic Token Pruning for Efficient Long Context LLM Inference
by: Fu, Qichen, et al.
Published: (2024)

Recursive Language Models Meet Uncertainty: The Surprising Effectiveness of Self-Reflective Program Search for Long Context
by: Alizadeh, Keivan, et al.
Published: (2026)

Hybrid Linear Attention Done Right: Efficient Distillation and Effective Architectures for Extremely Long Contexts
by: Chen, Yingfa, et al.
Published: (2026)

Retrieval meets Long Context Large Language Models
by: Xu, Peng, et al.
Published: (2023)

Predicting Task Performance with Context-aware Scaling Laws
by: Montgomery, Kyle, et al.
Published: (2025)

CoCA: Fusing Position Embedding with Collinear Constrained Attention in Transformers for Long Context Window Extending
by: Zhu, Shiyi, et al.
Published: (2023)

Accelerating Prefilling for Long-Context LLMs via Sparse Pattern Sharing
by: Peng, Dan, et al.
Published: (2025)

LLMs Know What to Drop: Self-Attention Guided KV Cache Eviction for Efficient Long-Context Inference
by: Wang, Guangtao, et al.
Published: (2025)

Language over Content: Tracing Cultural Understanding in Multilingual Large Language Models
by: Cho, Seungho, et al.
Published: (2025)

LongSafety: Enhance Safety for Long-Context LLMs
by: Huang, Mianqiu, et al.
Published: (2024)

Decomposing Attention To Find Context-Sensitive Neurons
by: Gibson, Alex
Published: (2025)

Which Attention Heads Matter for In-Context Learning?
by: Yin, Kayo, et al.
Published: (2025)

Scaling Reasoning without Attention
by: Zhao, Xueliang, et al.
Published: (2025)

Syntriever: How to Train Your Retriever with Synthetic Data from LLMs
by: Kim, Minsang, et al.
Published: (2025)

Integrating Locality-Aware Attention with Transformers for General Geometry PDEs
by: Koh, Minsu, et al.
Published: (2025)

The Impossibility Triangle of Long-Context Modeling
by: Zhou, Yan
Published: (2026)