Cover Image

Long Context Alignment with Short Instructions and Synthesized Positions

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Wu, Wenhao, Wang, Yizhong, Fu, Yao, Yue, Xiang, Zhu, Dawei, Li, Sujian
Format:	Preprint
Published:	2024
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

LongEmbed: Extending Embedding Models for Long Context Retrieval
by: Zhu, Dawei, et al.
Published: (2024)

Retrieval Head Mechanistically Explains Long-Context Factuality
by: Wu, Wenhao, et al.
Published: (2024)

PoSE: Efficient Context Window Extension of LLMs via Positional Skip-wise Training
by: Zhu, Dawei, et al.
Published: (2023)

Chain-of-Thought Matters: Improving Long-Context Language Models with Reasoning Path Supervision
by: Zhu, Dawei, et al.
Published: (2025)

CoUDA: Coherence Evaluation via Unified Data Augmentation
by: Zhu, Dawei, et al.
Published: (2024)

WildLong: Synthesizing Realistic Long-Context Instruction Data at Scale
by: Li, Jiaxi, et al.
Published: (2025)

Generalizing From Short to Long: Effective Data Synthesis for Long-Context Instruction Tuning
by: Zhu, Wenhao, et al.
Published: (2025)

LongAttn: Selecting Long-context Training Data via Token-level Attention
by: Wu, Longyun, et al.
Published: (2025)

Long-Short Alignment for Effective Long-Context Modeling in LLMs
by: Du, Tianqi, et al.
Published: (2025)

DocLens : A Tool-Augmented Multi-Agent Framework for Long Visual Document Understanding
by: Zhu, Dawei, et al.
Published: (2025)

More Tokens, Lower Precision: Towards the Optimal Token-Precision Trade-off in KV Cache Compression
by: Zhang, Jiebin, et al.
Published: (2024)

AgentBank: Towards Generalized LLM Agents via Fine-Tuning on 50000+ Interaction Trajectories
by: Song, Yifan, et al.
Published: (2024)

EERPD: Leveraging Emotion and Emotion Regulation for Improving Personality Detection
by: Li, Zheng, et al.
Published: (2024)

Short Data, Long Context: Distilling Positional Knowledge in Transformers
by: Huber, Patrick, et al.
Published: (2026)

Self-Consistency Falls Short! The Adverse Effects of Positional Bias on Long-Context Problems
by: Byerly, Adam, et al.
Published: (2024)

Shapley Value-based Contrastive Alignment for Multimodal Information Extraction
by: Luo, Wen, et al.
Published: (2024)

Third-Party Language Model Performance Prediction from Instruction
by: Nadkarni, Rahul, et al.
Published: (2024)

A Comprehensive Survey on Long Context Language Modeling
by: Liu, Jiaheng, et al.
Published: (2025)

Safety Alignment in NLP Tasks: Weakly Aligned Summarization as an In-Context Attack
by: Fu, Yu, et al.
Published: (2023)

SEGMENT+: Long Text Processing with Short-Context Language Models
by: Shi, Wei, et al.
Published: (2024)

GoLongRL: Capability-Oriented Long Context Reinforcement Learning with Multitask Alignment
by: Lv, Minxuan, et al.
Published: (2026)

LIFBench: Evaluating the Instruction Following Performance and Stability of Large Language Models in Long-Context Scenarios
by: Wu, Xiaodong, et al.
Published: (2024)

Video2GUI: Synthesizing Large-Scale Interaction Trajectories for Generalized GUI Agent Pretraining
by: Xiong, Weimin, et al.
Published: (2026)

XL$^2$Bench: A Benchmark for Extremely Long Context Understanding with Long-range Dependencies
by: Ni, Xuanfan, et al.
Published: (2024)

PaperBanana: Automating Academic Illustration for AI Scientists
by: Zhu, Dawei, et al.
Published: (2026)

Router Upcycling: Leveraging Mixture-of-Routers in Mixture-of-Experts Upcycling
by: Ran, Junfeng, et al.
Published: (2025)

Unlocking Efficient Long-to-Short LLM Reasoning with Model Merging
by: Wu, Han, et al.
Published: (2025)

Where Does Long-Context Supervision Actually Go? Effective-Context Exposure Balancing
by: Zhu, Jinchang, et al.
Published: (2026)

HREF: Human Response-Guided Evaluation of Instruction Following in Language Models
by: Lyu, Xinxi, et al.
Published: (2024)

UIO-LLMs: Unbiased Incremental Optimization for Long-Context LLMs
by: Li, Wenhao, et al.
Published: (2024)

ZigzagAttention: Efficient Long-Context Inference with Exclusive Retrieval and Streaming Heads
by: Liu, Zhuorui, et al.
Published: (2025)

Wavelet-based Positional Representation for Long Context
by: Oka, Yui, et al.
Published: (2025)

A Controllable Examination for Long-Context Language Models
by: Yang, Yijun, et al.
Published: (2025)

Layer-Specific Scaling of Positional Encodings for Superior Long-Context Modeling
by: Wang, Zhenghua, et al.
Published: (2025)

Set the Clock: Temporal Alignment of Pretrained Language Models
by: Zhao, Bowen, et al.
Published: (2024)

An Efficient Recipe for Long Context Extension via Middle-Focused Positional Encoding
by: Wu, Tong, et al.
Published: (2024)

From Real to Synthetic: Synthesizing Millions of Diversified and Complicated User Instructions with Attributed Grounding
by: Zhu, Chiwei, et al.
Published: (2025)

MAPO: Advancing Multilingual Reasoning through Multilingual Alignment-as-Preference Optimization
by: She, Shuaijie, et al.
Published: (2024)

LongIns: A Challenging Long-context Instruction-based Exam for LLMs
by: Gavin, Shawn, et al.
Published: (2024)

InsBank: Evolving Instruction Subset for Ongoing Alignment
by: Shi, Jiayi, et al.
Published: (2025)