Copertina

LADM: Long-context Training Data Selection with Attention-based Dependency Measurement for LLMs

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Chen, Jianghao, Wu, Junhong, Xu, Yangyifan, Zhang, Jiajun
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Computation and Language
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Hit the Sweet Spot! Span-Level Ensemble for Large Language Models
di: Xu, Yangyifan, et al.
Pubblicazione: (2024)

Bridging the Gap between Different Vocabularies for LLM Ensemble
di: Xu, Yangyifan, et al.
Pubblicazione: (2024)

Boosting LLM Translation Skills without General Ability Loss via Rationale Distillation
di: Wu, Junhong, et al.
Pubblicazione: (2024)

LongAttn: Selecting Long-context Training Data via Token-level Attention
di: Wu, Longyun, et al.
Pubblicazione: (2025)

ACE-RL: Adaptive Constraint-Enhanced Reward for Long-form Generation Reinforcement Learning
di: Chen, Jianghao, et al.
Pubblicazione: (2025)

Can LLMs Think Like Consumers? Benchmarking Crowd-Level Reaction Reconstruction with ConsumerSimBench
di: Wang, Tianyu, et al.
Pubblicazione: (2026)

Long-context LLMs Struggle with Long In-context Learning
di: Li, Tianle, et al.
Pubblicazione: (2024)

LR^2Bench: Evaluating Long-chain Reflective Reasoning Capabilities of Large Language Models via Constraint Satisfaction Problems
di: Chen, Jianghao, et al.
Pubblicazione: (2025)

LongIns: A Challenging Long-context Instruction-based Exam for LLMs
di: Gavin, Shawn, et al.
Pubblicazione: (2024)

LongRAG: Enhancing Retrieval-Augmented Generation with Long-context LLMs
di: Jiang, Ziyan, et al.
Pubblicazione: (2024)

Implicit Cross-Lingual Rewarding for Efficient Multilingual Preference Alignment
di: Yang, Wen, et al.
Pubblicazione: (2025)

Language Imbalance Driven Rewarding for Multilingual Self-improving
di: Yang, Wen, et al.
Pubblicazione: (2024)

Training-free Context-adaptive Attention for Efficient Long Context Modeling
di: You, Zeng, et al.
Pubblicazione: (2025)

Understanding the RoPE Extensions of Long-Context LLMs: An Attention Perspective
di: Zhong, Meizhi, et al.
Pubblicazione: (2024)

In-context KV-Cache Eviction for LLMs via Attention-Gate
di: Zeng, Zihao, et al.
Pubblicazione: (2024)

CLUES: Collaborative High-Quality Data Selection for LLMs via Training Dynamics
di: Zhao, Wanru, et al.
Pubblicazione: (2025)

DAWN: Dependency-Aware Fast Inference for Diffusion LLMs
di: Luo, Lizhuo, et al.
Pubblicazione: (2026)

LongCite: Enabling LLMs to Generate Fine-grained Citations in Long-context QA
di: Zhang, Jiajie, et al.
Pubblicazione: (2024)

Ref-Long: Benchmarking the Long-context Referencing Capability of Long-context Language Models
di: Wu, Junjie, et al.
Pubblicazione: (2025)

AttentionInfluence: Adopting Attention Head Influence for Weak-to-Strong Pretraining Data Selection
di: Hua, Kai, et al.
Pubblicazione: (2025)

LiteLong: Resource-Efficient Long-Context Data Synthesis for LLMs
di: Jia, Junlong, et al.
Pubblicazione: (2025)

Parallel Scaling Law: Unveiling Reasoning Generalization through A Cross-Linguistic Perspective
di: Yang, Wen, et al.
Pubblicazione: (2025)

Look Again, Think Slowly: Enhancing Visual Reflection in Vision-Language Models
di: Jian, Pu, et al.
Pubblicazione: (2025)

Emergent Hierarchical Reasoning in LLMs through Reinforcement Learning
di: Wang, Haozhe, et al.
Pubblicazione: (2025)

Ungrammatical-syntax-based In-context Example Selection for Grammatical Error Correction
di: Tang, Chenming, et al.
Pubblicazione: (2024)

A Decomposition Perspective to Long-context Reasoning for LLMs
di: Xiao, Yanling, et al.
Pubblicazione: (2026)

SimulPL: Aligning Human Preferences in Simultaneous Machine Translation
di: Yu, Donglei, et al.
Pubblicazione: (2025)

Leveraging Self-Attention for Input-Dependent Soft Prompting in LLMs
di: Muppidi, Ananth, et al.
Pubblicazione: (2025)

Long Context Pre-Training with Lighthouse Attention
di: Peng, Bowen, et al.
Pubblicazione: (2026)

Attention with Trained Embeddings Provably Selects Important Tokens
di: Wu, Diyuan, et al.
Pubblicazione: (2025)

ZigZagkv: Dynamic KV Cache Compression for Long-context Modeling based on Layer Uncertainty
di: Zhong, Meizhi, et al.
Pubblicazione: (2024)

SCOI: Syntax-augmented Coverage-based In-context Example Selection for Machine Translation
di: Tang, Chenming, et al.
Pubblicazione: (2024)

MoBA: Mixture of Block Attention for Long-Context LLMs
di: Lu, Enzhe, et al.
Pubblicazione: (2025)

Attention Reveals More Than Tokens: Training-Free Long-Context Reasoning with Attention-guided Retrieval
di: Zhang, Yuwei, et al.
Pubblicazione: (2025)

Long Context is Not Long at All: A Prospector of Long-Dependency Data for Large Language Models
di: Chen, Longze, et al.
Pubblicazione: (2024)

Lag-Relative Sparse Attention In Long Context Training
di: Liang, Manlai, et al.
Pubblicazione: (2025)

Speculating LLMs' Chinese Training Data Pollution from Their Tokens
di: Zhang, Qingjie, et al.
Pubblicazione: (2025)

A Survey on Data Selection for LLM Instruction Tuning
di: Zhang, Bolin, et al.
Pubblicazione: (2024)

GRKV: Global Regression for Training-Free KV Cache Compression in Long-Context LLMs
di: Peng, Junjie, et al.
Pubblicazione: (2026)

BLSP-Emo: Towards Empathetic Large Speech-Language Models
di: Wang, Chen, et al.
Pubblicazione: (2024)