LongAttn: Selecting Long-context Training Data via Token-level Attention

Documenti analoghi

• Router Upcycling: Leveraging Mixture-of-Routers in Mixture-of-Experts Upcycling
  di: Ran, Junfeng, et al.
  Pubblicazione: (2025)
• Chain-of-Thought Matters: Improving Long-Context Language Models with Reasoning Path Supervision
  di: Zhu, Dawei, et al.
  Pubblicazione: (2025)
• LADM: Long-context Training Data Selection with Attention-based Dependency Measurement for LLMs
  di: Chen, Jianghao, et al.
  Pubblicazione: (2025)
• LongEmbed: Extending Embedding Models for Long Context Retrieval
  di: Zhu, Dawei, et al.
  Pubblicazione: (2024)
• Long Context Alignment with Short Instructions and Synthesized Positions
  di: Wu, Wenhao, et al.
  Pubblicazione: (2024)