Copertina

Retrieval Backward Attention without Additional Training: Enhance Embeddings of Large Language Models via Repetition

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Duan, Yifei, Shang, Raphael, Liang, Deng, Cai, Yongqiang
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Computation and Language Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Repetition Improves Language Model Embeddings
di: Springer, Jacob Mitchell, et al.
Pubblicazione: (2024)

Unveiling and Harnessing Hidden Attention Sinks: Enhancing Large Language Models without Training through Attention Calibration
di: Yu, Zhongzhi, et al.
Pubblicazione: (2024)

DropBP: Accelerating Fine-Tuning of Large Language Models by Dropping Backward Propagation
di: Woo, Sunghyeon, et al.
Pubblicazione: (2024)

LongEmbed: Extending Embedding Models for Long Context Retrieval
di: Zhu, Dawei, et al.
Pubblicazione: (2024)

On the Thinking-Language Modeling Gap in Large Language Models
di: Liu, Chenxi, et al.
Pubblicazione: (2025)

Forward-Backward Reasoning in Large Language Models for Mathematical Verification
di: Jiang, Weisen, et al.
Pubblicazione: (2023)

Z-Pruner: Post-Training Pruning of Large Language Models for Efficiency without Retraining
di: Bhuiyan, Samiul Basir, et al.
Pubblicazione: (2025)

Distilling Large Language Models for Text-Attributed Graph Learning
di: Pan, Bo, et al.
Pubblicazione: (2024)

Mitigating the Language Mismatch and Repetition Issues in LLM-based Machine Translation via Model Editing
di: Wang, Weichuan, et al.
Pubblicazione: (2024)

Attention with Trained Embeddings Provably Selects Important Tokens
di: Wu, Diyuan, et al.
Pubblicazione: (2025)

Orion-14B: Open-source Multilingual Large Language Models
di: Chen, Du, et al.
Pubblicazione: (2024)

Sliding Window Attention Training for Efficient Large Language Models
di: Fu, Zichuan, et al.
Pubblicazione: (2025)

On Fine-Grained I/O Complexity of Attention Backward Passes
di: Li, Xiaoyu, et al.
Pubblicazione: (2024)

Language Model Training Paradigms for Clinical Feature Embeddings
di: Hu, Yurong, et al.
Pubblicazione: (2023)

Reusing Embeddings: Reproducible Reward Model Research in Large Language Model Alignment without GPUs
di: Sun, Hao, et al.
Pubblicazione: (2025)

Pre-trained Large Language Models Use Fourier Features to Compute Addition
di: Zhou, Tianyi, et al.
Pubblicazione: (2024)

Antislop: A Comprehensive Framework for Identifying and Eliminating Repetitive Patterns in Language Models
di: Paech, Samuel, et al.
Pubblicazione: (2025)

SCALPEL: Selective Capability Ablation via Low-rank Parameter Editing for Large Language Model Interpretability Analysis
di: Fu, Zihao, et al.
Pubblicazione: (2026)

Next Semantic Scale Prediction via Hierarchical Diffusion Language Models
di: Zhou, Cai, et al.
Pubblicazione: (2025)

Cost-Efficient Large Language Model Serving for Multi-turn Conversations with CachedAttention
di: Gao, Bin, et al.
Pubblicazione: (2024)

Scalable Efficient Training of Large Language Models with Low-dimensional Projected Attention
di: Lv, Xingtai, et al.
Pubblicazione: (2024)

Large Language Model Unlearning via Embedding-Corrupted Prompts
di: Liu, Chris Yuhao, et al.
Pubblicazione: (2024)

Synthetic Text Generation for Training Large Language Models via Gradient Matching
di: Nguyen, Dang, et al.
Pubblicazione: (2025)

LoRAP: Transformer Sub-Layers Deserve Differentiated Structured Compression for Large Language Models
di: Li, Guangyan, et al.
Pubblicazione: (2024)

The Rotary Position Embedding May Cause Dimension Inefficiency in Attention Heads for Long-Distance Retrieval
di: Chiang, Ting-Rui, et al.
Pubblicazione: (2025)

ArCHer: Training Language Model Agents via Hierarchical Multi-Turn RL
di: Zhou, Yifei, et al.
Pubblicazione: (2024)

RetrievalAttention: Accelerating Long-Context LLM Inference via Vector Retrieval
di: Liu, Di, et al.
Pubblicazione: (2024)

RedPajama: an Open Dataset for Training Large Language Models
di: Weber, Maurice, et al.
Pubblicazione: (2024)

Olica: Efficient Structured Pruning of Large Language Models without Retraining
di: He, Jiujun, et al.
Pubblicazione: (2025)

APTQ: Attention-aware Post-Training Mixed-Precision Quantization for Large Language Models
di: Guan, Ziyi, et al.
Pubblicazione: (2024)

Mechanistic Exploration of Backdoored Large Language Model Attention Patterns
di: Baker, Mohammed Abu, et al.
Pubblicazione: (2025)

TraceDet: Hallucination Detection from the Decoding Trace of Diffusion Large Language Models
di: Chang, Shenxu, et al.
Pubblicazione: (2025)

Interpretable Steering of Large Language Models with Feature Guided Activation Additions
di: Soo, Samuel, et al.
Pubblicazione: (2025)

Improving Uncertainty Quantification in Large Language Models via Semantic Embeddings
di: Grewal, Yashvir S., et al.
Pubblicazione: (2024)

Backward Lens: Projecting Language Model Gradients into the Vocabulary Space
di: Katz, Shahar, et al.
Pubblicazione: (2024)

Parallax: Parameterized Local Linear Attention for Language Modeling
di: Zuo, Yifei, et al.
Pubblicazione: (2026)

DEPT: Decoupled Embeddings for Pre-training Language Models
di: Iacob, Alex, et al.
Pubblicazione: (2024)

Experimental Design for Active Transductive Inference in Large Language Models
di: Mukherjee, Subhojyoti, et al.
Pubblicazione: (2024)

Self-Training Large Language Models with Confident Reasoning
di: Jang, Hyosoon, et al.
Pubblicazione: (2025)

Linear Dynamics in the RLVR Training of Large Language Models
di: Wang, Tianle, et al.
Pubblicazione: (2026)