Cover Image

LLM-based Embeddings: Attention Values Encode Sentence Semantics Better Than Hidden States

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zhang, Yeqin, Wang, Yunfei, Chen, Jiaxuan, Qin, Ke, Zhao, Yizheng, Nguyen, Cam-Tu
Format:	Preprint
Published:	2026
Subjects:	Computation and Language Information Retrieval
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

SemCSE: Semantic Contrastive Sentence Embeddings Using LLM-Generated Summaries For Scientific Abstracts
by: Brinner, Marc, et al.
Published: (2025)

Can Cross Encoders Produce Useful Sentence Embeddings?
by: Ananthakrishnan, Haritha, et al.
Published: (2025)

One Model Is Enough: Native Retrieval Embeddings from LLM Agent Hidden States
by: Jiang, Bo
Published: (2026)

Pooling And Attention: What Are Effective Designs For LLM-Based Embedding Models?
by: Tang, Yixuan, et al.
Published: (2024)

LLMs Meet Isolation Kernel: Lightweight, Learning-free Binary Embeddings for Fast Retrieval
by: Zhang, Zhibo, et al.
Published: (2026)

EncodeRec: An Embedding Backbone for Recommendation Systems
by: Hadad, Guy, et al.
Published: (2026)

Predicting Oscar-Nominated Screenplays with Sentence Embeddings
by: Gross, Francis
Published: (2025)

Data-CUBE: Data Curriculum for Instruction-based Sentence Representation Learning
by: Min, Yingqian, et al.
Published: (2024)

SAFE: Improving LLM Systems using Sentence-Level In-generation Attribution
by: Batista, João Eduardo, et al.
Published: (2025)

Topic mining based on fine-tuning Sentence-BERT and LDA
by: Li, Jianheng, et al.
Published: (2025)

MTEB-French: Resources for French Sentence Embedding Evaluation and Analysis
by: Ciancone, Mathieu, et al.
Published: (2024)

SetCSE: Set Operations using Contrastive Learning of Sentence Embeddings
by: Liu, Kang
Published: (2024)

Purely Semantic Indexing for LLM-based Generative Recommendation and Retrieval
by: Zhang, Ruohan, et al.
Published: (2025)

Is Semantic Chunking Worth the Computational Cost?
by: Qu, Renyi, et al.
Published: (2024)

Enhancing Document Retrieval in COVID-19 Research: Leveraging Large Language Models for Hidden Relation Extraction
by: Trieu, Hoang-An, et al.
Published: (2025)

EmbeddingRWKV: State-Centric Retrieval with Reusable States
by: Hou, Haowen, et al.
Published: (2026)

A Survey on Retrieval-Augmented Text Generation for Large Language Models
by: Huang, Yizheng, et al.
Published: (2024)

Pooling and Semantic Shift: The Fundamental Challenges in Long Text Embedding and Retrieval
by: Gao, Hang, et al.
Published: (2026)

Modeling Sequential Sentence Relation to Improve Cross-lingual Dense Retrieval
by: Zhang, Shunyu, et al.
Published: (2023)

Semantic Novelty Trajectories in 80,000 Books: A Cross-Corpus Embedding Analysis
by: Zimmerman, Fred
Published: (2026)

How Good are LLM-based Rerankers? An Empirical Analysis of State-of-the-Art Reranking Models
by: Abdallah, Abdelrahman, et al.
Published: (2025)

Better Generalizing to Unseen Concepts: An Evaluation Framework and An LLM-Based Auto-Labeled Pipeline for Biomedical Concept Recognition
by: Liu, Shanshan, et al.
Published: (2026)

Semantic Certainty Assessment in Vector Retrieval Systems: A Novel Framework for Embedding Quality Evaluation
by: Du, Y.
Published: (2025)

ListConRanker: A Contrastive Text Reranker with Listwise Encoding
by: Liu, Junlong, et al.
Published: (2025)

DS@GT eRisk 2024: Sentence Transformers for Social Media Risk Assessment
by: Guecha, David, et al.
Published: (2024)

UniRec: Unified Multimodal Encoding for LLM-Based Recommendations
by: Lei, Zijie, et al.
Published: (2026)

Real World Conversational Entity Linking Requires More Than Zeroshots
by: Hoveyda, Mohanna, et al.
Published: (2024)

ULLME: A Unified Framework for Large Language Model Embeddings with Generation-Augmented Learning
by: Man, Hieu, et al.
Published: (2024)

Information Leakage of Sentence Embeddings via Generative Embedding Inversion Attacks
by: Tragoudaras, Antonios, et al.
Published: (2025)

Towards Better Instruction Following Retrieval Models
by: Zhuang, Yuchen, et al.
Published: (2025)

Team LA at SCIDOCA shared task 2025: Citation Discovery via relation-based zero-shot retrieval
by: An, Trieu, et al.
Published: (2025)

Exploring ChatGPT for Next-generation Information Retrieval: Opportunities and Challenges
by: Huang, Yizheng, et al.
Published: (2024)

LUSIFER: Language Universal Space Integration for Enhanced Multilingual Embeddings with Large Language Models
by: Man, Hieu, et al.
Published: (2025)

ORBIT -- Open Recommendation Benchmark for Reproducible Research with Hidden Tests
by: He, Jingyuan, et al.
Published: (2025)

Unveiling the Hidden: Movie Genre and User Bias in Spoiler Detection
by: Zhang, Haokai, et al.
Published: (2025)

A Trio Neural Model for Dynamic Entity Relatedness Ranking
by: Nguyen, Tu, et al.
Published: (2018)

Benchmarking LLM-based Relevance Judgment Methods
by: Arabzadeh, Negar, et al.
Published: (2025)

Uncertainty Quantification and Decomposition for LLM-based Recommendation
by: Kweon, Wonbin, et al.
Published: (2025)

RDRec: Rationale Distillation for LLM-based Recommendation
by: Wang, Xinfeng, et al.
Published: (2024)

LLM-based Embedders for Prior Case Retrieval
by: Premasiri, Damith, et al.
Published: (2025)