Cover Image

Foundation Model's Embedded Representations May Detect Distribution Shift

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Vargas, Max, Tsou, Adam, Engel, Andrew, Chiang, Tony
Format:	Preprint
Published:	2023
Subjects:	Machine Learning Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

The Rotary Position Embedding May Cause Dimension Inefficiency in Attention Heads for Long-Distance Retrieval
by: Chiang, Ting-Rui, et al.
Published: (2025)

Understanding Generative AI Content with Embedding Models
by: Vargas, Max, et al.
Published: (2024)

Efficient kernel surrogates for neural network-based regression
by: Qadeer, Saad, et al.
Published: (2023)

A Baseline Analysis of Reward Models' Ability To Accurately Analyze Foundation Models Under Distribution Shift
by: LeVine, Will, et al.
Published: (2023)

GEM: Gaussian Embedding Modeling for Out-of-Distribution Detection in GUI Agents
by: Wu, Zheng, et al.
Published: (2025)

Geometry-Adaptive Explainer for Faithful Dictionary-Based Interpretability under Distribution Shift
by: Lim, Sungjun, et al.
Published: (2026)

Few-shot Adaptation to Distribution Shifts By Mixing Source and Target Embeddings
by: Xue, Yihao, et al.
Published: (2023)

EmbedLLM: Learning Compact Representations of Large Language Models
by: Zhuang, Richard, et al.
Published: (2024)

Enhancing Foundation Models in Transaction Understanding with LLM-based Sentence Embeddings
by: Fan, Xiran, et al.
Published: (2025)

A Representation-Level Assessment of Bias Mitigation in Foundation Models
by: Nizhnichenkov, Svetoslav, et al.
Published: (2026)

Omitted Variable Bias in Language Models Under Distribution Shift
by: Lin, Victoria, et al.
Published: (2026)

Multi-Lingual Malaysian Embedding: Leveraging Large Language Models for Semantic Representations
by: Zolkepli, Husein, et al.
Published: (2024)

Embedding Trajectory for Out-of-Distribution Detection in Mathematical Reasoning
by: Wang, Yiming, et al.
Published: (2024)

Shared Global and Local Geometry of Language Model Embeddings
by: Lee, Andrew, et al.
Published: (2025)

When In-Distribution Gains Fail: Evaluating Weak-to-Strong Reward Models under Preference Shift
by: Le, Khoi, et al.
Published: (2026)

Physically Interpretable AlphaEarth Foundation Model Embeddings Enable LLM-Based Land Surface Intelligence
by: Rahman, Mashrekur
Published: (2026)

MetaRM: Shifted Distributions Alignment via Meta-Learning
by: Dou, Shihan, et al.
Published: (2024)

Nostra Domina at EvaLatin 2024: Improving Latin Polarity Detection through Data Augmentation
by: Bothwell, Stephen, et al.
Published: (2024)

Tokenization Tradeoffs in Structured EHR Foundation Models
by: Guo, Lin Lawrence, et al.
Published: (2026)

Representation Learning of Structured Data for Medical Foundation Models
by: Dwivedi, Vijay Prakash, et al.
Published: (2024)

SMEC: Rethinking Matryoshka Representation Learning for Retrieval Embedding Compression
by: Zhang, Biao, et al.
Published: (2025)

On SkipGram Word Embedding Models with Negative Sampling: Unified Framework and Impact of Noise Distributions
by: Liu, Dezhi, et al.
Published: (2020)

Hallucination Detection and Mitigation with Diffusion in Multi-Variate Time-Series Foundation Models
by: Wichitwechkarn, Vijja, et al.
Published: (2025)

When Prompts Interact: Assessing Prompt Arithmetic for Deconfounding under Distribution Shift
by: Sheng, Zhecheng, et al.
Published: (2026)

Embedding-Based Approaches to Hyperpartisan News Detection
by: Mohan, Karthik
Published: (2025)

Self-Verified Distillation: Your Language Model Is Secretly Its Own Synthetic Data Pipeline
by: Lee, Tony, et al.
Published: (2026)

InCo-DPO: Balancing Distribution Shift and Data Quality for Enhanced Preference Optimization
by: Wang, Yunan, et al.
Published: (2025)

Tackling Distribution Shift in LLM via KILO: Knowledge-Instructed Learning for Continual Adaptation
by: Muttakhiroh, Iing, et al.
Published: (2025)

LEAF: Knowledge Distillation of Text Embedding Models with Teacher-Aligned Representations
by: Vujanic, Robin, et al.
Published: (2025)

View From Above: A Framework for Evaluating Distribution Shifts in Model Behavior
by: Chopra, Tanush, et al.
Published: (2024)

Detection of Human and Machine-Authored Fake News in Urdu
by: Ali, Muhammad Zain, et al.
Published: (2024)

LongEmbed: Extending Embedding Models for Long Context Retrieval
by: Zhu, Dawei, et al.
Published: (2024)

Sarcasm Detection in Tweets with BERT and GloVe Embeddings
by: Khatri, Akshay, et al.
Published: (2020)

DT-Transformer: A Foundation Model for Disease Trajectory Prediction on a Real-world Health System
by: Zhu, Yunying, et al.
Published: (2026)

Static Word Embeddings for Sentence Semantic Representation
by: Wada, Takashi, et al.
Published: (2025)

Enhancing Antibiotic Stewardship using a Natural Language Approach for Better Feature Representation
by: Lee, Simon A., et al.
Published: (2024)

Empirical Study of Named Entity Recognition Performance Using Distribution-aware Word Embedding
by: Chen, Xin, et al.
Published: (2021)

Scaling Embedding Layers in Language Models
by: Yu, Da, et al.
Published: (2025)

Repetition Improves Language Model Embeddings
by: Springer, Jacob Mitchell, et al.
Published: (2024)

MedRep: Medical Concept Representation for General Electronic Health Record Foundation Models
by: Kim, Junmo, et al.
Published: (2025)