Cover Image

Aligning Brain Signals with Multimodal Speech and Vision Embeddings

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Shapovalenko, Kateryna, Auster, Quentin
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

A Penny for Your Thoughts: Decoding Speech from Inexpensive Brain Signals
by: Auster, Quentin, et al.
Published: (2025)

MultiDiffNet: A Multi-Objective Diffusion Framework for Generalizable Brain Decoding
by: Zhang, Mengchun, et al.
Published: (2025)

ASPEN: Spectral-Temporal Fusion for Cross-Subject Brain Decoding
by: Lee, Megan, et al.
Published: (2026)

Explaining, Verifying, and Aligning Semantic Hierarchies in Vision-Language Model Embeddings
by: Schwalbe, Gesina, et al.
Published: (2026)

CogniAlign: Word-Level Multimodal Speech Alignment with Gated Cross-Attention for Alzheimer's Detection
by: Ortiz-Perez, David, et al.
Published: (2025)

Mechanistic Interpretability of Brain-to-Speech Models Across Speech Modes
by: Maghsoudi, Maryam, et al.
Published: (2026)

PLAICraft: Large-Scale Time-Aligned Vision-Speech-Action Dataset for Embodied AI
by: He, Yingchen, et al.
Published: (2025)

Structure-Aligned Protein Language Model
by: Chen, Can, et al.
Published: (2025)

Think Then Embed: Generative Context Improves Multimodal Embedding
by: Cui, Xuanming, et al.
Published: (2025)

Dywave: Event-Aligned Dynamic Tokenization for Heterogeneous IoT Sensing Signals
by: Kimura, Tomoyoshi, et al.
Published: (2026)

Beyond Fixed Frames: Dynamic Character-Aligned Speech Tokenization
by: Della Libera, Luca, et al.
Published: (2026)

Aligned at the Start: Conceptual Groupings in LLM Embeddings
by: Khatir, Mehrdad, et al.
Published: (2024)

Rethinking Multimodality: Optimizing Multimodal Deep Learning for Biomedical Signal Classification
by: Oladunni, Timothy, et al.
Published: (2025)

Multimodal Cancer Modeling in the Age of Foundation Model Embeddings
by: Song, Steven, et al.
Published: (2025)

ES-Merging: Biological MLLM Merging via Embedding Space Signals
by: Lee, Wonbin, et al.
Published: (2026)

Embedding-Aligned Language Models
by: Tennenholtz, Guy, et al.
Published: (2024)

SEM: Sparse Embedding Modulation for Post-Hoc Debiasing of Vision-Language Models
by: Guimard, Quentin, et al.
Published: (2026)

Revealing Vision-Language Integration in the Brain with Multimodal Networks
by: Subramaniam, Vighnesh, et al.
Published: (2024)

Aligning Generative Speech Enhancement with Perceptual Feedback
by: Li, Haoyang, et al.
Published: (2025)

LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixels
by: Maes, Lucas, et al.
Published: (2026)

Aligning Large Vision-Language Models by Deep Reinforcement Learning and Direct Preference Optimization
by: Nguyen, Thanh Thi, et al.
Published: (2025)

Interpretable Spatio-Temporal Embedding for Brain Structural-Effective Network with Ordinary Differential Equation
by: Tang, Haoteng, et al.
Published: (2024)

How Well Do Multimodal Models Reason on ECG Signals?
by: Xu, Maxwell A., et al.
Published: (2026)

SOTAlign: Semi-Supervised Alignment of Unimodal Vision and Language Models via Optimal Transport
by: Roschmann, Simon, et al.
Published: (2026)

UME-R1: Exploring Reasoning-Driven Generative Multimodal Embeddings
by: Lan, Zhibin, et al.
Published: (2025)

Meta-Learn Unimodal Signals with Weak Supervision for Multimodal Sentiment Analysis
by: Mai, Sijie, et al.
Published: (2024)

VL-KGE: Vision-Language Models Meet Knowledge Graph Embeddings
by: Efthymiou, Athanasios, et al.
Published: (2026)

Brain Foundation Models: A Survey on Advancements in Neural Signal Processing and Brain Discovery
by: Zhou, Xinliang, et al.
Published: (2025)

Multimodal Physiological Signals Representation Learning via Multiscale Contrasting for Depression Recognition
by: Shao, Kai, et al.
Published: (2024)

From Video to EEG: Adapting Joint Embedding Predictive Architecture to Uncover Saptiotemporal Dynamics in Brain Signal Analysis
by: Hojjati, Amirabbas, et al.
Published: (2025)

Evaluating Menu OCR and Translation: A Benchmark for Aligning Human and Automated Evaluations in Large Vision-Language Models
by: Wu, Zhanglin, et al.
Published: (2025)

Hyperbolic Kernel Graph Neural Networks for Neurocognitive Decline Analysis from Multimodal Brain Imaging
by: Yang, Meimei, et al.
Published: (2025)

CAMEL-CLIP: Channel-aware Multimodal Electroencephalography-text Alignment for Generalizable Brain Foundation Models
by: Choi, Hanseul, et al.
Published: (2026)

Cross-Temporal Attention Fusion (CTAF) for Multimodal Physiological Signals in Self-Supervised Learning
by: Khorasani, Arian, et al.
Published: (2026)

Forecasting Application Counts in Talent Acquisition Platforms: Harnessing Multimodal Signals using LMs
by: Kabir, Md Ahsanul, et al.
Published: (2024)

Context-Adaptive Multi-Prompt Embedding with Large Language Models for Vision-Language Alignment
by: Kim, Dahun, et al.
Published: (2025)

Recall: Empowering Multimodal Embedding for Edge Devices
by: Cai, Dongqi, et al.
Published: (2024)

EAGLE: Efficient Alignment of Generalized Latent Embeddings for Multimodal Survival Prediction with Interpretable Attribution Analysis
by: Tripathi, Aakash, et al.
Published: (2025)

PhysioME: A Robust Multimodal Self-Supervised Framework for Physiological Signals with Missing Modalities
by: Lee, Cheol-Hui, et al.
Published: (2025)

HeartcareGPT: A Unified Multimodal ECG Suite for Dual Signal-Image Modeling and Understanding
by: Xie, Yihan, et al.
Published: (2025)