Cover Image

Multimodal Audio-based Disease Prediction with Transformer-based Hierarchical Fusion Network

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Cai, Jinjin, Wang, Ruiqi, Zhao, Dezhong, Yuan, Ziqin, McKenna, Victoria, Friedman, Aaron, Foot, Rachel, Storey, Susan, Boente, Ryan, Vhaduri, Sudip, Min, Byung-Cheol
Format:	Preprint
Published:	2024
Subjects:	Sound Artificial Intelligence Machine Learning Audio and Speech Processing
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Mitigating Sex Bias in Audio Data-driven COPD and COVID-19 Breathing Pattern Detection Models
by: Pfeifer, Rachel, et al.
Published: (2024)

Sound Classification of Four Insect Classes
by: Wang, Yinxuan, et al.
Published: (2024)

PrefCLM: Enhancing Preference-based Reinforcement Learning with Crowdsourced Large Language Models
by: Wang, Ruiqi, et al.
Published: (2024)

PrefMMT: Modeling Human Preferences in Preference-based Reinforcement Learning with Multimodal Transformers
by: Zhao, Dezhong, et al.
Published: (2024)

Personalization in Human-Robot Interaction through Preference-based Action Representation Learning
by: Wang, Ruiqi, et al.
Published: (2024)

Attentive Fusion: A Transformer-based Approach to Multimodal Hate Speech Detection
by: Mandal, Atanu, et al.
Published: (2024)

FusionAudio-1.2M: Towards Fine-grained Audio Captioning with Multimodal Contextual Fusion
by: Chen, Shunian, et al.
Published: (2025)

PrefMoE: Robust Preference Modeling with Mixture-of-Experts Reward Learning
by: Yuan, Ziqin, et al.
Published: (2026)

PRIMT: Preference-based Reinforcement Learning with Multimodal Feedback and Trajectory Synthesis from Foundation Models
by: Wang, Ruiqi, et al.
Published: (2025)

Audio-Guided Fusion Techniques for Multimodal Emotion Analysis
by: Shi, Pujin, et al.
Published: (2024)

Audio-text Retrieval with Transformer-based Hierarchical Alignment and Disentangled Cross-modal Representation
by: Xin, Yifei, et al.
Published: (2024)

Towards Fusion of Neural Audio Codec-based Representations with Spectral for Heart Murmur Classification via Bandit-based Cross-Attention Mechanism
by: Phukan, Orchid Chetia, et al.
Published: (2025)

Text-based Audio Retrieval by Learning from Similarities between Audio Captions
by: Xie, Huang, et al.
Published: (2024)

Multimodal Magic Elevating Depression Detection with a Fusion of Text and Audio Intelligence
by: Gan, Lindy, et al.
Published: (2025)

MLCA-AVSR: Multi-Layer Cross Attention Fusion based Audio-Visual Speech Recognition
by: Wang, He, et al.
Published: (2024)

Audio Spatially-Guided Fusion for Audio-Visual Navigation
by: Zhou, Xinyu, et al.
Published: (2026)

Audio-Visual Person Verification based on Recursive Fusion of Joint Cross-Attention
by: Praveen, R. Gnana, et al.
Published: (2024)

Adaptive Task Allocation in Multi-Human Multi-Robot Teams under Team Heterogeneity and Dynamic Information Uncertainty
by: Yuan, Ziqin, et al.
Published: (2024)

An Octave-based Multi-Resolution CQT Architecture for Diffusion-based Audio Generation
by: da Costa, Maurício do V. M., et al.
Published: (2025)

DSA-Tokenizer: Disentangled Semantic-Acoustic Tokenization via Flow Matching-based Hierarchical Fusion
by: Zhang, Hanlin, et al.
Published: (2026)

A Survey of Audio Reasoning in Multimodal Foundation Models
by: Guo, Zhihan, et al.
Published: (2026)

Multimodal Sentiment Analysis based on Video and Audio Inputs
by: Fernandez, Antonio, et al.
Published: (2024)

Continuous Learning of Transformer-based Audio Deepfake Detection
by: Le, Tuan Duy Nguyen, et al.
Published: (2024)

Cross-Modal Bottleneck Fusion For Noise Robust Audio-Visual Speech Recognition
by: Ok, Seaone, et al.
Published: (2026)

Towards Multimodal Query-Based Spatial Audio Source Extraction
by: Yu, Chenxin, et al.
Published: (2025)

BLAT: Bootstrapping Language-Audio Pre-training based on AudioSet Tag-guided Synthetic Data
by: Xu, Xuenan, et al.
Published: (2023)

Multi-Source Evidence Fusion for Audio Question Answering
by: Olev, Aivo, et al.
Published: (2026)

Language-based Audio Moment Retrieval
by: Munakata, Hokuto, et al.
Published: (2024)

Fusing Audio and Metadata Embeddings Improves Language-based Audio Retrieval
by: Primus, Paul, et al.
Published: (2024)

Measuring Audio Prompt Adherence with Distribution-based Embedding Distances
by: Grachten, Maarten
Published: (2024)

Audio Deepfake Detection with Self-Supervised WavLM and Multi-Fusion Attentive Classifier
by: Guo, Yinlin, et al.
Published: (2023)

REBEL: Rule-based and Experience-enhanced Learning with LLMs for Initial Task Allocation in Multi-Human Multi-Robot Teaming
by: Gupte, Arjun, et al.
Published: (2024)

SAM Audio Judge: A Unified Multimodal Framework for Perceptual Evaluation of Audio Separation
by: Wang, Helin, et al.
Published: (2026)

Towards Audio Codec-based Speech Separation
by: Yip, Jia Qi, et al.
Published: (2024)

Multimodal Representation Loss Between Timed Text and Audio for Regularized Speech Separation
by: Hsieh, Tsun-An, et al.
Published: (2024)

PromptSep: Generative Audio Separation via Multimodal Prompting
by: Wen, Yutong, et al.
Published: (2025)

From Contrast to Commonality: Audio Commonality Captioning for Enhanced Audio-Text Cross-modal Understanding in Multimodal LLMs
by: Jia, Yuhang, et al.
Published: (2025)

Spatial-Aware Conditioned Fusion for Audio-Visual Navigation
by: Wu, Shaohang, et al.
Published: (2026)

The T12 System for AudioMOS Challenge 2025: Audio Aesthetics Score Prediction System Using KAN- and VERSA-based Models
by: Yamamoto, Katsuhiko, et al.
Published: (2025)

AudioRouter: Data Efficient Audio Understanding via RL based Dual Reasoning
by: Chen, Liyang, et al.
Published: (2026)