Cover Image

Towards the Vision-Sound-Language-Action Paradigm: The HEAR Framework for Sound-Centric Manipulation

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Nie, Chang, Deng, Tianchen, Wang, Guangming, Liu, Zhe, Wang, Hesheng
Format:	Preprint
Published:	2026
Subjects:	Robotics Artificial Intelligence Computer Vision and Pattern Recognition Sound
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

MID: A Self-supervised Multimodal Iterative Denoising Framework
by: Nie, Chang, et al.
Published: (2025)

Action2Sound: Ambient-Aware Generation of Action Sounds from Egocentric Videos
by: Chen, Changan, et al.
Published: (2024)

When Vision Speaks for Sound
by: Wen, Xiaofei, et al.
Published: (2026)

Sat2Sound: A Unified Framework for Zero-Shot Soundscape Mapping
by: Khanal, Subash, et al.
Published: (2025)

Compact 3D Gaussian Splatting For Dense Visual SLAM
by: Deng, Tianchen, et al.
Published: (2024)

ManiSoft: Towards Vision-Language Manipulation for Soft Continuum Robotics
by: Wei, Ziyu, et al.
Published: (2026)

PEAfowl: Perception-Enhanced Multi-View Vision-Language-Action for Bimanual Manipulation
by: Fan, Qingyu, et al.
Published: (2026)

SoundVista: Novel-View Ambient Sound Synthesis via Visual-Acoustic Binding
by: Chen, Mingfei, et al.
Published: (2025)

ManiWAV: Learning Robot Manipulation from In-the-Wild Audio-Visual Data
by: Liu, Zeyi, et al.
Published: (2024)

VLA-OS: Structuring and Dissecting Planning Representations and Paradigms in Vision-Language-Action Models
by: Gao, Chongkai, et al.
Published: (2025)

Planning from Imagination: Episodic Simulation and Episodic Memory for Vision-and-Language Navigation
by: Pan, Yiyuan, et al.
Published: (2024)

Global Prior Meets Local Consistency: Dual-Memory Augmented Vision-Language-Action Model for Efficient Robotic Manipulation
by: Li, Zaijing, et al.
Published: (2026)

From Vision to Sound: Advancing Audio Anomaly Detection with Vision-Based Algorithms
by: Barusco, Manuel, et al.
Published: (2025)

Multimodal Spatial Language Maps for Robot Navigation and Manipulation
by: Huang, Chenguang, et al.
Published: (2025)

Explainable Adversarial-Robust Vision-Language-Action Model for Robotic Manipulation
by: Kim, Ju-Young, et al.
Published: (2025)

SaPaVe: Towards Active Perception and Manipulation in Vision-Language-Action Models for Robotics
by: Liu, Mengzhen, et al.
Published: (2026)

Audio-Visual World Models: Towards Multisensory Imagination in Sight and Sound
by: Wang, Jiahua, et al.
Published: (2025)

SoundingActions: Learning How Actions Sound from Narrated Egocentric Videos
by: Chen, Changan, et al.
Published: (2024)

Unison: Harmonizing Motion, Speech, and Sound for Human-Centric Audio-Video Generation
by: Cheng, Shihao, et al.
Published: (2026)

From Instruction to Event: Sound-Triggered Mobile Manipulation
by: Ju, Hao, et al.
Published: (2026)

Continually Evolving Skill Knowledge in Vision Language Action Model
by: Wu, Yuxuan, et al.
Published: (2025)

NeSLAM: Neural Implicit Mapping and Self-Supervised Feature Tracking With Depth Completion and Denoising
by: Deng, Tianchen, et al.
Published: (2024)

Object-Centric World Model for Language-Guided Manipulation
by: Jeong, Youngjoon, et al.
Published: (2025)

SemGauss-SLAM: Dense Semantic Gaussian Splatting SLAM
by: Zhu, Siting, et al.
Published: (2024)

Large VLM-based Vision-Language-Action Models for Robotic Manipulation: A Survey
by: Shao, Rui, et al.
Published: (2025)

SeeingSounds: Learning Audio-to-Visual Alignment via Text
by: Carnemolla, Simone, et al.
Published: (2025)

MAP-VLA: Memory-Augmented Prompting for Vision-Language-Action Model in Robotic Manipulation
by: Li, Runhao, et al.
Published: (2025)

LIBERO-X: Robustness Litmus for Vision-Language-Action Models
by: Wang, Guodong, et al.
Published: (2026)

FLAME: Learning to Navigate with Multimodal LLM in Urban Environments
by: Xu, Yunzhe, et al.
Published: (2024)

ERMV: Editing 4D Robotic Multi-view images to enhance embodied agents
by: Nie, Chang, et al.
Published: (2025)

ALAM: Algebraically Consistent Latent Action Model for Vision-Language-Action Models
by: Tang, Zuojin, et al.
Published: (2026)

TAG: Target-Agnostic Guidance for Stable Object-Centric Inference in Vision-Language-Action Models
by: Zhou, Jiaying, et al.
Published: (2026)

CronusVLA: Towards Efficient and Robust Manipulation via Multi-Frame Vision-Language-Action Modeling
by: Li, Hao, et al.
Published: (2025)

Incremental Joint Learning of Depth, Pose and Implicit Scene Representation on Monocular Camera in Large-scale Scenes
by: Deng, Tianchen, et al.
Published: (2024)

Towards Long-horizon Embodied Agents with Tool-Aligned Vision-Language-Action Models
by: Lei, Zixing, et al.
Published: (2026)

BitVLA: 1-bit Vision-Language-Action Models for Robotics Manipulation
by: Wang, Hongyu, et al.
Published: (2025)

ROSA: Harnessing Robot States for Vision-Language and Action Alignment
by: Wen, Yuqing, et al.
Published: (2025)

Grounding Actions in Camera Space: Observation-Centric Vision-Language-Action Policy
by: Zhang, Tianyi, et al.
Published: (2025)

Taming Text-to-Sounding Video Generation via Advanced Modality Condition and Interaction
by: Guan, Kaisi, et al.
Published: (2025)

VITA: Vision-to-Action Flow Matching Policy
by: Gao, Dechen, et al.
Published: (2025)