Artificial-Intelligence-Assisted Multi-Modal Terahertz Sensing and Environment Reconstruction
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914620877307904 |
|---|---|
| author | Lyu, Yejian Fang, Zitong Yuan, Zhiqiang Wymeersch, Henk Han, Chong |
| author_facet | Lyu, Yejian Fang, Zitong Yuan, Zhiqiang Wymeersch, Henk Han, Chong |
| contents | Multi-modal sensing is an important enabler for future environment-aware wireless systems, since a single sensing modality is generally insufficient to provide accurate metric geometry, material awareness, and semantic interpretability in complex environments. This paper presents a measurement-based multi-modal THz sensing and vision framework for indoor environment reconstruction. A three-dimensional monostatic THz channel sounding system operating at 290-310 GHz is integrated with an omnidirectional fisheye camera to acquire radio-frequency and visual observations from a common sensing viewpoint. From the measured THz data, a signal processing pipeline extracts multipath components and infers geometryand material-consistent structural primitives through trajectory tracking-assisted parameter estimation, graph-based structure discovery, planar reconstruction, and reflection-loss analysis. In parallel, AI-based visual perception modules extract object-level semantic masks and depth priors from panoramic images. To associate these heterogeneous representations, an agentic-AI-based task-driven THz-agent module is developed to select appropriate integration tools according to the attributes of the modality-specific outputs. Through angular alignment and consistency analysis, THz-derived metric geometry and material information are associated with vision-derived semantic regions and depth priors, enabling geometry-consistent and semantically interpretable environment reconstruction directly from measurements. Experimental validation in the indoor L-shaped hallway demonstrates that the proposed framework reconstructs dominant structural elements with centimeter-level accuracy while identifying semantic categories and material attributes of representative indoor objects. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2606_01186 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Artificial-Intelligence-Assisted Multi-Modal Terahertz Sensing and Environment Reconstruction Lyu, Yejian Fang, Zitong Yuan, Zhiqiang Wymeersch, Henk Han, Chong Signal Processing Multi-modal sensing is an important enabler for future environment-aware wireless systems, since a single sensing modality is generally insufficient to provide accurate metric geometry, material awareness, and semantic interpretability in complex environments. This paper presents a measurement-based multi-modal THz sensing and vision framework for indoor environment reconstruction. A three-dimensional monostatic THz channel sounding system operating at 290-310 GHz is integrated with an omnidirectional fisheye camera to acquire radio-frequency and visual observations from a common sensing viewpoint. From the measured THz data, a signal processing pipeline extracts multipath components and infers geometryand material-consistent structural primitives through trajectory tracking-assisted parameter estimation, graph-based structure discovery, planar reconstruction, and reflection-loss analysis. In parallel, AI-based visual perception modules extract object-level semantic masks and depth priors from panoramic images. To associate these heterogeneous representations, an agentic-AI-based task-driven THz-agent module is developed to select appropriate integration tools according to the attributes of the modality-specific outputs. Through angular alignment and consistency analysis, THz-derived metric geometry and material information are associated with vision-derived semantic regions and depth priors, enabling geometry-consistent and semantically interpretable environment reconstruction directly from measurements. Experimental validation in the indoor L-shaped hallway demonstrates that the proposed framework reconstructs dominant structural elements with centimeter-level accuracy while identifying semantic categories and material attributes of representative indoor objects. |
| title | Artificial-Intelligence-Assisted Multi-Modal Terahertz Sensing and Environment Reconstruction |
| topic | Signal Processing |
| url | https://arxiv.org/abs/2606.01186 |