MunchSonic: Tracking Fine-grained Dietary Actions through Active Acoustic Sensing on Eyeglasses
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913457172905984 |
|---|---|
| author | Mahmud, Saif Agarwal, Devansh Ajit, Ashwin Liang, Qikang Viranda, Thalia Guimbretiere, Francois Zhang, Cheng |
| author_facet | Mahmud, Saif Agarwal, Devansh Ajit, Ashwin Liang, Qikang Viranda, Thalia Guimbretiere, Francois Zhang, Cheng |
| contents | We introduce MunchSonic, an AI-powered active acoustic sensing system integrated into eyeglasses to track fine-grained dietary actions. MunchSonic emits inaudible ultrasonic waves from the eyeglass frame, with the reflected signals capturing detailed positions and movements of body parts, including the mouth, jaw, arms, and hands involved in eating. These signals are processed by a deep learning pipeline to classify six actions: hand-to-mouth movements for food intake, chewing, drinking, talking, face-hand touching, and other activities (null). In an unconstrained study with 12 participants, MunchSonic achieved a 93.5% macro F1-score in a user-independent evaluation with a 2-second resolution in tracking these actions, also demonstrating its effectiveness in tracking eating episodes and food intake frequency within those episodes. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_21004 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | MunchSonic: Tracking Fine-grained Dietary Actions through Active Acoustic Sensing on Eyeglasses Mahmud, Saif Agarwal, Devansh Ajit, Ashwin Liang, Qikang Viranda, Thalia Guimbretiere, Francois Zhang, Cheng Human-Computer Interaction Emerging Technologies We introduce MunchSonic, an AI-powered active acoustic sensing system integrated into eyeglasses to track fine-grained dietary actions. MunchSonic emits inaudible ultrasonic waves from the eyeglass frame, with the reflected signals capturing detailed positions and movements of body parts, including the mouth, jaw, arms, and hands involved in eating. These signals are processed by a deep learning pipeline to classify six actions: hand-to-mouth movements for food intake, chewing, drinking, talking, face-hand touching, and other activities (null). In an unconstrained study with 12 participants, MunchSonic achieved a 93.5% macro F1-score in a user-independent evaluation with a 2-second resolution in tracking these actions, also demonstrating its effectiveness in tracking eating episodes and food intake frequency within those episodes. |
| title | MunchSonic: Tracking Fine-grained Dietary Actions through Active Acoustic Sensing on Eyeglasses |
| topic | Human-Computer Interaction Emerging Technologies |
| url | https://arxiv.org/abs/2405.21004 |