Multi-Frame Vision-Language Model for Long-form Reasoning in Driver Behavior Analysis
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866914899556302848 |
|---|---|
| author | Takato, Hiroshi Tsutsui, Hiroshi Soda, Komei Kamigaito, Hidetaka |
| author_facet | Takato, Hiroshi Tsutsui, Hiroshi Soda, Komei Kamigaito, Hidetaka |
| contents | Identifying risky driving behavior in real-world situations is essential for the safety of both drivers and pedestrians. However, integrating natural language models in this field remains relatively untapped. To address this, we created a novel multi-modal instruction tuning dataset and driver coaching inference system. Our primary use case is dashcam-based coaching for commercial drivers. The North American Dashcam Market is expected to register a CAGR of 15.4 percent from 2022 to 2027. Our dataset enables language models to learn visual instructions across various risky driving scenarios, emphasizing detailed reasoning crucial for effective driver coaching and managerial comprehension. Our model is trained on road-facing and driver-facing RGB camera footage, capturing the comprehensive scope of driving behavior in vehicles equipped with dashcams. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2408_01682 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Multi-Frame Vision-Language Model for Long-form Reasoning in Driver Behavior Analysis Takato, Hiroshi Tsutsui, Hiroshi Soda, Komei Kamigaito, Hidetaka Computation and Language Computer Vision and Pattern Recognition Identifying risky driving behavior in real-world situations is essential for the safety of both drivers and pedestrians. However, integrating natural language models in this field remains relatively untapped. To address this, we created a novel multi-modal instruction tuning dataset and driver coaching inference system. Our primary use case is dashcam-based coaching for commercial drivers. The North American Dashcam Market is expected to register a CAGR of 15.4 percent from 2022 to 2027. Our dataset enables language models to learn visual instructions across various risky driving scenarios, emphasizing detailed reasoning crucial for effective driver coaching and managerial comprehension. Our model is trained on road-facing and driver-facing RGB camera footage, capturing the comprehensive scope of driving behavior in vehicles equipped with dashcams. |
| title | Multi-Frame Vision-Language Model for Long-form Reasoning in Driver Behavior Analysis |
| topic | Computation and Language Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2408.01682 |