Saved in:
| Main Authors: | , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2503.10212 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915214778171392 |
|---|---|
| author | Xu, Teng Zhou, Taotao Wang, Youjia Yang, Peng Tang, Simin Shao, Kuixiang Tang, Zifeng Liu, Yifei Chen, Xinyuan Wang, Hongshuang Wang, Xiaohui Luo, Huoqing Wang, Jingya Hu, Ji Yu, Jingyi |
| author_facet | Xu, Teng Zhou, Taotao Wang, Youjia Yang, Peng Tang, Simin Shao, Kuixiang Tang, Zifeng Liu, Yifei Chen, Xinyuan Wang, Hongshuang Wang, Xiaohui Luo, Huoqing Wang, Jingya Hu, Ji Yu, Jingyi |
| contents | Analyzing animal behavior is crucial in advancing neuroscience, yet quantifying and deciphering its intricate dynamics remains a significant challenge. Traditional machine vision approaches, despite their ability to detect spontaneous behaviors, fall short due to limited interpretability and reliance on manual labeling, which restricts the exploration of the full behavioral spectrum. Here, we introduce MouseGPT, a Vision-Language Model (VLM) that integrates visual cues with natural language to revolutionize mouse behavior analysis. Built upon our first-of-its-kind dataset - incorporating pose dynamics and open-vocabulary behavioral annotations across over 42 million frames of diverse psychiatric conditions - MouseGPT provides a novel, context-rich method for comprehensive behavior interpretation. Our holistic analysis framework enables detailed behavior profiling, clustering, and novel behavior discovery, offering deep insights without the need for labor - intensive manual annotation. Evaluations reveal that MouseGPT surpasses existing models in precision, adaptability, and descriptive richness, positioning it as a transformative tool for ethology and for unraveling complex behavioral dynamics in animal models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_10212 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | MouseGPT: A Large-scale Vision-Language Model for Mouse Behavior Analysis Xu, Teng Zhou, Taotao Wang, Youjia Yang, Peng Tang, Simin Shao, Kuixiang Tang, Zifeng Liu, Yifei Chen, Xinyuan Wang, Hongshuang Wang, Xiaohui Luo, Huoqing Wang, Jingya Hu, Ji Yu, Jingyi Computer Vision and Pattern Recognition Analyzing animal behavior is crucial in advancing neuroscience, yet quantifying and deciphering its intricate dynamics remains a significant challenge. Traditional machine vision approaches, despite their ability to detect spontaneous behaviors, fall short due to limited interpretability and reliance on manual labeling, which restricts the exploration of the full behavioral spectrum. Here, we introduce MouseGPT, a Vision-Language Model (VLM) that integrates visual cues with natural language to revolutionize mouse behavior analysis. Built upon our first-of-its-kind dataset - incorporating pose dynamics and open-vocabulary behavioral annotations across over 42 million frames of diverse psychiatric conditions - MouseGPT provides a novel, context-rich method for comprehensive behavior interpretation. Our holistic analysis framework enables detailed behavior profiling, clustering, and novel behavior discovery, offering deep insights without the need for labor - intensive manual annotation. Evaluations reveal that MouseGPT surpasses existing models in precision, adaptability, and descriptive richness, positioning it as a transformative tool for ethology and for unraveling complex behavioral dynamics in animal models. |
| title | MouseGPT: A Large-scale Vision-Language Model for Mouse Behavior Analysis |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2503.10212 |