Large model retrieval enhancement framework for construction site risk identification
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866917068597624832 |
|---|---|
| author | Li, Jiawei Yang, Chengye Zhang, Yaochen Sun, Weilin Meng, Lei Meng, Xiangxu |
| author_facet | Li, Jiawei Yang, Chengye Zhang, Yaochen Sun, Weilin Meng, Lei Meng, Xiangxu |
| contents | This study addresses construction site hazard identification by proposing a retrieval-augmented framework that enhances large language models (LLMs) without requiring fine-tuning. Current LLM-based approaches face limitations: image-text matching struggles with complex hazards, while instruction tuning lacks generalization and is resource-intensive. Our method dynamically integrates external knowledge and retrieved similar cases via prompt tuning, overcoming LLMs' limitations in domain knowledge and feature correlation. The framework comprises a case database, an image retrieval module, and an LLM-based reasoning module. Evaluated on real-site data, our approach boosted GLM-4V's accuracy to 50%, a 35.49% improvement over baselines, with consistent gains across hazard types. Ablation studies validated the effectiveness of our image retrieval strategy, showing the superiority of our LPIPS- and CLIP-based method. The proposed technique significantly improves identification accuracy and contextual understanding, demonstrating strong generalization and offering a practical path for intelligent safety risk detection in construction. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_02073 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Large model retrieval enhancement framework for construction site risk identification Li, Jiawei Yang, Chengye Zhang, Yaochen Sun, Weilin Meng, Lei Meng, Xiangxu Artificial Intelligence This study addresses construction site hazard identification by proposing a retrieval-augmented framework that enhances large language models (LLMs) without requiring fine-tuning. Current LLM-based approaches face limitations: image-text matching struggles with complex hazards, while instruction tuning lacks generalization and is resource-intensive. Our method dynamically integrates external knowledge and retrieved similar cases via prompt tuning, overcoming LLMs' limitations in domain knowledge and feature correlation. The framework comprises a case database, an image retrieval module, and an LLM-based reasoning module. Evaluated on real-site data, our approach boosted GLM-4V's accuracy to 50%, a 35.49% improvement over baselines, with consistent gains across hazard types. Ablation studies validated the effectiveness of our image retrieval strategy, showing the superiority of our LPIPS- and CLIP-based method. The proposed technique significantly improves identification accuracy and contextual understanding, demonstrating strong generalization and offering a practical path for intelligent safety risk detection in construction. |
| title | Large model retrieval enhancement framework for construction site risk identification |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2508.02073 |