Scaling Open-Weight Large Language Models for Hydropower Regulatory Information Extraction: A Systematic Analysis
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866912710454673408 |
|---|---|
| author | Yoon, Hong-Jun Ashraf, Faisal Ruggles, Thomas A. Singh, Debjani |
| author_facet | Yoon, Hong-Jun Ashraf, Faisal Ruggles, Thomas A. Singh, Debjani |
| contents | Information extraction from regulatory documents using large language models presents critical trade-offs between performance and computational resources. We evaluated seven open-weight models (0.6B-70B parameters) on hydropower licensing documentation to provide empirical deployment guidance.
Our analysis identified a pronounced 14B parameter threshold where validation methods transition from ineffective (F1 $<$ 0.15) to viable (F1 = 0.64). Consumer-deployable models achieve 64\% F1 through appropriate validation, while smaller models plateau at 51\%. Large-scale models approach 77\% F1 but require enterprise infrastructure.
We identified systematic hallucination patterns where perfect recall indicates extraction failure rather than success in smaller models. Our findings establish the first comprehensive resource-performance mapping for open-weight information extraction in regulatory contexts, enabling evidence-based model selection.
These results provide immediate value for hydropower compliance while contributing insights into parameter scaling effects that generalize across information extraction tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_11821 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Scaling Open-Weight Large Language Models for Hydropower Regulatory Information Extraction: A Systematic Analysis Yoon, Hong-Jun Ashraf, Faisal Ruggles, Thomas A. Singh, Debjani Computation and Language Artificial Intelligence Information extraction from regulatory documents using large language models presents critical trade-offs between performance and computational resources. We evaluated seven open-weight models (0.6B-70B parameters) on hydropower licensing documentation to provide empirical deployment guidance. Our analysis identified a pronounced 14B parameter threshold where validation methods transition from ineffective (F1 $<$ 0.15) to viable (F1 = 0.64). Consumer-deployable models achieve 64\% F1 through appropriate validation, while smaller models plateau at 51\%. Large-scale models approach 77\% F1 but require enterprise infrastructure. We identified systematic hallucination patterns where perfect recall indicates extraction failure rather than success in smaller models. Our findings establish the first comprehensive resource-performance mapping for open-weight information extraction in regulatory contexts, enabling evidence-based model selection. These results provide immediate value for hydropower compliance while contributing insights into parameter scaling effects that generalize across information extraction tasks. |
| title | Scaling Open-Weight Large Language Models for Hydropower Regulatory Information Extraction: A Systematic Analysis |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2511.11821 |