| _version_ | 1866901741672333312 |
|---|---|
| author | Edelman, Brice Skolnick, Jeffrey |
| author_facet | Edelman, Brice Skolnick, Jeffrey |
| contents | <p><span>Valsci, an open-source, self-hostable utility that automates large-batch scientific claim verification using any OpenAI-compatible large language model (LLM). Valsci unites retrieval-augmented generation (RAG) with structured bibliometric scoring, enabling users to efficiently search, evaluate, and summarize evidence from the Semantic Scholar database and other academic sources. Unlike conventional standalone LLMs, which often suffer from hallucinations and unreliable citations, Valsci grounds its analyses in verifiable published findings. A guided prompt-flow approach is employed to generate query expansions, retrieve relevant excerpts, and synthesize coherent, evidence-based reports. Preliminary evaluations across claims from the SciFact benchmark dataset reveal that Valsci significantly outperforms base GPT-4o outputs in true/false annotation accuracy and citation hallucination rate. The system is highly scalable, processing hundreds of claims per hour through asynchronous parallelization. By providing an open and transparent platform for large-batch literature verification, Valsci substantially lowers the barrier to comprehensive evidence-based reviews and fosters a more reproducible research ecosystem.</span></p> |
| format | Recurso digital |
| id | zenodo_https___doi_org_10_5281_zenodo_14927291 |
| institution | Zenodo |
| language | |
| publishDate | 2025 |
| publisher | Zenodo |
| record_format | zenodo |
| spellingShingle | Valsci - BMC Bioinformatics submission source code and supporting data Edelman, Brice Skolnick, Jeffrey bioinformatics literature review Review Literature as Topic <p><span>Valsci, an open-source, self-hostable utility that automates large-batch scientific claim verification using any OpenAI-compatible large language model (LLM). Valsci unites retrieval-augmented generation (RAG) with structured bibliometric scoring, enabling users to efficiently search, evaluate, and summarize evidence from the Semantic Scholar database and other academic sources. Unlike conventional standalone LLMs, which often suffer from hallucinations and unreliable citations, Valsci grounds its analyses in verifiable published findings. A guided prompt-flow approach is employed to generate query expansions, retrieve relevant excerpts, and synthesize coherent, evidence-based reports. Preliminary evaluations across claims from the SciFact benchmark dataset reveal that Valsci significantly outperforms base GPT-4o outputs in true/false annotation accuracy and citation hallucination rate. The system is highly scalable, processing hundreds of claims per hour through asynchronous parallelization. By providing an open and transparent platform for large-batch literature verification, Valsci substantially lowers the barrier to comprehensive evidence-based reviews and fosters a more reproducible research ecosystem.</span></p> |
| title | Valsci - BMC Bioinformatics submission source code and supporting data |
| topic | bioinformatics literature review Review Literature as Topic |
| url | https://doi.org/10.5281/zenodo.14927291 |