ChatGPT for Automated Cross‐Checking of Authors' Conflicts of Interest Against Industry Payments
Fuente:
Wiley Open Access
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Artículo Open Access |
| Published: |
Wiley
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1867012390734790656 |
|---|---|
| author | Conrad Safranek Christina Liu Rhys Richmond Trinithas Boyi Ryan Rimmer R. Peter Manes |
| author_facet | Conrad Safranek Christina Liu Rhys Richmond Trinithas Boyi Ryan Rimmer R. Peter Manes Conrad Safranek Christina Liu Rhys Richmond Trinithas Boyi Ryan Rimmer R. Peter Manes |
| collection | Wiley Open Access |
| contents | ChatGPT for Automated Cross‐Checking of Authors' Conflicts of Interest Against Industry Payments Conrad Safranek Christina Liu Rhys Richmond Trinithas Boyi Ryan Rimmer R. Peter Manes Otolaryngology–Head and Neck Surgery AbstractObjectiveThe Centers for Medicare & Medicaid Services “OpenPayments” database tracks industry payments to US physicians to improve research conflicts of interest (COIs) transparency, but manual cross‐checking of articles' authors against this database is labor‐intensive. This study aims to assess the potential of large language models (LLMs) like ChatGPT to automate COI data analysis in medical publications.Study DesignAn observational study analyzing the accuracy of ChatGPT in automating the cross‐checking of COI disclosures in medical research articles against the OpenPayments database.SettingPublications regarding Food and Drug Administration‐approved biologics for chronic rhinosinusitis with nasal polyposis: omalizumab, mepolizumab, and dupilumab.MethodsFirst, ChatGPT evaluated author affiliations from PubMed to identify those based in the United States. Second, for author names matching 1 or multiple payment recipients in OpenPayments, ChatGPT undertook a comparative analysis between author affiliation and OpenPayments recipient metadata. Third, ChatGPT scrutinized full article COI statements, producing an intricate matrix of disclosures for each author against each relevant company (Sanofi, Regeneron, Genentech, Novartis, and GlaxoSmithKline). A random subset of responses was manually checked for accuracy.ResultsIn total, 78 relevant articles and 294 unique US authors were included, leading to 980 LLM queries. Manual verification showed accuracies of 100% (200/200; 95% confidence interval [CI]: 98.1%‐100%) for country analysis, 97.4% (113/116; 95% CI: 92.7%‐99.1%) for matching author affiliations with OpenPayments metadata, and 99.2% (1091/1100; 95% CI: 98.5%‐99.6%) for COI statement data extraction.ConclusionLLMs have robust potential to automate author‐company‐specific COI cross‐checking against the OpenPayments database. Our findings pave the way for streamlined, efficient, and accurate COI assessment that could be widely employed across medical research. 10.1002/ohn.720 http://onlinelibrary.wiley.com/termsAndConditions#vor |
| doi_str_mv | 10.1002/ohn.720 |
| format | Artículo Open Access |
| id | wiley_oa_10_1002_ohn_720 |
| institution | Wiley Open Access |
| license_str_mv | http://onlinelibrary.wiley.com/termsAndConditions#vor |
| publishDate | 2024 |
| publisher | Wiley |
| record_format | wiley_oa |
| spellingShingle | ChatGPT for Automated Cross‐Checking of Authors' Conflicts of Interest Against Industry Payments Conrad Safranek Christina Liu Rhys Richmond Trinithas Boyi Ryan Rimmer R. Peter Manes Otolaryngology–Head and Neck Surgery ChatGPT for Automated Cross‐Checking of Authors' Conflicts of Interest Against Industry Payments Conrad Safranek Christina Liu Rhys Richmond Trinithas Boyi Ryan Rimmer R. Peter Manes Otolaryngology–Head and Neck Surgery AbstractObjectiveThe Centers for Medicare & Medicaid Services “OpenPayments” database tracks industry payments to US physicians to improve research conflicts of interest (COIs) transparency, but manual cross‐checking of articles' authors against this database is labor‐intensive. This study aims to assess the potential of large language models (LLMs) like ChatGPT to automate COI data analysis in medical publications.Study DesignAn observational study analyzing the accuracy of ChatGPT in automating the cross‐checking of COI disclosures in medical research articles against the OpenPayments database.SettingPublications regarding Food and Drug Administration‐approved biologics for chronic rhinosinusitis with nasal polyposis: omalizumab, mepolizumab, and dupilumab.MethodsFirst, ChatGPT evaluated author affiliations from PubMed to identify those based in the United States. Second, for author names matching 1 or multiple payment recipients in OpenPayments, ChatGPT undertook a comparative analysis between author affiliation and OpenPayments recipient metadata. Third, ChatGPT scrutinized full article COI statements, producing an intricate matrix of disclosures for each author against each relevant company (Sanofi, Regeneron, Genentech, Novartis, and GlaxoSmithKline). A random subset of responses was manually checked for accuracy.ResultsIn total, 78 relevant articles and 294 unique US authors were included, leading to 980 LLM queries. Manual verification showed accuracies of 100% (200/200; 95% confidence interval [CI]: 98.1%‐100%) for country analysis, 97.4% (113/116; 95% CI: 92.7%‐99.1%) for matching author affiliations with OpenPayments metadata, and 99.2% (1091/1100; 95% CI: 98.5%‐99.6%) for COI statement data extraction.ConclusionLLMs have robust potential to automate author‐company‐specific COI cross‐checking against the OpenPayments database. Our findings pave the way for streamlined, efficient, and accurate COI assessment that could be widely employed across medical research. 10.1002/ohn.720 http://onlinelibrary.wiley.com/termsAndConditions#vor |
| title | ChatGPT for Automated Cross‐Checking of Authors' Conflicts of Interest Against Industry Payments |
| topic | Otolaryngology–Head and Neck Surgery |
| url | https://aao-hnsfjournals.onlinelibrary.wiley.com/doi/10.1002/ohn.720 |