On Classification with Large Language Models in Cultural Analytics
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866914974165630976 |
|---|---|
| author | Bamman, David Chang, Kent K. Lucy, Li Zhou, Naitian |
| author_facet | Bamman, David Chang, Kent K. Lucy, Li Zhou, Naitian |
| contents | In this work, we survey the way in which classification is used as a sensemaking practice in cultural analytics, and assess where large language models can fit into this landscape. We identify ten tasks supported by publicly available datasets on which we empirically assess the performance of LLMs compared to traditional supervised methods, and explore the ways in which LLMs can be employed for sensemaking goals beyond mere accuracy. We find that prompt-based LLMs are competitive with traditional supervised models for established tasks, but perform less well on de novo tasks. In addition, LLMs can assist sensemaking by acting as an intermediary input to formal theory testing. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_12029 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | On Classification with Large Language Models in Cultural Analytics Bamman, David Chang, Kent K. Lucy, Li Zhou, Naitian Computation and Language Computers and Society In this work, we survey the way in which classification is used as a sensemaking practice in cultural analytics, and assess where large language models can fit into this landscape. We identify ten tasks supported by publicly available datasets on which we empirically assess the performance of LLMs compared to traditional supervised methods, and explore the ways in which LLMs can be employed for sensemaking goals beyond mere accuracy. We find that prompt-based LLMs are competitive with traditional supervised models for established tasks, but perform less well on de novo tasks. In addition, LLMs can assist sensemaking by acting as an intermediary input to formal theory testing. |
| title | On Classification with Large Language Models in Cultural Analytics |
| topic | Computation and Language Computers and Society |
| url | https://arxiv.org/abs/2410.12029 |