The Sociolinguistic Foundations of Language Modeling
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866914867725729792 |
|---|---|
| author | Grieve, Jack Bartl, Sara Fuoli, Matteo Grafmiller, Jason Huang, Weihang Jawerbaum, Alejandro Murakami, Akira Perlman, Marcus Roemling, Dana Winter, Bodo |
| author_facet | Grieve, Jack Bartl, Sara Fuoli, Matteo Grafmiller, Jason Huang, Weihang Jawerbaum, Alejandro Murakami, Akira Perlman, Marcus Roemling, Dana Winter, Bodo |
| contents | In this paper, we introduce a sociolinguistic perspective on language modeling. We claim that large language models are inherently models of varieties of language, and we consider how this insight can inform the development and deployment of large language models. We begin by presenting a technical definition of the concept of a variety of language as developed in sociolinguistics. We then discuss how this perspective can help address five basic challenges in language modeling: social bias, domain adaptation, alignment, language change, and scale. Ultimately, we argue that it is crucial to carefully define and compile training corpora that accurately represent the specific varieties of language being modeled to maximize the performance and societal value of large language models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_09241 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | The Sociolinguistic Foundations of Language Modeling Grieve, Jack Bartl, Sara Fuoli, Matteo Grafmiller, Jason Huang, Weihang Jawerbaum, Alejandro Murakami, Akira Perlman, Marcus Roemling, Dana Winter, Bodo Computation and Language In this paper, we introduce a sociolinguistic perspective on language modeling. We claim that large language models are inherently models of varieties of language, and we consider how this insight can inform the development and deployment of large language models. We begin by presenting a technical definition of the concept of a variety of language as developed in sociolinguistics. We then discuss how this perspective can help address five basic challenges in language modeling: social bias, domain adaptation, alignment, language change, and scale. Ultimately, we argue that it is crucial to carefully define and compile training corpora that accurately represent the specific varieties of language being modeled to maximize the performance and societal value of large language models. |
| title | The Sociolinguistic Foundations of Language Modeling |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2407.09241 |