Profiling Bias in LLMs: Stereotype Dimensions in Contextual Word Embeddings
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915098372603904 |
|---|---|
| author | Schuster, Carolin M. Dinisor, Maria-Alexandra Ghatiwala, Shashwat Groh, Georg |
| author_facet | Schuster, Carolin M. Dinisor, Maria-Alexandra Ghatiwala, Shashwat Groh, Georg |
| contents | Large language models (LLMs) are the foundation of the current successes of artificial intelligence (AI), however, they are unavoidably biased. To effectively communicate the risks and encourage mitigation efforts these models need adequate and intuitive descriptions of their discriminatory properties, appropriate for all audiences of AI. We suggest bias profiles with respect to stereotype dimensions based on dictionaries from social psychology research. Along these dimensions we investigate gender bias in contextual embeddings, across contexts and layers, and generate stereotype profiles for twelve different LLMs, demonstrating their intuition and use case for exposing and visualizing bias. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_16527 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Profiling Bias in LLMs: Stereotype Dimensions in Contextual Word Embeddings Schuster, Carolin M. Dinisor, Maria-Alexandra Ghatiwala, Shashwat Groh, Georg Computation and Language Large language models (LLMs) are the foundation of the current successes of artificial intelligence (AI), however, they are unavoidably biased. To effectively communicate the risks and encourage mitigation efforts these models need adequate and intuitive descriptions of their discriminatory properties, appropriate for all audiences of AI. We suggest bias profiles with respect to stereotype dimensions based on dictionaries from social psychology research. Along these dimensions we investigate gender bias in contextual embeddings, across contexts and layers, and generate stereotype profiles for twelve different LLMs, demonstrating their intuition and use case for exposing and visualizing bias. |
| title | Profiling Bias in LLMs: Stereotype Dimensions in Contextual Word Embeddings |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2411.16527 |