Do LLMs Adhere to Label Definitions? Examining Their Receptivity to External Label Definitions
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866910032250011648 |
|---|---|
| author | Mohammadi, Seyedali Vedula, Bhaskara Hanuma Lamba, Hemank Raff, Edward Kumaraguru, Ponnurangam Ferraro, Francis Gaur, Manas |
| author_facet | Mohammadi, Seyedali Vedula, Bhaskara Hanuma Lamba, Hemank Raff, Edward Kumaraguru, Ponnurangam Ferraro, Francis Gaur, Manas |
| contents | Do LLMs genuinely incorporate external definitions, or do they primarily rely on their parametric knowledge? To address these questions, we conduct controlled experiments across multiple explanation benchmark datasets (general and domain-specific) and label definition conditions, including expert-curated, LLM-generated, perturbed, and swapped definitions. Our results reveal that while explicit label definitions can enhance accuracy and explainability, their integration into an LLM's task-solving processes is neither guaranteed nor consistent, suggesting reliance on internalized representations in many cases. Models often default to their internal representations, particularly in general tasks, whereas domain-specific tasks benefit more from explicit definitions. These findings underscore the need for a deeper understanding of how LLMs process external knowledge alongside their pre-existing capabilities. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_02452 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Do LLMs Adhere to Label Definitions? Examining Their Receptivity to External Label Definitions Mohammadi, Seyedali Vedula, Bhaskara Hanuma Lamba, Hemank Raff, Edward Kumaraguru, Ponnurangam Ferraro, Francis Gaur, Manas Computation and Language Artificial Intelligence Machine Learning Do LLMs genuinely incorporate external definitions, or do they primarily rely on their parametric knowledge? To address these questions, we conduct controlled experiments across multiple explanation benchmark datasets (general and domain-specific) and label definition conditions, including expert-curated, LLM-generated, perturbed, and swapped definitions. Our results reveal that while explicit label definitions can enhance accuracy and explainability, their integration into an LLM's task-solving processes is neither guaranteed nor consistent, suggesting reliance on internalized representations in many cases. Models often default to their internal representations, particularly in general tasks, whereas domain-specific tasks benefit more from explicit definitions. These findings underscore the need for a deeper understanding of how LLMs process external knowledge alongside their pre-existing capabilities. |
| title | Do LLMs Adhere to Label Definitions? Examining Their Receptivity to External Label Definitions |
| topic | Computation and Language Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2509.02452 |