Leveraging Large Language Models for Concept Graph Recovery and Question Answering in NLP Education
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866914689103953920 |
|---|---|
| author | Yang, Rui Yang, Boming Ouyang, Sixun She, Tianwei Feng, Aosong Jiang, Yuang Lecue, Freddy Lu, Jinghui Li, Irene |
| author_facet | Yang, Rui Yang, Boming Ouyang, Sixun She, Tianwei Feng, Aosong Jiang, Yuang Lecue, Freddy Lu, Jinghui Li, Irene |
| contents | In the domain of Natural Language Processing (NLP), Large Language Models (LLMs) have demonstrated promise in text-generation tasks. However, their educational applications, particularly for domain-specific queries, remain underexplored. This study investigates LLMs' capabilities in educational scenarios, focusing on concept graph recovery and question-answering (QA). We assess LLMs' zero-shot performance in creating domain-specific concept graphs and introduce TutorQA, a new expert-verified NLP-focused benchmark for scientific graph reasoning and QA. TutorQA consists of five tasks with 500 QA pairs. To tackle TutorQA queries, we present CGLLM, a pipeline integrating concept graphs with LLMs for answering diverse questions. Our results indicate that LLMs' zero-shot concept graph recovery is competitive with supervised methods, showing an average 3% F1 score improvement. In TutorQA tasks, LLMs achieve up to 26% F1 score enhancement. Moreover, human evaluation and analysis show that CGLLM generates answers with more fine-grained concepts. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_14293 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Leveraging Large Language Models for Concept Graph Recovery and Question Answering in NLP Education Yang, Rui Yang, Boming Ouyang, Sixun She, Tianwei Feng, Aosong Jiang, Yuang Lecue, Freddy Lu, Jinghui Li, Irene Computation and Language In the domain of Natural Language Processing (NLP), Large Language Models (LLMs) have demonstrated promise in text-generation tasks. However, their educational applications, particularly for domain-specific queries, remain underexplored. This study investigates LLMs' capabilities in educational scenarios, focusing on concept graph recovery and question-answering (QA). We assess LLMs' zero-shot performance in creating domain-specific concept graphs and introduce TutorQA, a new expert-verified NLP-focused benchmark for scientific graph reasoning and QA. TutorQA consists of five tasks with 500 QA pairs. To tackle TutorQA queries, we present CGLLM, a pipeline integrating concept graphs with LLMs for answering diverse questions. Our results indicate that LLMs' zero-shot concept graph recovery is competitive with supervised methods, showing an average 3% F1 score improvement. In TutorQA tasks, LLMs achieve up to 26% F1 score enhancement. Moreover, human evaluation and analysis show that CGLLM generates answers with more fine-grained concepts. |
| title | Leveraging Large Language Models for Concept Graph Recovery and Question Answering in NLP Education |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2402.14293 |