Atomic Inference for NLI with Generated Facts as Atoms
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913525466660864 |
|---|---|
| author | Stacey, Joe Minervini, Pasquale Dubossarsky, Haim Camburu, Oana-Maria Rei, Marek |
| author_facet | Stacey, Joe Minervini, Pasquale Dubossarsky, Haim Camburu, Oana-Maria Rei, Marek |
| contents | With recent advances, neural models can achieve human-level performance on various natural language tasks. However, there are no guarantees that any explanations from these models are faithful, i.e. that they reflect the inner workings of the model. Atomic inference overcomes this issue, providing interpretable and faithful model decisions. This approach involves making predictions for different components (or atoms) of an instance, before using interpretable and deterministic rules to derive the overall prediction based on the individual atom-level predictions. We investigate the effectiveness of using LLM-generated facts as atoms, decomposing Natural Language Inference premises into lists of facts. While directly using generated facts in atomic inference systems can result in worse performance, with 1) a multi-stage fact generation process, and 2) a training regime that incorporates the facts, our fact-based method outperforms other approaches. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2305_13214 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Atomic Inference for NLI with Generated Facts as Atoms Stacey, Joe Minervini, Pasquale Dubossarsky, Haim Camburu, Oana-Maria Rei, Marek Computation and Language I.2.7 With recent advances, neural models can achieve human-level performance on various natural language tasks. However, there are no guarantees that any explanations from these models are faithful, i.e. that they reflect the inner workings of the model. Atomic inference overcomes this issue, providing interpretable and faithful model decisions. This approach involves making predictions for different components (or atoms) of an instance, before using interpretable and deterministic rules to derive the overall prediction based on the individual atom-level predictions. We investigate the effectiveness of using LLM-generated facts as atoms, decomposing Natural Language Inference premises into lists of facts. While directly using generated facts in atomic inference systems can result in worse performance, with 1) a multi-stage fact generation process, and 2) a training regime that incorporates the facts, our fact-based method outperforms other approaches. |
| title | Atomic Inference for NLI with Generated Facts as Atoms |
| topic | Computation and Language I.2.7 |
| url | https://arxiv.org/abs/2305.13214 |