Analysis of Error Sources in LLM-based Hypothesis Search for Few-Shot Rule Induction
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866912563306954752 |
|---|---|
| author | Parab, Aishni Lu, Hongjing Wu, Ying Nian Gulwani, Sumit |
| author_facet | Parab, Aishni Lu, Hongjing Wu, Ying Nian Gulwani, Sumit |
| contents | Inductive reasoning enables humans to infer abstract rules from limited examples and apply them to novel situations. In this work, we compare an LLM-based hypothesis search framework with direct program generation approaches on few-shot rule induction tasks. Our findings show that hypothesis search achieves performance comparable to humans, while direct program generation falls notably behind. An error analysis reveals key bottlenecks in hypothesis generation and suggests directions for advancing program induction methods. Overall, this paper underscores the potential of LLM-based hypothesis search for modeling inductive reasoning and the challenges in building more efficient systems. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_01016 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Analysis of Error Sources in LLM-based Hypothesis Search for Few-Shot Rule Induction Parab, Aishni Lu, Hongjing Wu, Ying Nian Gulwani, Sumit Artificial Intelligence Computation and Language Machine Learning Neural and Evolutionary Computing Inductive reasoning enables humans to infer abstract rules from limited examples and apply them to novel situations. In this work, we compare an LLM-based hypothesis search framework with direct program generation approaches on few-shot rule induction tasks. Our findings show that hypothesis search achieves performance comparable to humans, while direct program generation falls notably behind. An error analysis reveals key bottlenecks in hypothesis generation and suggests directions for advancing program induction methods. Overall, this paper underscores the potential of LLM-based hypothesis search for modeling inductive reasoning and the challenges in building more efficient systems. |
| title | Analysis of Error Sources in LLM-based Hypothesis Search for Few-Shot Rule Induction |
| topic | Artificial Intelligence Computation and Language Machine Learning Neural and Evolutionary Computing |
| url | https://arxiv.org/abs/2509.01016 |