Can Large Language Models Code Like a Linguist?: A Case Study in Low Resource Sound Law Induction
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917974551560192 |
|---|---|
| author | Naik, Atharva Zhang, Kexun Robinson, Nathaniel Mysore, Aravind Marr, Clayton Sng, Hong Byrnes, Rebecca Cai, Anna Chang, Kalvin Mortensen, David |
| author_facet | Naik, Atharva Zhang, Kexun Robinson, Nathaniel Mysore, Aravind Marr, Clayton Sng, Hong Byrnes, Rebecca Cai, Anna Chang, Kalvin Mortensen, David |
| contents | Historical linguists have long written a kind of incompletely formalized ''program'' that converts reconstructed words in an ancestor language into words in one of its attested descendants that consist of a series of ordered string rewrite functions (called sound laws). They do this by observing pairs of words in the reconstructed language (protoforms) and the descendent language (reflexes) and constructing a program that transforms protoforms into reflexes. However, writing these programs is error-prone and time-consuming. Prior work has successfully scaffolded this process computationally, but fewer researchers have tackled Sound Law Induction (SLI), which we approach in this paper by casting it as Programming by Examples. We propose a language-agnostic solution that utilizes the programming ability of Large Language Models (LLMs) by generating Python sound law programs from sound change examples. We evaluate the effectiveness of our approach for various LLMs, propose effective methods to generate additional language-agnostic synthetic data to fine-tune LLMs for SLI, and compare our method with existing automated SLI methods showing that while LLMs lag behind them they can complement some of their weaknesses. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_12725 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Can Large Language Models Code Like a Linguist?: A Case Study in Low Resource Sound Law Induction Naik, Atharva Zhang, Kexun Robinson, Nathaniel Mysore, Aravind Marr, Clayton Sng, Hong Byrnes, Rebecca Cai, Anna Chang, Kalvin Mortensen, David Computation and Language Artificial Intelligence Historical linguists have long written a kind of incompletely formalized ''program'' that converts reconstructed words in an ancestor language into words in one of its attested descendants that consist of a series of ordered string rewrite functions (called sound laws). They do this by observing pairs of words in the reconstructed language (protoforms) and the descendent language (reflexes) and constructing a program that transforms protoforms into reflexes. However, writing these programs is error-prone and time-consuming. Prior work has successfully scaffolded this process computationally, but fewer researchers have tackled Sound Law Induction (SLI), which we approach in this paper by casting it as Programming by Examples. We propose a language-agnostic solution that utilizes the programming ability of Large Language Models (LLMs) by generating Python sound law programs from sound change examples. We evaluate the effectiveness of our approach for various LLMs, propose effective methods to generate additional language-agnostic synthetic data to fine-tune LLMs for SLI, and compare our method with existing automated SLI methods showing that while LLMs lag behind them they can complement some of their weaknesses. |
| title | Can Large Language Models Code Like a Linguist?: A Case Study in Low Resource Sound Law Induction |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2406.12725 |