Adaptive Two-Phase Finetuning LLMs for Japanese Legal Text Retrieval
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866912160169328640 |
|---|---|
| author | Trung, Quang Hoang Phuc, Nguyen Van Hoang Hoang, Le Trung Hieu, Quang Huu Duy, Vo Nguyen Le |
| author_facet | Trung, Quang Hoang Phuc, Nguyen Van Hoang Hoang, Le Trung Hieu, Quang Huu Duy, Vo Nguyen Le |
| contents | Text Retrieval (TR) involves finding and retrieving text-based content relevant to a user's query from a large repository, with applications in real-world scenarios such as legal document retrieval. While most existing studies focus on English, limited work addresses Japanese contexts. In this paper, we introduce a new dataset specifically designed for Japanese legal contexts and propose a novel two-phase pipeline tailored to this domain.
In the first phase, the model learns a broad understanding of global contexts, enhancing its generalization and adaptability to diverse queries. In the second phase, the model is fine-tuned to address complex queries specific to legal scenarios. Extensive experiments are conducted to demonstrate the superior performance of our method, which outperforms existing baselines.
Furthermore, our pipeline proves effective in English contexts, surpassing comparable baselines on the MS MARCO dataset. We have made our code publicly available on GitHub, and the model checkpoints are accessible via HuggingFace. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_13205 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Adaptive Two-Phase Finetuning LLMs for Japanese Legal Text Retrieval Trung, Quang Hoang Phuc, Nguyen Van Hoang Hoang, Le Trung Hieu, Quang Huu Duy, Vo Nguyen Le Information Retrieval Computation and Language Machine Learning Text Retrieval (TR) involves finding and retrieving text-based content relevant to a user's query from a large repository, with applications in real-world scenarios such as legal document retrieval. While most existing studies focus on English, limited work addresses Japanese contexts. In this paper, we introduce a new dataset specifically designed for Japanese legal contexts and propose a novel two-phase pipeline tailored to this domain. In the first phase, the model learns a broad understanding of global contexts, enhancing its generalization and adaptability to diverse queries. In the second phase, the model is fine-tuned to address complex queries specific to legal scenarios. Extensive experiments are conducted to demonstrate the superior performance of our method, which outperforms existing baselines. Furthermore, our pipeline proves effective in English contexts, surpassing comparable baselines on the MS MARCO dataset. We have made our code publicly available on GitHub, and the model checkpoints are accessible via HuggingFace. |
| title | Adaptive Two-Phase Finetuning LLMs for Japanese Legal Text Retrieval |
| topic | Information Retrieval Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2412.13205 |