Salvato in:
| Autori principali: | , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | https://arxiv.org/abs/2406.04926 |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866916279586127872 |
|---|---|
| author | Romaszewski, Michał Sekuła, Przemysław Głomb, Przemysław Cholewa, Michał Kołodziej, Katarzyna |
| author_facet | Romaszewski, Michał Sekuła, Przemysław Głomb, Przemysław Cholewa, Michał Kołodziej, Katarzyna |
| contents | Large Language Models (LLMs) have shown exceptional performance in text processing. Notably, LLMs can synthesize information from large datasets and explain their decisions similarly to human reasoning through a chain of thought (CoT). An emerging application of LLMs is the handling and interpreting of numerical data, where fine-tuning enhances their performance over basic inference methods. This paper proposes a novel approach to training LLMs using knowledge transfer from a random forest (RF) ensemble, leveraging its efficiency and accuracy. By converting RF decision paths into natural language statements, we generate outputs for LLM fine-tuning, enhancing the model's ability to classify and explain its decisions. Our method includes verifying these rules through established classification metrics, ensuring their correctness. We also examine the impact of preprocessing techniques on the representation of numerical data and their influence on classification accuracy and rule correctness |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_04926 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Through the Thicket: A Study of Number-Oriented LLMs derived from Random Forest Models Romaszewski, Michał Sekuła, Przemysław Głomb, Przemysław Cholewa, Michał Kołodziej, Katarzyna Computation and Language Machine Learning Large Language Models (LLMs) have shown exceptional performance in text processing. Notably, LLMs can synthesize information from large datasets and explain their decisions similarly to human reasoning through a chain of thought (CoT). An emerging application of LLMs is the handling and interpreting of numerical data, where fine-tuning enhances their performance over basic inference methods. This paper proposes a novel approach to training LLMs using knowledge transfer from a random forest (RF) ensemble, leveraging its efficiency and accuracy. By converting RF decision paths into natural language statements, we generate outputs for LLM fine-tuning, enhancing the model's ability to classify and explain its decisions. Our method includes verifying these rules through established classification metrics, ensuring their correctness. We also examine the impact of preprocessing techniques on the representation of numerical data and their influence on classification accuracy and rule correctness |
| title | Through the Thicket: A Study of Number-Oriented LLMs derived from Random Forest Models |
| topic | Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2406.04926 |