Can Low-Rank Knowledge Distillation in LLMs be Useful for Microelectronic Reasoning?
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910503663566848 |
|---|---|
| author | Rouf, Nirjhor Amin, Fin Franzon, Paul D. |
| author_facet | Rouf, Nirjhor Amin, Fin Franzon, Paul D. |
| contents | In this work, we present empirical results regarding the feasibility of using offline large language models (LLMs) in the context of electronic design automation (EDA). The goal is to investigate and evaluate a contemporary language model's (Llama-2-7B) ability to function as a microelectronic Q & A expert as well as its reasoning, and generation capabilities in solving microelectronic-related problems. Llama-2-7B was tested across a variety of adaptation methods, including introducing a novel low-rank knowledge distillation (LoRA-KD) scheme. Our experiments produce both qualitative and quantitative results. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_13808 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Can Low-Rank Knowledge Distillation in LLMs be Useful for Microelectronic Reasoning? Rouf, Nirjhor Amin, Fin Franzon, Paul D. Machine Learning In this work, we present empirical results regarding the feasibility of using offline large language models (LLMs) in the context of electronic design automation (EDA). The goal is to investigate and evaluate a contemporary language model's (Llama-2-7B) ability to function as a microelectronic Q & A expert as well as its reasoning, and generation capabilities in solving microelectronic-related problems. Llama-2-7B was tested across a variety of adaptation methods, including introducing a novel low-rank knowledge distillation (LoRA-KD) scheme. Our experiments produce both qualitative and quantitative results. |
| title | Can Low-Rank Knowledge Distillation in LLMs be Useful for Microelectronic Reasoning? |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2406.13808 |