Fine-tuning LLaMA 2 interference: a comparative study of language implementations for optimal efficiency
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916595426656256 |
|---|---|
| author | Hossain, Sazzad Seyam, Touhidul Alam Chowdhury, Avijit Xamidov, Munis Ghose, Rajib Pathak, Abhijit |
| author_facet | Hossain, Sazzad Seyam, Touhidul Alam Chowdhury, Avijit Xamidov, Munis Ghose, Rajib Pathak, Abhijit |
| contents | This paper presents a comparative study aimed at optimizing Llama2 inference, a critical aspect of machine learning and natural language processing (NLP). We evaluate various programming languages and frameworks, including TensorFlow, PyTorch, Python, Mojo, C++, and Java, analyzing their performance in terms of speed, memory consumption, and ease of implementation through extensive benchmarking. Strengths and limitations of each approach are highlighted, along with proposed optimization strategies for parallel processing and hardware utilization. Furthermore, we investigate the Mojo SDK, a novel framework designed for large language model (LLM) inference on Apple Silicon, benchmarking its performance against implementations in C, C++, Rust, Zig, Go, and Julia. Our experiments, conducted on an Apple M1 Max, demonstrate Mojo SDK's competitive performance, ease of use, and seamless Python compatibility, positioning it as a strong alternative for LLM inference on Apple Silicon. We also discuss broader implications for LLM deployment on resource-constrained hardware and identify potential directions for future research. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_01651 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Fine-tuning LLaMA 2 interference: a comparative study of language implementations for optimal efficiency Hossain, Sazzad Seyam, Touhidul Alam Chowdhury, Avijit Xamidov, Munis Ghose, Rajib Pathak, Abhijit Machine Learning Artificial Intelligence This paper presents a comparative study aimed at optimizing Llama2 inference, a critical aspect of machine learning and natural language processing (NLP). We evaluate various programming languages and frameworks, including TensorFlow, PyTorch, Python, Mojo, C++, and Java, analyzing their performance in terms of speed, memory consumption, and ease of implementation through extensive benchmarking. Strengths and limitations of each approach are highlighted, along with proposed optimization strategies for parallel processing and hardware utilization. Furthermore, we investigate the Mojo SDK, a novel framework designed for large language model (LLM) inference on Apple Silicon, benchmarking its performance against implementations in C, C++, Rust, Zig, Go, and Julia. Our experiments, conducted on an Apple M1 Max, demonstrate Mojo SDK's competitive performance, ease of use, and seamless Python compatibility, positioning it as a strong alternative for LLM inference on Apple Silicon. We also discuss broader implications for LLM deployment on resource-constrained hardware and identify potential directions for future research. |
| title | Fine-tuning LLaMA 2 interference: a comparative study of language implementations for optimal efficiency |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2502.01651 |