Reactor Mk.1 performances: MMLU, HumanEval and BBH test results
Fuente:
arXiv
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929437924130816 |
|---|---|
| author | Dunham, TJ Syahputra, Henry |
| author_facet | Dunham, TJ Syahputra, Henry |
| contents | The paper presents the performance results of Reactor Mk.1, ARCs flagship large language model, through a benchmarking process analysis. The model utilizes the Lychee AI engine and possesses less than 100 billion parameters, resulting in a combination of efficiency and potency. The Reactor Mk.1 outperformed models such as GPT-4o, Claude Opus, and Llama 3, with achieved scores of 92% on the MMLU dataset, 91% on HumanEval dataset, and 88% on BBH dataset. It excels in both managing difficult jobs and reasoning, establishing as a prominent AI solution in the present cutting-edge AI technology. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_10515 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Reactor Mk.1 performances: MMLU, HumanEval and BBH test results Dunham, TJ Syahputra, Henry Artificial Intelligence Computation and Language The paper presents the performance results of Reactor Mk.1, ARCs flagship large language model, through a benchmarking process analysis. The model utilizes the Lychee AI engine and possesses less than 100 billion parameters, resulting in a combination of efficiency and potency. The Reactor Mk.1 outperformed models such as GPT-4o, Claude Opus, and Llama 3, with achieved scores of 92% on the MMLU dataset, 91% on HumanEval dataset, and 88% on BBH dataset. It excels in both managing difficult jobs and reasoning, establishing as a prominent AI solution in the present cutting-edge AI technology. |
| title | Reactor Mk.1 performances: MMLU, HumanEval and BBH test results |
| topic | Artificial Intelligence Computation and Language |
| url | https://arxiv.org/abs/2406.10515 |