Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2402.09099 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912521718333440 |
|---|---|
| author | Xiao, Xiongye Ping, Heng Zhou, Chenyu Cao, Defu Li, Yaxing Zhou, Yi-Zhuo Li, Shixuan Kanakaris, Nikos Bogdan, Paul |
| author_facet | Xiao, Xiongye Ping, Heng Zhou, Chenyu Cao, Defu Li, Yaxing Zhou, Yi-Zhuo Li, Shixuan Kanakaris, Nikos Bogdan, Paul |
| contents | In recent years, there has been increasing attention on the capabilities of large models, particularly in handling complex tasks that small-scale models are unable to perform. Notably, large language models (LLMs) have demonstrated ``intelligent'' abilities such as complex reasoning and abstract language comprehension, reflecting cognitive-like behaviors. However, current research on emergent abilities in large models predominantly focuses on the relationship between model performance and size, leaving a significant gap in the systematic quantitative analysis of the internal structures and mechanisms driving these emergent abilities. Drawing inspiration from neuroscience research on brain network structure and self-organization, we propose (i) a general network representation of large models, (ii) a new analytical framework, called Neuron-based Multifractal Analysis (NeuroMFA), for structural analysis, and (iii) a novel structure-based metric as a proxy for emergent abilities of large models. By linking structural features to the capabilities of large models, NeuroMFA provides a quantitative framework for analyzing emergent phenomena in large models. Our experiments show that the proposed method yields a comprehensive measure of network's evolving heterogeneity and organization, offering theoretical foundations and a new perspective for investigating emergent abilities in large models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_09099 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Neuron-based Multifractal Analysis of Neuron Interaction Dynamics in Large Models Xiao, Xiongye Ping, Heng Zhou, Chenyu Cao, Defu Li, Yaxing Zhou, Yi-Zhuo Li, Shixuan Kanakaris, Nikos Bogdan, Paul Artificial Intelligence In recent years, there has been increasing attention on the capabilities of large models, particularly in handling complex tasks that small-scale models are unable to perform. Notably, large language models (LLMs) have demonstrated ``intelligent'' abilities such as complex reasoning and abstract language comprehension, reflecting cognitive-like behaviors. However, current research on emergent abilities in large models predominantly focuses on the relationship between model performance and size, leaving a significant gap in the systematic quantitative analysis of the internal structures and mechanisms driving these emergent abilities. Drawing inspiration from neuroscience research on brain network structure and self-organization, we propose (i) a general network representation of large models, (ii) a new analytical framework, called Neuron-based Multifractal Analysis (NeuroMFA), for structural analysis, and (iii) a novel structure-based metric as a proxy for emergent abilities of large models. By linking structural features to the capabilities of large models, NeuroMFA provides a quantitative framework for analyzing emergent phenomena in large models. Our experiments show that the proposed method yields a comprehensive measure of network's evolving heterogeneity and organization, offering theoretical foundations and a new perspective for investigating emergent abilities in large models. |
| title | Neuron-based Multifractal Analysis of Neuron Interaction Dynamics in Large Models |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2402.09099 |