Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | https://arxiv.org/abs/2502.20408 |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866908750377385984 |
|---|---|
| author | Liu, Yiheng Liu, Zhengliang Wu, Zihao Ning, Junhao Sun, Haiyang Xia, Sichen Yang, Yang Gao, Xiaohui Qiang, Ning Ge, Bao Liu, Tianming Han, Junwei Hu, Xintao |
| author_facet | Liu, Yiheng Liu, Zhengliang Wu, Zihao Ning, Junhao Sun, Haiyang Xia, Sichen Yang, Yang Gao, Xiaohui Qiang, Ning Ge, Bao Liu, Tianming Han, Junwei Hu, Xintao |
| contents | In recent years, the rapid advancement of large language models (LLMs) in natural language processing has sparked significant interest among researchers to understand their mechanisms and functional characteristics. Although prior studies have attempted to explain LLM functionalities by identifying and interpreting specific neurons, these efforts mostly focus on individual neuron contributions, neglecting the fact that human brain functions are realized through intricate interaction networks. Inspired by research on functional brain networks (FBNs) in the field of neuroscience, we utilize similar methodologies estabilished in FBN analysis to explore the "functional networks" within LLMs in this study. Experimental results highlight that, much like the human brain, LLMs exhibit certain functional networks that recur frequently during their operation. Further investigation reveals that these functional networks are indispensable for LLM performance. Inhibiting key functional networks severely impairs the model's capabilities. Conversely, amplifying the activity of neurons within these networks can enhance either the model's overall performance or its performance on specific tasks. This suggests that these functional networks are strongly associated with either specific tasks or the overall performance of the LLM. Code is available at https://github.com/WhatAboutMyStar/LLM_ACTIVATION. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_20408 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Brain-Inspired Exploration of Functional Networks and Key Neurons in Large Language Models Liu, Yiheng Liu, Zhengliang Wu, Zihao Ning, Junhao Sun, Haiyang Xia, Sichen Yang, Yang Gao, Xiaohui Qiang, Ning Ge, Bao Liu, Tianming Han, Junwei Hu, Xintao Neurons and Cognition Artificial Intelligence Computation and Language Machine Learning I.2.0 In recent years, the rapid advancement of large language models (LLMs) in natural language processing has sparked significant interest among researchers to understand their mechanisms and functional characteristics. Although prior studies have attempted to explain LLM functionalities by identifying and interpreting specific neurons, these efforts mostly focus on individual neuron contributions, neglecting the fact that human brain functions are realized through intricate interaction networks. Inspired by research on functional brain networks (FBNs) in the field of neuroscience, we utilize similar methodologies estabilished in FBN analysis to explore the "functional networks" within LLMs in this study. Experimental results highlight that, much like the human brain, LLMs exhibit certain functional networks that recur frequently during their operation. Further investigation reveals that these functional networks are indispensable for LLM performance. Inhibiting key functional networks severely impairs the model's capabilities. Conversely, amplifying the activity of neurons within these networks can enhance either the model's overall performance or its performance on specific tasks. This suggests that these functional networks are strongly associated with either specific tasks or the overall performance of the LLM. Code is available at https://github.com/WhatAboutMyStar/LLM_ACTIVATION. |
| title | Brain-Inspired Exploration of Functional Networks and Key Neurons in Large Language Models |
| topic | Neurons and Cognition Artificial Intelligence Computation and Language Machine Learning I.2.0 |
| url | https://arxiv.org/abs/2502.20408 |