Enhancing Large Language Model Efficiencyvia Symbolic Compression: A Formal Approach Towards Interpretability
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866917906940428288 |
|---|---|
| author | AI, Lumen School, Tengzhou No. 1 Middle Ji, Shihao Song, Zihui Zhong, Fucheng Jia, Jisen Wu, Zhaobo Cao, Zheyi Xu, Tianhao |
| author_facet | AI, Lumen School, Tengzhou No. 1 Middle Ji, Shihao Song, Zihui Zhong, Fucheng Jia, Jisen Wu, Zhaobo Cao, Zheyi Xu, Tianhao |
| contents | Large language models (LLMs) face significant token efficiency bottlenecks in code generation and logical reasoning tasks, a challenge that directly impacts inference cost and model interpretability. This paper proposes a formal framework based on symbolic compression,integrating combinatory logic, information-theoretic optimal encoding, and context-aware inference techniques to achieve a step-change improvement in token efficiency while preserving semantic integrity. We establish a mathematical framework within a functional programming paradigm, derive the quantitative relationship between symbolic density and model interpretability, and propose a differentiable compression factor metric to evaluate encoding efficiency. Furthermore, we leverage parameter-efficient fine-tuning (PEFT) techniques to achieve a low-cost application of the GAEL language. Experimental results show that this method achieves a 78.3% token compression rate in code generation tasks while improving logical traceability by 62% through structural explicitness. This research provides new theoretical tools for efficient inference in LLMs and opens a symbolic path for modelinterpretability research. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_18657 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Enhancing Large Language Model Efficiencyvia Symbolic Compression: A Formal Approach Towards Interpretability AI, Lumen School, Tengzhou No. 1 Middle Ji, Shihao Song, Zihui Zhong, Fucheng Jia, Jisen Wu, Zhaobo Cao, Zheyi Xu, Tianhao Artificial Intelligence Software Engineering Large language models (LLMs) face significant token efficiency bottlenecks in code generation and logical reasoning tasks, a challenge that directly impacts inference cost and model interpretability. This paper proposes a formal framework based on symbolic compression,integrating combinatory logic, information-theoretic optimal encoding, and context-aware inference techniques to achieve a step-change improvement in token efficiency while preserving semantic integrity. We establish a mathematical framework within a functional programming paradigm, derive the quantitative relationship between symbolic density and model interpretability, and propose a differentiable compression factor metric to evaluate encoding efficiency. Furthermore, we leverage parameter-efficient fine-tuning (PEFT) techniques to achieve a low-cost application of the GAEL language. Experimental results show that this method achieves a 78.3% token compression rate in code generation tasks while improving logical traceability by 62% through structural explicitness. This research provides new theoretical tools for efficient inference in LLMs and opens a symbolic path for modelinterpretability research. |
| title | Enhancing Large Language Model Efficiencyvia Symbolic Compression: A Formal Approach Towards Interpretability |
| topic | Artificial Intelligence Software Engineering |
| url | https://arxiv.org/abs/2501.18657 |