Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2510.03265 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909918958714880 |
|---|---|
| author | Tian, Bowei He, Yexiao Ye, Wanghao Wang, Ziyao Liu, Meng Li, Ang |
| author_facet | Tian, Bowei He, Yexiao Ye, Wanghao Wang, Ziyao Liu, Meng Li, Ang |
| contents | Large-scale foundation models demonstrate strong performance across language, vision, and reasoning tasks. However, how they internally structure and stabilize concepts remains elusive. Inspired by causal inference, we introduce the MindCraft framework built upon Concept Trees. By applying spectral decomposition at each layer and linking principal directions into branching Concept Paths, Concept Trees reconstruct the hierarchical emergence of concepts, revealing exactly when they diverge from shared representations into linearly separable subspaces. Empirical evaluations across diverse scenarios across disciplines, including medical diagnosis, physics reasoning, and political decision-making, show that Concept Trees recover semantic hierarchies, disentangle latent concepts, and can be widely applied across multiple domains. The Concept Tree establishes a widely applicable and powerful framework that enables in-depth analysis of conceptual representations in deep models, marking a significant step forward in the foundation of interpretable AI. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_03265 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | MindCraft: How Concept Trees Take Shape In Deep Models Tian, Bowei He, Yexiao Ye, Wanghao Wang, Ziyao Liu, Meng Li, Ang Machine Learning Artificial Intelligence Large-scale foundation models demonstrate strong performance across language, vision, and reasoning tasks. However, how they internally structure and stabilize concepts remains elusive. Inspired by causal inference, we introduce the MindCraft framework built upon Concept Trees. By applying spectral decomposition at each layer and linking principal directions into branching Concept Paths, Concept Trees reconstruct the hierarchical emergence of concepts, revealing exactly when they diverge from shared representations into linearly separable subspaces. Empirical evaluations across diverse scenarios across disciplines, including medical diagnosis, physics reasoning, and political decision-making, show that Concept Trees recover semantic hierarchies, disentangle latent concepts, and can be widely applied across multiple domains. The Concept Tree establishes a widely applicable and powerful framework that enables in-depth analysis of conceptual representations in deep models, marking a significant step forward in the foundation of interpretable AI. |
| title | MindCraft: How Concept Trees Take Shape In Deep Models |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2510.03265 |