TIERED HIERARCHICAL MULTI-PATH INTELLIGENCE: COMPLEXITY-GRADED DUAL-SYSTEM REASONING IN TRANSFORMERS
Fuente:
Zenodo
Gespeichert in:
| Hauptverfasser: | , |
|---|---|
| Format: | Recurso digital |
| Sprache: | Englisch |
| Veröffentlicht: |
Zenodo
2026
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866901963286773760 |
|---|---|
| author | Nambiar, Rohan Reddy, P.R.N. Samaikya |
| author_facet | Nambiar, Rohan Reddy, P.R.N. Samaikya |
| contents | <p>Tiered Hierarchical Multi-Path Intelligence (THMI) is a multi-path transformer architecture developed as an architectural analysis instrument to study <strong>complexity-graded dual-system reasoning</strong>. Unlike production reasoning systems, THMI is designed to examine how explicit architectural constraints influence reasoning behavior within a controlled environment. Results are scoped specifically to the evaluated mathematical reasoning benchmarks.</p> <h3><strong>Architectural Design</strong></h3> <p>THMI implements three concurrent processing paths that operate in tier-specific representational spaces via multi-head context projection:</p> <ul> <li> <p><strong>Shallow Path (System 1):</strong> Focuses on fast heuristics (256d).</p> </li> <li> <p><strong>Medium Path (System 2a):</strong> Designed for analytical processing (768d).</p> </li> <li> <p><strong>Deep Path (System 2b):</strong> Designed for deliberate reasoning (768d).</p> </li> <li> <p><strong>Memory Projection:</strong> Utilizes a 512d episodic memory projection for confidence estimation.</p> </li> </ul> <h3><strong>Key Findings and Performance</strong></h3> <p>Evaluated on 2,648 mathematical reasoning tasks (simplified <strong>GSM8K</strong> and <strong>SVAMP</strong>), THMI achieved:</p> <ul> <li> <p><strong>Validation Accuracy:</strong> 98.58%, representing a <strong>+7.52% improvement</strong> over the FLAN-T5 baseline.</p> </li> <li> <p><strong>Complexity-Graded Engagement:</strong> System 2 confidence increases significantly with problem difficulty (from 0.537 for 0-operation problems to 0.564 for 4+ operations), while System 1 confidence remains stable.</p> </li> <li> <p><strong>Functional Diversity:</strong> A high degree of diversity exists between paths (88.0% prediction disagreement), with the ensemble providing an <strong>+8.4% improvement</strong> over the strongest individual path.</p> </li> <li> <p><strong>Training Dynamics:</strong> Episodic memory was found to accelerate early training convergence (+8.95% at epoch 1) without significantly altering final-epoch peak performance.</p> </li> </ul> <h3><strong>Diagnostic and Robustness Analysis</strong></h3> <ul> <li> <p><strong>Lateralized Layer (LL) Diagnostic:</strong> Analysis shows that both lateral attention entropy and energy decay slope increase with problem complexity. This indicates slower lateral representational convergence on more difficult problems, aligning with the expected behavior of System 2 engagement.</p> </li> <li> <p><strong>Cross-Validation:</strong> 5-fold stratified cross-validation confirmed robust performance with a mean accuracy of <strong>98.79% ± 0.39%</strong>, significantly outperforming the FLAN-T5 baseline (92.79% ± 0.68%).</p> </li> <li> <p><strong>Ablation Studies:</strong> To ensure results were not a product of parameter count, THMI was compared against:</p> <ul> <li> <p><strong>Parameter-Matched Baseline (PMB):</strong> 81.10% accuracy (315M parameters).</p> </li> <li> <p><strong>Mixture-of-Depths (MoD):</strong> 79.88% accuracy (223M parameters).</p> </li> <li> <p>Both models were outperformed by THMI (98.58%) under identical frozen-encoder conditions.</p> </li> </ul> </li> </ul> |
| format | Recurso digital |
| id | zenodo_https___doi_org_10_5281_zenodo_19126970 |
| institution | Zenodo |
| language | eng |
| publishDate | 2026 |
| publisher | Zenodo |
| record_format | zenodo |
| spellingShingle | TIERED HIERARCHICAL MULTI-PATH INTELLIGENCE: COMPLEXITY-GRADED DUAL-SYSTEM REASONING IN TRANSFORMERS Nambiar, Rohan Reddy, P.R.N. Samaikya transformer architecture dual-system reasoning multi-path inference mathematical reasoning System 1 System 2 Adapative computation <p>Tiered Hierarchical Multi-Path Intelligence (THMI) is a multi-path transformer architecture developed as an architectural analysis instrument to study <strong>complexity-graded dual-system reasoning</strong>. Unlike production reasoning systems, THMI is designed to examine how explicit architectural constraints influence reasoning behavior within a controlled environment. Results are scoped specifically to the evaluated mathematical reasoning benchmarks.</p> <h3><strong>Architectural Design</strong></h3> <p>THMI implements three concurrent processing paths that operate in tier-specific representational spaces via multi-head context projection:</p> <ul> <li> <p><strong>Shallow Path (System 1):</strong> Focuses on fast heuristics (256d).</p> </li> <li> <p><strong>Medium Path (System 2a):</strong> Designed for analytical processing (768d).</p> </li> <li> <p><strong>Deep Path (System 2b):</strong> Designed for deliberate reasoning (768d).</p> </li> <li> <p><strong>Memory Projection:</strong> Utilizes a 512d episodic memory projection for confidence estimation.</p> </li> </ul> <h3><strong>Key Findings and Performance</strong></h3> <p>Evaluated on 2,648 mathematical reasoning tasks (simplified <strong>GSM8K</strong> and <strong>SVAMP</strong>), THMI achieved:</p> <ul> <li> <p><strong>Validation Accuracy:</strong> 98.58%, representing a <strong>+7.52% improvement</strong> over the FLAN-T5 baseline.</p> </li> <li> <p><strong>Complexity-Graded Engagement:</strong> System 2 confidence increases significantly with problem difficulty (from 0.537 for 0-operation problems to 0.564 for 4+ operations), while System 1 confidence remains stable.</p> </li> <li> <p><strong>Functional Diversity:</strong> A high degree of diversity exists between paths (88.0% prediction disagreement), with the ensemble providing an <strong>+8.4% improvement</strong> over the strongest individual path.</p> </li> <li> <p><strong>Training Dynamics:</strong> Episodic memory was found to accelerate early training convergence (+8.95% at epoch 1) without significantly altering final-epoch peak performance.</p> </li> </ul> <h3><strong>Diagnostic and Robustness Analysis</strong></h3> <ul> <li> <p><strong>Lateralized Layer (LL) Diagnostic:</strong> Analysis shows that both lateral attention entropy and energy decay slope increase with problem complexity. This indicates slower lateral representational convergence on more difficult problems, aligning with the expected behavior of System 2 engagement.</p> </li> <li> <p><strong>Cross-Validation:</strong> 5-fold stratified cross-validation confirmed robust performance with a mean accuracy of <strong>98.79% ± 0.39%</strong>, significantly outperforming the FLAN-T5 baseline (92.79% ± 0.68%).</p> </li> <li> <p><strong>Ablation Studies:</strong> To ensure results were not a product of parameter count, THMI was compared against:</p> <ul> <li> <p><strong>Parameter-Matched Baseline (PMB):</strong> 81.10% accuracy (315M parameters).</p> </li> <li> <p><strong>Mixture-of-Depths (MoD):</strong> 79.88% accuracy (223M parameters).</p> </li> <li> <p>Both models were outperformed by THMI (98.58%) under identical frozen-encoder conditions.</p> </li> </ul> </li> </ul> |
| title | TIERED HIERARCHICAL MULTI-PATH INTELLIGENCE: COMPLEXITY-GRADED DUAL-SYSTEM REASONING IN TRANSFORMERS |
| topic | transformer architecture dual-system reasoning multi-path inference mathematical reasoning System 1 System 2 Adapative computation |
| url | https://doi.org/10.5281/zenodo.19126970 |