Inference in Normal Form: Unifying LLM Tricks via TRoT
Fuente:
Zenodo
Salvato in:
| Autore principale: | |
|---|---|
| Natura: | Recurso digital |
| Lingua: | inglese |
| Pubblicazione: |
Zenodo
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866901962566402048 |
|---|---|
| author | Takahashi, K |
| author_facet | Takahashi, K |
| contents | <p>This record contains the camera-ready paper <strong>“LLM Inference in Normal Form: A TRoT-Based Unification.”</strong><br>We show that many <em>inference-time</em> methods for large language models (LLMs)—e.g., <strong>MBR/eMBR decoding, Conformal prediction gates, Verifier/Judge ensembles, Self-RAG, Chain/Tree/Graph-of-Thought, and Verifier-Sandwich (VS)</strong>—admit a single <strong>normal form</strong> derived from the <strong>Theory of Relativity of Theories (TRoT)</strong>.</p> <p>Our formulation uses <strong>enriched category theory</strong> over the <strong>Lawvere cost quantale</strong> (tropical <em>(min,+)</em> semiring): left/right <strong>Kan extensions</strong> <span><span>\Lan/\Ran\Lan/\Ran</span><span><span><span><span>\Lan</span></span><span>/</span><span><span>\Ran</span></span></span></span></span>, <strong>residuation</strong> (elementwise residuals), <strong>masking</strong> and <strong>nuclei</strong> (1-Lipschitz projectors). In this normal form, generation/lifts (K3), safety consolidation (K4), and auditing (K6–K10 <strong>RAVE</strong>) become <strong>GraphBLAS-style sparse operators</strong> with explicit stability and auditability contracts.</p> <p><strong>Provable guarantees</strong></p> <ul> <li> <p><strong>Implementation equivalence (normal form):</strong> common decoding “tricks” map to <span><span>\Lan\Lan</span><span><span><span><span>\Lan</span></span></span></span></span> → Obs → <span><span>\Ran\Ran</span><span><span><span><span>\Ran</span></span></span></span></span> with the same outputs under masking/nucleus conditions.</p> </li> <li> <p><strong>Stability:</strong> <span><span>\Lan\Lan</span><span><span><span><span>\Lan</span></span></span></span></span> and the residual are <strong>1-Lipschitz</strong> in <span><span>ℓ∞\ell_\infty</span><span><span><span>ℓ<span><span><span><span><span><span>∞</span></span></span><span></span></span></span></span></span></span></span></span>.</p> </li> <li> <p><strong>Approximation bounds:</strong> geometric truncation error <span><span>q\*k+11−q\*\frac{q_\*^{k+1}}{1-q_\*}</span><span><span><span><span><span><span><span><span><span>1<span>−</span><span>q</span><span><span><span>\*</span></span><span></span></span></span></span><span><span><span>q</span><span><span><span>\*</span></span><span><span>k</span><span>+</span>1</span><span></span></span></span></span></span><span></span></span></span></span></span></span></span></span>; soft-min (log-sum-exp) gap <span><span>k logdmaxλcost\frac{k\,\log d_{\max}}{\lambda_{\text{cost}}}</span><span><span><span><span><span><span><span><span><span><span>λ</span><span><span><span>cost</span></span><span></span></span></span></span><span><span><span>k</span><span><span>l</span><span>o</span><span>g</span></span><span>d</span><span><span><span><span>m</span><span>a</span><span>x</span></span></span><span></span></span></span></span></span><span></span></span></span></span></span></span></span></span> (with <strong>join ≡ numeric infimum</strong> in Cost polarity).</p> </li> <li> <p><strong>Auditing:</strong> a <strong>deterministic spending</strong> schedule <span><span>∑tαt≤αglobal\sum_t\alpha_t\le\alpha_{\text{global}}</span><span><span><span><span>∑</span><span><span><span><span><span><span>t</span></span></span><span></span></span></span></span></span><span><span>α</span><span><span><span><span><span><span>t</span></span></span><span></span></span></span></span></span><span>≤</span></span><span><span><span>α</span><span><span><span><span><span><span><span>global</span></span></span></span><span></span></span></span></span></span></span></span></span> with test-(super)martingale <strong>e-processes</strong> gives time-uniform <strong>FWER control</strong>; LR/mixture-LR constructions yield valid e-values.</p> </li> </ul> <p><strong>Implementation blueprint (GPU/GraphBLAS-ready)</strong></p> <ul> <li> <p><span><span>\Lan\Lan</span><span><span><span><span>\Lan</span></span></span></span></span>: sparse <strong>SpMV/SpGEMM</strong> on <em>(min,+)</em>;</p> </li> <li> <p><span><span>\Ran\Ran</span><span><span><span><span>\Ran</span></span></span></span></span>: <strong>elementwise residual → max-reduce</strong>;</p> </li> <li> <p>Numerics: <strong>log-domain + row-wise max-shift</strong>; masks send forbidden entries to <span><span>+∞+\infty</span><span><span><span>+</span><span>∞</span></span></span></span>.</p> </li> <li> <p><strong>Reproducibility log (minimum fields):</strong> seed; model/revision; tokenizer; prompts; <strong>forward map <span><span>JJ</span><span><span><span>J</span></span></span></span></strong>; <strong>transport <span><span>KK</span><span><span><span>K</span></span></span></span></strong>; masks; <span><span>λcost\lambda_{\text{cost}}</span><span><span><span><span>λ</span><span><span><span><span><span><span><span>cost</span></span></span></span><span></span></span></span></span></span></span></span></span>; <span><span>kk</span><span><span><span>k</span></span></span></span>; effective <span><span>dmaxd_{\max}</span><span><span><span><span>d</span><span><span><span><span><span><span><span><span>m</span><span>a</span><span>x</span></span></span></span></span><span></span></span></span></span></span></span></span></span>; <span><span>\Lan/\Ran\Lan/\Ran</span><span><span><span><span>\Lan</span></span><span>/</span><span><span>\Ran</span></span></span></span></span> kernel + build flags; hardware/BLAS; e-process updates <span><span>(et,Mt)(e_t,M_t)</span><span><span><span>(</span><span><span>e</span><span><span><span><span><span><span>t</span></span></span><span></span></span></span></span></span><span>,</span><span><span>M</span><span><span><span><span><span><span>t</span></span></span><span></span></span></span></span></span><span>)</span></span></span></span>; Conformal calibration snapshots.</p> </li> </ul> |
| format | Recurso digital |
| id | zenodo_https___doi_org_10_5281_zenodo_17389109 |
| institution | Zenodo |
| language | eng |
| publishDate | 2025 |
| publisher | Zenodo |
| record_format | zenodo |
| spellingShingle | Inference in Normal Form: Unifying LLM Tricks via TRoT Takahashi, K LLM Large Language Models Artificial intelligence LLM inference Machine learning decoding MBR conformal prediction verifier RAG Chain-of-Thought Tree-of-Thought Graph-of-Thought Theory of Relativity of Theories Verifier-Sandwich category theory Lawvere metric quantale tropical algebra GraphBLAS Kan extension e-process Algorithms <p>This record contains the camera-ready paper <strong>“LLM Inference in Normal Form: A TRoT-Based Unification.”</strong><br>We show that many <em>inference-time</em> methods for large language models (LLMs)—e.g., <strong>MBR/eMBR decoding, Conformal prediction gates, Verifier/Judge ensembles, Self-RAG, Chain/Tree/Graph-of-Thought, and Verifier-Sandwich (VS)</strong>—admit a single <strong>normal form</strong> derived from the <strong>Theory of Relativity of Theories (TRoT)</strong>.</p> <p>Our formulation uses <strong>enriched category theory</strong> over the <strong>Lawvere cost quantale</strong> (tropical <em>(min,+)</em> semiring): left/right <strong>Kan extensions</strong> <span><span>\Lan/\Ran\Lan/\Ran</span><span><span><span><span>\Lan</span></span><span>/</span><span><span>\Ran</span></span></span></span></span>, <strong>residuation</strong> (elementwise residuals), <strong>masking</strong> and <strong>nuclei</strong> (1-Lipschitz projectors). In this normal form, generation/lifts (K3), safety consolidation (K4), and auditing (K6–K10 <strong>RAVE</strong>) become <strong>GraphBLAS-style sparse operators</strong> with explicit stability and auditability contracts.</p> <p><strong>Provable guarantees</strong></p> <ul> <li> <p><strong>Implementation equivalence (normal form):</strong> common decoding “tricks” map to <span><span>\Lan\Lan</span><span><span><span><span>\Lan</span></span></span></span></span> → Obs → <span><span>\Ran\Ran</span><span><span><span><span>\Ran</span></span></span></span></span> with the same outputs under masking/nucleus conditions.</p> </li> <li> <p><strong>Stability:</strong> <span><span>\Lan\Lan</span><span><span><span><span>\Lan</span></span></span></span></span> and the residual are <strong>1-Lipschitz</strong> in <span><span>ℓ∞\ell_\infty</span><span><span><span>ℓ<span><span><span><span><span><span>∞</span></span></span><span></span></span></span></span></span></span></span></span>.</p> </li> <li> <p><strong>Approximation bounds:</strong> geometric truncation error <span><span>q\*k+11−q\*\frac{q_\*^{k+1}}{1-q_\*}</span><span><span><span><span><span><span><span><span><span>1<span>−</span><span>q</span><span><span><span>\*</span></span><span></span></span></span></span><span><span><span>q</span><span><span><span>\*</span></span><span><span>k</span><span>+</span>1</span><span></span></span></span></span></span><span></span></span></span></span></span></span></span></span>; soft-min (log-sum-exp) gap <span><span>k logdmaxλcost\frac{k\,\log d_{\max}}{\lambda_{\text{cost}}}</span><span><span><span><span><span><span><span><span><span><span>λ</span><span><span><span>cost</span></span><span></span></span></span></span><span><span><span>k</span><span><span>l</span><span>o</span><span>g</span></span><span>d</span><span><span><span><span>m</span><span>a</span><span>x</span></span></span><span></span></span></span></span></span><span></span></span></span></span></span></span></span></span> (with <strong>join ≡ numeric infimum</strong> in Cost polarity).</p> </li> <li> <p><strong>Auditing:</strong> a <strong>deterministic spending</strong> schedule <span><span>∑tαt≤αglobal\sum_t\alpha_t\le\alpha_{\text{global}}</span><span><span><span><span>∑</span><span><span><span><span><span><span>t</span></span></span><span></span></span></span></span></span><span><span>α</span><span><span><span><span><span><span>t</span></span></span><span></span></span></span></span></span><span>≤</span></span><span><span><span>α</span><span><span><span><span><span><span><span>global</span></span></span></span><span></span></span></span></span></span></span></span></span> with test-(super)martingale <strong>e-processes</strong> gives time-uniform <strong>FWER control</strong>; LR/mixture-LR constructions yield valid e-values.</p> </li> </ul> <p><strong>Implementation blueprint (GPU/GraphBLAS-ready)</strong></p> <ul> <li> <p><span><span>\Lan\Lan</span><span><span><span><span>\Lan</span></span></span></span></span>: sparse <strong>SpMV/SpGEMM</strong> on <em>(min,+)</em>;</p> </li> <li> <p><span><span>\Ran\Ran</span><span><span><span><span>\Ran</span></span></span></span></span>: <strong>elementwise residual → max-reduce</strong>;</p> </li> <li> <p>Numerics: <strong>log-domain + row-wise max-shift</strong>; masks send forbidden entries to <span><span>+∞+\infty</span><span><span><span>+</span><span>∞</span></span></span></span>.</p> </li> <li> <p><strong>Reproducibility log (minimum fields):</strong> seed; model/revision; tokenizer; prompts; <strong>forward map <span><span>JJ</span><span><span><span>J</span></span></span></span></strong>; <strong>transport <span><span>KK</span><span><span><span>K</span></span></span></span></strong>; masks; <span><span>λcost\lambda_{\text{cost}}</span><span><span><span><span>λ</span><span><span><span><span><span><span><span>cost</span></span></span></span><span></span></span></span></span></span></span></span></span>; <span><span>kk</span><span><span><span>k</span></span></span></span>; effective <span><span>dmaxd_{\max}</span><span><span><span><span>d</span><span><span><span><span><span><span><span><span>m</span><span>a</span><span>x</span></span></span></span></span><span></span></span></span></span></span></span></span></span>; <span><span>\Lan/\Ran\Lan/\Ran</span><span><span><span><span>\Lan</span></span><span>/</span><span><span>\Ran</span></span></span></span></span> kernel + build flags; hardware/BLAS; e-process updates <span><span>(et,Mt)(e_t,M_t)</span><span><span><span>(</span><span><span>e</span><span><span><span><span><span><span>t</span></span></span><span></span></span></span></span></span><span>,</span><span><span>M</span><span><span><span><span><span><span>t</span></span></span><span></span></span></span></span></span><span>)</span></span></span></span>; Conformal calibration snapshots.</p> </li> </ul> |
| title | Inference in Normal Form: Unifying LLM Tricks via TRoT |
| topic | LLM Large Language Models Artificial intelligence LLM inference Machine learning decoding MBR conformal prediction verifier RAG Chain-of-Thought Tree-of-Thought Graph-of-Thought Theory of Relativity of Theories Verifier-Sandwich category theory Lawvere metric quantale tropical algebra GraphBLAS Kan extension e-process Algorithms |
| url | https://doi.org/10.5281/zenodo.17389109 |