By Their Fruits You Will Know Them: Comparing Formalizations of Law by the Decisions They Encode
Fuente:
arXiv
Salvato in:
| Autori principali: | , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866910252626083840 |
|---|---|
| author | Vernie, Julius Grabmair, Matthias |
| author_facet | Vernie, Julius Grabmair, Matthias |
| contents | Formalizing legal provisions promises machine-accessible law and automated legal reasoning, and recent LLMs make it tempting to generate such formalizations directly from statutory text. However, any formalization makes implicit interpretive choices whose consequences are hard to anticipate, especially if an LLM is the author. We present a method for systematically comparing different formalizations of the same legal provision by their inferences on individual cases. Given multiple formalizations of a provision, we match them at the node level, derive a shared interface for each pair from the matching, and use a SAT solver to enumerate the edge cases on which any two formalizations disagree. Selected edge cases are then verbalized into concrete factual scenarios that a legal expert can examine and act on. We apply our method to formalizations of ten EU provisions generated by nine frontier LLMs. We find that behavioral divergence between formalizations is essentially uncorrelated with their structural agreement and that the verbalized cases reveal qualitatively distinct types of disagreement, including divergences that mirror genuine controversies in the legal commentary. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_25186 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | By Their Fruits You Will Know Them: Comparing Formalizations of Law by the Decisions They Encode Vernie, Julius Grabmair, Matthias Computation and Language Artificial Intelligence I.2.7; I.2.4; I.2.3 Formalizing legal provisions promises machine-accessible law and automated legal reasoning, and recent LLMs make it tempting to generate such formalizations directly from statutory text. However, any formalization makes implicit interpretive choices whose consequences are hard to anticipate, especially if an LLM is the author. We present a method for systematically comparing different formalizations of the same legal provision by their inferences on individual cases. Given multiple formalizations of a provision, we match them at the node level, derive a shared interface for each pair from the matching, and use a SAT solver to enumerate the edge cases on which any two formalizations disagree. Selected edge cases are then verbalized into concrete factual scenarios that a legal expert can examine and act on. We apply our method to formalizations of ten EU provisions generated by nine frontier LLMs. We find that behavioral divergence between formalizations is essentially uncorrelated with their structural agreement and that the verbalized cases reveal qualitatively distinct types of disagreement, including divergences that mirror genuine controversies in the legal commentary. |
| title | By Their Fruits You Will Know Them: Comparing Formalizations of Law by the Decisions They Encode |
| topic | Computation and Language Artificial Intelligence I.2.7; I.2.4; I.2.3 |
| url | https://arxiv.org/abs/2605.25186 |