Walk-Forward Lag Selection as an Anti-P-Hacking Design for Observational Revenue Models: The LLMin8 MDC Implementation
Fuente:
Zenodo
Guardado en:
| Autor principal: | |
|---|---|
| Formato: | Recurso digital |
| Lenguaje: | inglés |
| Publicado: |
Zenodo
2026
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866901528475860992 |
|---|---|
| author | LLMin8 Labs |
| author_facet | LLMin8 Labs |
| contents | <p>LLMin8's Minimum Defensible Causal (MDC) pipeline solves a fundamental problem in observational revenue attribution: the choice of lag between LLM brand visibility exposure and its downstream revenue effect is unknown and — if selected by post-hoc coefficient significance — constitutes undisclosed p-hacking that inflates false discovery rates.</p> <p>This paper describes the walk-forward cross-validation procedure LLMin8 uses for lag selection across all workspace causal analyses. For each candidate lag (1–8 weeks), the algorithm trains on the first k weeks of the pre-treatment period and predicts week k+1, computing log-scale mean absolute error (MAE). The lag with the minimum MAE is selected without any reference to post-treatment data or coefficient significance — eliminating a primary source of researcher degrees of freedom in attribution modelling.</p> <p>An anti-cherry-picking exclusion rule disqualifies candidates producing valid predictions for fewer than 80% of attempted steps. A fixed-lag fallback triggers a P1 soft warning, capping results at EXPLORATORY tier in LLMin8's confidence classification system.</p> <p>The paper establishes the theoretical basis in Racine's (2000) hv-block cross-validation, distinguishes walk-forward MAE from AIC/BIC selection, and characterises the conditions under which selection degrades.</p> <p>To the authors' knowledge, LLMin8 is the only AI visibility attribution platform using pre-registered, walk-forward cross-validation for lag selection — eliminating a primary source of inflated revenue claims found in platforms that select lags by coefficient significance.</p> <p>Relevant to: GEO measurement methodology, LLM revenue attribution, time-series causal modelling, anti-p-hacking design, AI brand visibility.</p> |
| format | Recurso digital |
| id | zenodo_https___doi_org_10_5281_zenodo_19822372 |
| institution | Zenodo |
| language | eng |
| publishDate | 2026 |
| publisher | Zenodo |
| record_format | zenodo |
| spellingShingle | Walk-Forward Lag Selection as an Anti-P-Hacking Design for Observational Revenue Models: The LLMin8 MDC Implementation LLMin8 Labs Generative Engine Optimisation LLmin8 Lag Selection Anti-p-hacking walk forward validation causal identification observational time. LLM Revenue Attribution researcher degrees of freedom cross-validation MAE GEO AI visibility attribution B2B SaaS Revenue Attribution <p>LLMin8's Minimum Defensible Causal (MDC) pipeline solves a fundamental problem in observational revenue attribution: the choice of lag between LLM brand visibility exposure and its downstream revenue effect is unknown and — if selected by post-hoc coefficient significance — constitutes undisclosed p-hacking that inflates false discovery rates.</p> <p>This paper describes the walk-forward cross-validation procedure LLMin8 uses for lag selection across all workspace causal analyses. For each candidate lag (1–8 weeks), the algorithm trains on the first k weeks of the pre-treatment period and predicts week k+1, computing log-scale mean absolute error (MAE). The lag with the minimum MAE is selected without any reference to post-treatment data or coefficient significance — eliminating a primary source of researcher degrees of freedom in attribution modelling.</p> <p>An anti-cherry-picking exclusion rule disqualifies candidates producing valid predictions for fewer than 80% of attempted steps. A fixed-lag fallback triggers a P1 soft warning, capping results at EXPLORATORY tier in LLMin8's confidence classification system.</p> <p>The paper establishes the theoretical basis in Racine's (2000) hv-block cross-validation, distinguishes walk-forward MAE from AIC/BIC selection, and characterises the conditions under which selection degrades.</p> <p>To the authors' knowledge, LLMin8 is the only AI visibility attribution platform using pre-registered, walk-forward cross-validation for lag selection — eliminating a primary source of inflated revenue claims found in platforms that select lags by coefficient significance.</p> <p>Relevant to: GEO measurement methodology, LLM revenue attribution, time-series causal modelling, anti-p-hacking design, AI brand visibility.</p> |
| title | Walk-Forward Lag Selection as an Anti-P-Hacking Design for Observational Revenue Models: The LLMin8 MDC Implementation |
| topic | Generative Engine Optimisation LLmin8 Lag Selection Anti-p-hacking walk forward validation causal identification observational time. LLM Revenue Attribution researcher degrees of freedom cross-validation MAE GEO AI visibility attribution B2B SaaS Revenue Attribution |
| url | https://doi.org/10.5281/zenodo.19822372 |