NoLBERT: A No Lookahead(back) Foundational Language Model
Fuente:
arXiv
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909905016848384 |
|---|---|
| author | Kakhbod, Ali Li, Peiyao |
| author_facet | Kakhbod, Ali Li, Peiyao |
| contents | We present NoLBERT, a lightweight, timestamped foundational language model for empirical research -- particularly for forecasting in economics, finance, and the social sciences. By pretraining exclusively on text from 1976 to 1995, NoLBERT avoids both lookback and lookahead biases (information leakage) that can undermine econometric inference. It exceeds domain-specific baselines on NLP benchmarks while maintaining temporal consistency. Applied to patent texts, NoLBERT enables the construction of firm-level innovation networks and shows that gains in innovation centrality predict higher long-run profit growth. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_01110 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | NoLBERT: A No Lookahead(back) Foundational Language Model Kakhbod, Ali Li, Peiyao General Economics Economics Artificial Intelligence Machine Learning General Finance We present NoLBERT, a lightweight, timestamped foundational language model for empirical research -- particularly for forecasting in economics, finance, and the social sciences. By pretraining exclusively on text from 1976 to 1995, NoLBERT avoids both lookback and lookahead biases (information leakage) that can undermine econometric inference. It exceeds domain-specific baselines on NLP benchmarks while maintaining temporal consistency. Applied to patent texts, NoLBERT enables the construction of firm-level innovation networks and shows that gains in innovation centrality predict higher long-run profit growth. |
| title | NoLBERT: A No Lookahead(back) Foundational Language Model |
| topic | General Economics Economics Artificial Intelligence Machine Learning General Finance |
| url | https://arxiv.org/abs/2509.01110 |