Enregistré dans:
| Auteurs principaux: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | https://arxiv.org/abs/2604.14815 |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866910135069179904 |
|---|---|
| author | Luisto, Rami Petäinen, Liisa Grönholm, Tommi Böhm, Jan Ahtiainen, Maarit Lilja, Tomi Pölönen, Ilkka Äyrämö, Sami |
| author_facet | Luisto, Rami Petäinen, Liisa Grönholm, Tommi Böhm, Jan Ahtiainen, Maarit Lilja, Tomi Pölönen, Ilkka Äyrämö, Sami |
| contents | In NLP classification tasks where little labeled data exists, domain fine-tuning of transformer models on unlabeled data is an established approach. In this paper we have two aims.
(1) We describe our observations from fine-tuning the Finnish BERT model on Finnish medical text data.
(2) We report on our attempts to predict the benefit of domain-specific pre-training of Finnish BERT from observing the geometry of embedding changes due to domain fine-tuning.
Our driving motivation is the common\situation in healthcare AI where we might experience long delays in acquiring datasets, especially with respect to labels. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_14815 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Domain Fine-Tuning FinBERT on Finnish Histopathological Reports: Train-Time Signals and Downstream Correlations Luisto, Rami Petäinen, Liisa Grönholm, Tommi Böhm, Jan Ahtiainen, Maarit Lilja, Tomi Pölönen, Ilkka Äyrämö, Sami Computation and Language In NLP classification tasks where little labeled data exists, domain fine-tuning of transformer models on unlabeled data is an established approach. In this paper we have two aims. (1) We describe our observations from fine-tuning the Finnish BERT model on Finnish medical text data. (2) We report on our attempts to predict the benefit of domain-specific pre-training of Finnish BERT from observing the geometry of embedding changes due to domain fine-tuning. Our driving motivation is the common\situation in healthcare AI where we might experience long delays in acquiring datasets, especially with respect to labels. |
| title | Domain Fine-Tuning FinBERT on Finnish Histopathological Reports: Train-Time Signals and Downstream Correlations |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2604.14815 |