Investigating OCR-Sensitive Neurons to Improve Entity Recognition in Historical Documents
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866916484993777664 |
|---|---|
| author | Boros, Emanuela Ehrmann, Maud |
| author_facet | Boros, Emanuela Ehrmann, Maud |
| contents | This paper investigates the presence of OCR-sensitive neurons within the Transformer architecture and their influence on named entity recognition (NER) performance on historical documents. By analysing neuron activation patterns in response to clean and noisy text inputs, we identify and then neutralise OCR-sensitive neurons to improve model performance. Based on two open access large language models (Llama2 and Mistral), experiments demonstrate the existence of OCR-sensitive regions and show improvements in NER performance on historical newspapers and classical commentaries, highlighting the potential of targeted neuron modulation to improve models' performance on noisy text. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_16934 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Investigating OCR-Sensitive Neurons to Improve Entity Recognition in Historical Documents Boros, Emanuela Ehrmann, Maud Computation and Language Artificial Intelligence This paper investigates the presence of OCR-sensitive neurons within the Transformer architecture and their influence on named entity recognition (NER) performance on historical documents. By analysing neuron activation patterns in response to clean and noisy text inputs, we identify and then neutralise OCR-sensitive neurons to improve model performance. Based on two open access large language models (Llama2 and Mistral), experiments demonstrate the existence of OCR-sensitive regions and show improvements in NER performance on historical newspapers and classical commentaries, highlighting the potential of targeted neuron modulation to improve models' performance on noisy text. |
| title | Investigating OCR-Sensitive Neurons to Improve Entity Recognition in Historical Documents |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2409.16934 |