Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2501.06278 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913675319705600 |
|---|---|
| author | Lamprou, Zenon Polick, Frank Moshfeghi, Yashar |
| author_facet | Lamprou, Zenon Polick, Frank Moshfeghi, Yashar |
| contents | This research examines the congruence between neural activity and advanced transformer models, emphasizing the semantic significance of punctuation in text understanding. Utilizing an innovative approach originally proposed by Toneva and Wehbe, we evaluate four advanced transformer models RoBERTa, DistiliBERT, ALBERT, and ELECTRA against neural activity data. Our findings indicate that RoBERTa exhibits the closest alignment with neural activity, surpassing BERT in accuracy. Furthermore, we investigate the impact of punctuation removal on model performance and neural alignment, revealing that BERT's accuracy enhances in the absence of punctuation. This study contributes to the comprehension of how neural networks represent language and the influence of punctuation on semantic processing within the human brain. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_06278 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Aligning Brain Activity with Advanced Transformer Models: Exploring the Role of Punctuation in Semantic Processing Lamprou, Zenon Polick, Frank Moshfeghi, Yashar Computation and Language Machine Learning This research examines the congruence between neural activity and advanced transformer models, emphasizing the semantic significance of punctuation in text understanding. Utilizing an innovative approach originally proposed by Toneva and Wehbe, we evaluate four advanced transformer models RoBERTa, DistiliBERT, ALBERT, and ELECTRA against neural activity data. Our findings indicate that RoBERTa exhibits the closest alignment with neural activity, surpassing BERT in accuracy. Furthermore, we investigate the impact of punctuation removal on model performance and neural alignment, revealing that BERT's accuracy enhances in the absence of punctuation. This study contributes to the comprehension of how neural networks represent language and the influence of punctuation on semantic processing within the human brain. |
| title | Aligning Brain Activity with Advanced Transformer Models: Exploring the Role of Punctuation in Semantic Processing |
| topic | Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2501.06278 |