Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2508.03520 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911282123243520 |
|---|---|
| author | Hasan, Md Rakibul Hossain, Md Zakir Krishna, Aneesh Rahman, Shafin Gedeon, Tom |
| author_facet | Hasan, Md Rakibul Hossain, Md Zakir Krishna, Aneesh Rahman, Shafin Gedeon, Tom |
| contents | Noisy self-reported empathy scores challenge supervised learning for empathy regression. While many algorithms have been proposed for learning with noisy labels in textual classification problems, the regression counterpart is relatively under-explored. We propose UPLME, an uncertainty-aware probabilistic language modelling framework to capture label noise in empathy regression tasks. One of the novelties in UPLME is a probabilistic language model that predicts both empathy scores and heteroscedastic uncertainty, and is trained using Bayesian concepts with variational model ensembling. We further introduce two novel loss components: one penalises degenerate Uncertainty Quantification (UQ), and another enforces similarity between the input pairs on which empathy is being predicted. UPLME achieves state-of-the-art performance (Pearson Correlation Coefficient: $0.558\rightarrow0.580$ and $0.629\rightarrow0.634$) in terms of the performance reported in the literature on two public benchmarks with label noise. Through synthetic label noise injection, we demonstrate that UPLME is effective in distinguishing between noisy and clean samples based on the predicted uncertainty. UPLME further outperform (Calibration error: $0.571\rightarrow0.376$) a recent variational model ensembling-based UQ method designed for regression problems. Code is publicly available at https://github.com/hasan-rakibul/UPLME. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_03520 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | UPLME: Uncertainty-Aware Probabilistic Language Modelling for Robust Empathy Regression Hasan, Md Rakibul Hossain, Md Zakir Krishna, Aneesh Rahman, Shafin Gedeon, Tom Computation and Language Machine Learning Noisy self-reported empathy scores challenge supervised learning for empathy regression. While many algorithms have been proposed for learning with noisy labels in textual classification problems, the regression counterpart is relatively under-explored. We propose UPLME, an uncertainty-aware probabilistic language modelling framework to capture label noise in empathy regression tasks. One of the novelties in UPLME is a probabilistic language model that predicts both empathy scores and heteroscedastic uncertainty, and is trained using Bayesian concepts with variational model ensembling. We further introduce two novel loss components: one penalises degenerate Uncertainty Quantification (UQ), and another enforces similarity between the input pairs on which empathy is being predicted. UPLME achieves state-of-the-art performance (Pearson Correlation Coefficient: $0.558\rightarrow0.580$ and $0.629\rightarrow0.634$) in terms of the performance reported in the literature on two public benchmarks with label noise. Through synthetic label noise injection, we demonstrate that UPLME is effective in distinguishing between noisy and clean samples based on the predicted uncertainty. UPLME further outperform (Calibration error: $0.571\rightarrow0.376$) a recent variational model ensembling-based UQ method designed for regression problems. Code is publicly available at https://github.com/hasan-rakibul/UPLME. |
| title | UPLME: Uncertainty-Aware Probabilistic Language Modelling for Robust Empathy Regression |
| topic | Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2508.03520 |