Beyond Least Squares: Robust Regression Transformer (R2T)
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866918114274312192 |
|---|---|
| author | Gutierrez, Roman Tang, Tony Kai Gutierrez, Isabel |
| author_facet | Gutierrez, Roman Tang, Tony Kai Gutierrez, Isabel |
| contents | Robust regression techniques rely on least-squares optimization, which works well for Gaussian noise but fails in the presence of asymmetric structured noise. We propose a hybrid neural-symbolic architecture where a transformer encoder processes numerical sequences, a compression NN predicts symbolic parameters, and a fixed symbolic equation reconstructs the original sequence. Using synthetic data, the training objective is to recover the original sequence after adding asymmetric structured noise, effectively learning a symbolic fit guided by neural parameter estimation. Our model achieves a median regression MSE of 6e-6 to 3.5e-5 on synthetic wearable data, which is a 10-300 times improvement when compared with ordinary least squares fit and robust regression techniques such as Huber loss or SoftL1. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_02874 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Beyond Least Squares: Robust Regression Transformer (R2T) Gutierrez, Roman Tang, Tony Kai Gutierrez, Isabel Machine Learning Artificial Intelligence 68T30, 65D10, 62J02, 68T07, 62F35, 62J02 I.2.6; G.1.2; G.3 Robust regression techniques rely on least-squares optimization, which works well for Gaussian noise but fails in the presence of asymmetric structured noise. We propose a hybrid neural-symbolic architecture where a transformer encoder processes numerical sequences, a compression NN predicts symbolic parameters, and a fixed symbolic equation reconstructs the original sequence. Using synthetic data, the training objective is to recover the original sequence after adding asymmetric structured noise, effectively learning a symbolic fit guided by neural parameter estimation. Our model achieves a median regression MSE of 6e-6 to 3.5e-5 on synthetic wearable data, which is a 10-300 times improvement when compared with ordinary least squares fit and robust regression techniques such as Huber loss or SoftL1. |
| title | Beyond Least Squares: Robust Regression Transformer (R2T) |
| topic | Machine Learning Artificial Intelligence 68T30, 65D10, 62J02, 68T07, 62F35, 62J02 I.2.6; G.1.2; G.3 |
| url | https://arxiv.org/abs/2508.02874 |