An hybrid stochastic Newton algorithm for logistic regression
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915647162679296 |
|---|---|
| author | Bercu, Bernard Fredes, Luis Gbaguidi, Eméric |
| author_facet | Bercu, Bernard Fredes, Luis Gbaguidi, Eméric |
| contents | In this paper, we investigate a second-order stochastic algorithm for solving large-scale binary classification problems. We propose to make use of a new hybrid stochastic Newton algorithm that includes two weighted components in the Hessian matrix estimation: the first one coming from the natural Hessian estimate and the second associated with the stochastic gradient information. Our motivation comes from the fact that both parts evaluated at the true parameter of logistic regression, are equal to the Hessian matrix. This new formulation has several advantages and it enables us to prove the almost sure convergence of our stochastic algorithm to the true parameter. Moreover, we significantly improve the almost sure rate of convergence to the Hessian matrix. Furthermore, we establish the central limit theorem for our hybrid stochastic Newton algorithm. Finally, we show a surprising result on the almost sure convergence of the cumulative excess risk. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_01790 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | An hybrid stochastic Newton algorithm for logistic regression Bercu, Bernard Fredes, Luis Gbaguidi, Eméric Computation Probability Statistics Theory Machine Learning 49M15, 68W27, 62J12, 60B10, 60F05 In this paper, we investigate a second-order stochastic algorithm for solving large-scale binary classification problems. We propose to make use of a new hybrid stochastic Newton algorithm that includes two weighted components in the Hessian matrix estimation: the first one coming from the natural Hessian estimate and the second associated with the stochastic gradient information. Our motivation comes from the fact that both parts evaluated at the true parameter of logistic regression, are equal to the Hessian matrix. This new formulation has several advantages and it enables us to prove the almost sure convergence of our stochastic algorithm to the true parameter. Moreover, we significantly improve the almost sure rate of convergence to the Hessian matrix. Furthermore, we establish the central limit theorem for our hybrid stochastic Newton algorithm. Finally, we show a surprising result on the almost sure convergence of the cumulative excess risk. |
| title | An hybrid stochastic Newton algorithm for logistic regression |
| topic | Computation Probability Statistics Theory Machine Learning 49M15, 68W27, 62J12, 60B10, 60F05 |
| url | https://arxiv.org/abs/2512.01790 |