Cover Image

Correlated Noise Mechanisms for Differentially Private Learning

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Pillutla, Krishna, Upadhyay, Jalaj, Choquette-Choo, Christopher A., Dvijotham, Krishnamurthy, Ganesh, Arun, Henzinger, Monika, Katz, Jonathan, McKenna, Ryan, McMahan, H. Brendan, Rush, Keith, Steinke, Thomas, Thakurta, Abhradeep
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Cryptography and Security
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

_version_	1866918052403085312
author	Pillutla, Krishna Upadhyay, Jalaj Choquette-Choo, Christopher A. Dvijotham, Krishnamurthy Ganesh, Arun Henzinger, Monika Katz, Jonathan McKenna, Ryan McMahan, H. Brendan Rush, Keith Steinke, Thomas Thakurta, Abhradeep
author_facet	Pillutla, Krishna Upadhyay, Jalaj Choquette-Choo, Christopher A. Dvijotham, Krishnamurthy Ganesh, Arun Henzinger, Monika Katz, Jonathan McKenna, Ryan McMahan, H. Brendan Rush, Keith Steinke, Thomas Thakurta, Abhradeep
contents	This monograph explores the design and analysis of correlated noise mechanisms for differential privacy (DP), focusing on their application to private training of AI and machine learning models via the core primitive of estimation of weighted prefix sums. While typical DP mechanisms inject independent noise into each step of a stochastic gradient (SGD) learning algorithm in order to protect the privacy of the training data, a growing body of recent research demonstrates that introducing (anti-)correlations in the noise can significantly improve privacy-utility trade-offs by carefully canceling out some of the noise added on earlier steps in subsequent steps. Such correlated noise mechanisms, known variously as matrix mechanisms, factorization mechanisms, and DP-Follow-the-Regularized-Leader (DP-FTRL) when applied to learning algorithms, have also been influential in practice, with industrial deployment at a global scale.
format	Preprint
id	arxiv_https___arxiv_org_abs_2506_08201
institution	arXiv
publishDate	2025
record_format	arxiv
spellingShingle	Correlated Noise Mechanisms for Differentially Private Learning Pillutla, Krishna Upadhyay, Jalaj Choquette-Choo, Christopher A. Dvijotham, Krishnamurthy Ganesh, Arun Henzinger, Monika Katz, Jonathan McKenna, Ryan McMahan, H. Brendan Rush, Keith Steinke, Thomas Thakurta, Abhradeep Machine Learning Cryptography and Security This monograph explores the design and analysis of correlated noise mechanisms for differential privacy (DP), focusing on their application to private training of AI and machine learning models via the core primitive of estimation of weighted prefix sums. While typical DP mechanisms inject independent noise into each step of a stochastic gradient (SGD) learning algorithm in order to protect the privacy of the training data, a growing body of recent research demonstrates that introducing (anti-)correlations in the noise can significantly improve privacy-utility trade-offs by carefully canceling out some of the noise added on earlier steps in subsequent steps. Such correlated noise mechanisms, known variously as matrix mechanisms, factorization mechanisms, and DP-Follow-the-Regularized-Leader (DP-FTRL) when applied to learning algorithms, have also been influential in practice, with industrial deployment at a global scale.
title	Correlated Noise Mechanisms for Differentially Private Learning
topic	Machine Learning Cryptography and Security
url	https://arxiv.org/abs/2506.08201