Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2410.16705 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913559192010752 |
|---|---|
| author | Burgess, Mark A. Hosking, Brendan Reguant, Roc Kaphle, Anubhav O'Brien, Mitchell J. Sng, Letitia M. F. Jain, Yatish Bauer, Denis C. |
| author_facet | Burgess, Mark A. Hosking, Brendan Reguant, Roc Kaphle, Anubhav O'Brien, Mitchell J. Sng, Letitia M. F. Jain, Yatish Bauer, Denis C. |
| contents | Machine-generated data is a valuable resource for training Artificial Intelligence algorithms, evaluating rare workflows, and sharing data under stricter data legislations. The challenge is to generate data that is accurate and private. Current statistical and deep learning methods struggle with large data volumes, are prone to hallucinating scenarios incompatible with reality, and seldom quantify privacy meaningfully. Here we introduce Genomator, a logic solving approach (SAT solving), which efficiently produces private and realistic representations of the original data. We demonstrate the method on genomic data, which arguably is the most complex and private information. Synthetic genomes hold great potential for balancing underrepresented populations in medical research and advancing global data exchange. We benchmark Genomator against state-of-the-art methodologies (Markov generation, Restricted Boltzmann Machine, Generative Adversarial Network and Conditional Restricted Boltzmann Machines), demonstrating an 84-93% accuracy improvement and 95-98% higher privacy. Genomator is also 1000-1600 times more efficient, making it the only tested method that scales to whole genomes. We show the universal trade-off between privacy and accuracy, and use Genomator's tuning capability to cater to all applications along the spectrum, from provable private representations of sensitive cohorts, to datasets with indistinguishable pharmacogenomic profiles. Demonstrating the production-scale generation of tuneable synthetic data can increase trust and pave the way into the clinic. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_16705 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Privacy-hardened and hallucination-resistant synthetic data generation with logic-solvers Burgess, Mark A. Hosking, Brendan Reguant, Roc Kaphle, Anubhav O'Brien, Mitchell J. Sng, Letitia M. F. Jain, Yatish Bauer, Denis C. Artificial Intelligence Cryptography and Security Computers and Society Machine Learning Machine-generated data is a valuable resource for training Artificial Intelligence algorithms, evaluating rare workflows, and sharing data under stricter data legislations. The challenge is to generate data that is accurate and private. Current statistical and deep learning methods struggle with large data volumes, are prone to hallucinating scenarios incompatible with reality, and seldom quantify privacy meaningfully. Here we introduce Genomator, a logic solving approach (SAT solving), which efficiently produces private and realistic representations of the original data. We demonstrate the method on genomic data, which arguably is the most complex and private information. Synthetic genomes hold great potential for balancing underrepresented populations in medical research and advancing global data exchange. We benchmark Genomator against state-of-the-art methodologies (Markov generation, Restricted Boltzmann Machine, Generative Adversarial Network and Conditional Restricted Boltzmann Machines), demonstrating an 84-93% accuracy improvement and 95-98% higher privacy. Genomator is also 1000-1600 times more efficient, making it the only tested method that scales to whole genomes. We show the universal trade-off between privacy and accuracy, and use Genomator's tuning capability to cater to all applications along the spectrum, from provable private representations of sensitive cohorts, to datasets with indistinguishable pharmacogenomic profiles. Demonstrating the production-scale generation of tuneable synthetic data can increase trust and pave the way into the clinic. |
| title | Privacy-hardened and hallucination-resistant synthetic data generation with logic-solvers |
| topic | Artificial Intelligence Cryptography and Security Computers and Society Machine Learning |
| url | https://arxiv.org/abs/2410.16705 |