NeurIPS 2023 Competition: Privacy Preserving Federated Learning Document VQA
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866912410011435008 |
|---|---|
| author | Tobaben, Marlon Souibgui, Mohamed Ali Tito, Rubèn Nguyen, Khanh Kerkouche, Raouf Jung, Kangsoo Jälkö, Joonas Kang, Lei Barsky, Andrey d'Andecy, Vincent Poulain Joseph, Aurélie Muhamed, Aashiq Kuo, Kevin Smith, Virginia Yamasaki, Yusuke Fukami, Takumi Niwa, Kenta Tyou, Iifan Ishii, Hiro Yokota, Rio N, Ragul Kutum, Rintu Llados, Josep Valveny, Ernest Honkela, Antti Fritz, Mario Karatzas, Dimosthenis |
| author_facet | Tobaben, Marlon Souibgui, Mohamed Ali Tito, Rubèn Nguyen, Khanh Kerkouche, Raouf Jung, Kangsoo Jälkö, Joonas Kang, Lei Barsky, Andrey d'Andecy, Vincent Poulain Joseph, Aurélie Muhamed, Aashiq Kuo, Kevin Smith, Virginia Yamasaki, Yusuke Fukami, Takumi Niwa, Kenta Tyou, Iifan Ishii, Hiro Yokota, Rio N, Ragul Kutum, Rintu Llados, Josep Valveny, Ernest Honkela, Antti Fritz, Mario Karatzas, Dimosthenis |
| contents | The Privacy Preserving Federated Learning Document VQA (PFL-DocVQA) competition challenged the community to develop provably private and communication-efficient solutions in a federated setting for a real-life use case: invoice processing. The competition introduced a dataset of real invoice documents, along with associated questions and answers requiring information extraction and reasoning over the document images. Thereby, it brings together researchers and expertise from the document analysis, privacy, and federated learning communities. Participants fine-tuned a pre-trained, state-of-the-art Document Visual Question Answering model provided by the organizers for this new domain, mimicking a typical federated invoice processing setup. The base model is a multi-modal generative language model, and sensitive information could be exposed through either the visual or textual input modality. Participants proposed elegant solutions to reduce communication costs while maintaining a minimum utility threshold in track 1 and to protect all information from each document provider using differential privacy in track 2. The competition served as a new testbed for developing and testing private federated learning methods, simultaneously raising awareness about privacy within the document image analysis and recognition community. Ultimately, the competition analysis provides best practices and recommendations for successfully running privacy-focused federated learning challenges in the future. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_03730 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | NeurIPS 2023 Competition: Privacy Preserving Federated Learning Document VQA Tobaben, Marlon Souibgui, Mohamed Ali Tito, Rubèn Nguyen, Khanh Kerkouche, Raouf Jung, Kangsoo Jälkö, Joonas Kang, Lei Barsky, Andrey d'Andecy, Vincent Poulain Joseph, Aurélie Muhamed, Aashiq Kuo, Kevin Smith, Virginia Yamasaki, Yusuke Fukami, Takumi Niwa, Kenta Tyou, Iifan Ishii, Hiro Yokota, Rio N, Ragul Kutum, Rintu Llados, Josep Valveny, Ernest Honkela, Antti Fritz, Mario Karatzas, Dimosthenis Machine Learning Cryptography and Security Computer Vision and Pattern Recognition The Privacy Preserving Federated Learning Document VQA (PFL-DocVQA) competition challenged the community to develop provably private and communication-efficient solutions in a federated setting for a real-life use case: invoice processing. The competition introduced a dataset of real invoice documents, along with associated questions and answers requiring information extraction and reasoning over the document images. Thereby, it brings together researchers and expertise from the document analysis, privacy, and federated learning communities. Participants fine-tuned a pre-trained, state-of-the-art Document Visual Question Answering model provided by the organizers for this new domain, mimicking a typical federated invoice processing setup. The base model is a multi-modal generative language model, and sensitive information could be exposed through either the visual or textual input modality. Participants proposed elegant solutions to reduce communication costs while maintaining a minimum utility threshold in track 1 and to protect all information from each document provider using differential privacy in track 2. The competition served as a new testbed for developing and testing private federated learning methods, simultaneously raising awareness about privacy within the document image analysis and recognition community. Ultimately, the competition analysis provides best practices and recommendations for successfully running privacy-focused federated learning challenges in the future. |
| title | NeurIPS 2023 Competition: Privacy Preserving Federated Learning Document VQA |
| topic | Machine Learning Cryptography and Security Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2411.03730 |