Development of Automated Data Quality Assessment and Evaluation Indices by Analytical Experience
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866913775449276416 |
|---|---|
| author | Haruki, Yuka Kato, Kei Enami, Yuki Takeuchi, Hiroaki Kazuno, Daiki Yamada, Kotaro Hayashi, Teruaki |
| author_facet | Haruki, Yuka Kato, Kei Enami, Yuki Takeuchi, Hiroaki Kazuno, Daiki Yamada, Kotaro Hayashi, Teruaki |
| contents | The societal need to leverage third-party data has driven the data-distribution market and increased the importance of data quality assessment (DQA) in data transactions between organizations. However, DQA requires expert knowledge of raw data and related data attributes, which hinders consensus-building in data purchasing. This study focused on the differences in DQAs between experienced and inexperienced data handlers. We performed two experiments: The first was a questionnaire survey involving 41 participants with varying levels of data-handling experience, who evaluated 12 data samples using 10 predefined indices with and without quality metadata generated by the automated tool. The second was an eye-tracking experiment to reveal the viewing behavior of participants during data evaluation. It was revealed that using quality metadata generated by the automated tool can reduce misrecognition in DQA. While experienced data handlers rated the quality metadata highly, semi-experienced users gave it the lowest ratings. This study contributes to enhancing data understanding within organizations and promoting the distribution of valuable data by proposing an automated tool to support DQAs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_02663 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Development of Automated Data Quality Assessment and Evaluation Indices by Analytical Experience Haruki, Yuka Kato, Kei Enami, Yuki Takeuchi, Hiroaki Kazuno, Daiki Yamada, Kotaro Hayashi, Teruaki Human-Computer Interaction The societal need to leverage third-party data has driven the data-distribution market and increased the importance of data quality assessment (DQA) in data transactions between organizations. However, DQA requires expert knowledge of raw data and related data attributes, which hinders consensus-building in data purchasing. This study focused on the differences in DQAs between experienced and inexperienced data handlers. We performed two experiments: The first was a questionnaire survey involving 41 participants with varying levels of data-handling experience, who evaluated 12 data samples using 10 predefined indices with and without quality metadata generated by the automated tool. The second was an eye-tracking experiment to reveal the viewing behavior of participants during data evaluation. It was revealed that using quality metadata generated by the automated tool can reduce misrecognition in DQA. While experienced data handlers rated the quality metadata highly, semi-experienced users gave it the lowest ratings. This study contributes to enhancing data understanding within organizations and promoting the distribution of valuable data by proposing an automated tool to support DQAs. |
| title | Development of Automated Data Quality Assessment and Evaluation Indices by Analytical Experience |
| topic | Human-Computer Interaction |
| url | https://arxiv.org/abs/2504.02663 |