Supervised Learning for data cleaning in the coherence and completeness dimensions
Fuente:
Redalyc
Saved in:
| Main Author: | |
|---|---|
| Format: | Artículo científico |
| Language: | en |
| Published: |
Universidad del Valle
2022
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1876475049814261760 |
|---|---|
| author | Juan C Amézquita |
| author_facet | Juan C Amézquita |
| contents | Supervised Learning for data cleaning in the coherence and completeness dimensions Juan C Amézquita Hermes J Eslava Ingeniería Data Quality Coherence Completeness Machine learning Information has become an asset for companies because most business strategic decisions are made based on data analysis; however, the best results are not always obtained in these analyses due to the low quality of information. It as several evaluation dimensions, making the task complex of achieving an adequate level of quality. One of the main activities before proceeding with any type of analysis is the pre-processing of the data. This activity is one of the most demanding in time; the expected levels of quality are not always obtained, nor are the evaluation dimensions with the most significant impact are covered. This work presents the use of machine learning as a tool to clean data in the dimension of completeness and coherence; its validation is done on a data set provided by a government entity in charge of protecting children’s rights at the national level. It starts from the selection of the information processing tools, the descriptive analysis of the data, the specific identification of the problems in which the machine learning techniques will be applied to improve the quality of the data, experimentation, and evaluation of the different models, and finally the implementation of the best performing model. Among the results of this work, there is an improvement in the completeness dimension, decreasing the null data by 4.9%. In the coherence dimension, 2.6% of the records were identified with contradictions, thus validating machine learning for data cleaning. 2022 artículo científico 0123-3033 https://www.redalyc.org/articulo.oa?id=291374362009 https://www.redalyc.org/journal/2913/291374362009/ https://www.redalyc.org/journal/2913/291374362009/html/ https://www.redalyc.org/journal/2913/291374362009/291374362009.epub https://www.redalyc.org/journal/2913/291374362009/movil 10.25100/iyc.v24i2.11361 en http://www.redalyc.org/revista.oa?id=2913 Ingeniería y Competitividad application/pdf Universidad del Valle Ingeniería y Competitividad (Colombia) Num.2 Vol.24 |
| format | Artículo científico |
| id | redalyc_291374362009 |
| institution | Redalyc |
| language | en |
| publishDate | 2022 |
| publisher | Universidad del Valle |
| spellingShingle | Supervised Learning for data cleaning in the coherence and completeness dimensions Juan C Amézquita Ingeniería Data Quality Coherence Completeness Machine learning Supervised Learning for data cleaning in the coherence and completeness dimensions Juan C Amézquita Hermes J Eslava Ingeniería Data Quality Coherence Completeness Machine learning Information has become an asset for companies because most business strategic decisions are made based on data analysis; however, the best results are not always obtained in these analyses due to the low quality of information. It as several evaluation dimensions, making the task complex of achieving an adequate level of quality. One of the main activities before proceeding with any type of analysis is the pre-processing of the data. This activity is one of the most demanding in time; the expected levels of quality are not always obtained, nor are the evaluation dimensions with the most significant impact are covered. This work presents the use of machine learning as a tool to clean data in the dimension of completeness and coherence; its validation is done on a data set provided by a government entity in charge of protecting children’s rights at the national level. It starts from the selection of the information processing tools, the descriptive analysis of the data, the specific identification of the problems in which the machine learning techniques will be applied to improve the quality of the data, experimentation, and evaluation of the different models, and finally the implementation of the best performing model. Among the results of this work, there is an improvement in the completeness dimension, decreasing the null data by 4.9%. In the coherence dimension, 2.6% of the records were identified with contradictions, thus validating machine learning for data cleaning. 2022 artículo científico 0123-3033 https://www.redalyc.org/articulo.oa?id=291374362009 https://www.redalyc.org/journal/2913/291374362009/ https://www.redalyc.org/journal/2913/291374362009/html/ https://www.redalyc.org/journal/2913/291374362009/291374362009.epub https://www.redalyc.org/journal/2913/291374362009/movil 10.25100/iyc.v24i2.11361 en http://www.redalyc.org/revista.oa?id=2913 Ingeniería y Competitividad application/pdf Universidad del Valle Ingeniería y Competitividad (Colombia) Num.2 Vol.24 |
| title | Supervised Learning for data cleaning in the coherence and completeness dimensions |
| topic | Ingeniería Data Quality Coherence Completeness Machine learning |
| url | https://www.redalyc.org/articulo.oa?id=291374362009 https://www.redalyc.org/journal/2913/291374362009/ https://www.redalyc.org/journal/2913/291374362009/html/ https://www.redalyc.org/journal/2913/291374362009/291374362009.epub https://www.redalyc.org/journal/2913/291374362009/movil |