SSE: Multimodal Semantic Data Selection and Enrichment for Industrial-scale Data Assimilation
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866914954391584768 |
|---|---|
| author | Shen, Maying Chang, Nadine Liu, Sifei Alvarez, Jose M. |
| author_facet | Shen, Maying Chang, Nadine Liu, Sifei Alvarez, Jose M. |
| contents | In recent years, the data collected for artificial intelligence has grown to an unmanageable amount. Particularly within industrial applications, such as autonomous vehicles, model training computation budgets are being exceeded while model performance is saturating -- and yet more data continues to pour in. To navigate the flood of data, we propose a framework to select the most semantically diverse and important dataset portion. Then, we further semantically enrich it by discovering meaningful new data from a massive unlabeled data pool. Importantly, we can provide explainability by leveraging foundation models to generate semantics for every data point. We quantitatively show that our Semantic Selection and Enrichment framework (SSE) can a) successfully maintain model performance with a smaller training dataset and b) improve model performance by enriching the smaller dataset without exceeding the original dataset size. Consequently, we demonstrate that semantic diversity is imperative for optimal data selection and model performance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_13860 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | SSE: Multimodal Semantic Data Selection and Enrichment for Industrial-scale Data Assimilation Shen, Maying Chang, Nadine Liu, Sifei Alvarez, Jose M. Computer Vision and Pattern Recognition In recent years, the data collected for artificial intelligence has grown to an unmanageable amount. Particularly within industrial applications, such as autonomous vehicles, model training computation budgets are being exceeded while model performance is saturating -- and yet more data continues to pour in. To navigate the flood of data, we propose a framework to select the most semantically diverse and important dataset portion. Then, we further semantically enrich it by discovering meaningful new data from a massive unlabeled data pool. Importantly, we can provide explainability by leveraging foundation models to generate semantics for every data point. We quantitatively show that our Semantic Selection and Enrichment framework (SSE) can a) successfully maintain model performance with a smaller training dataset and b) improve model performance by enriching the smaller dataset without exceeding the original dataset size. Consequently, we demonstrate that semantic diversity is imperative for optimal data selection and model performance. |
| title | SSE: Multimodal Semantic Data Selection and Enrichment for Industrial-scale Data Assimilation |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2409.13860 |