Improving Clustering on Occupational Text Data through Dimensionality Reduction
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866909682624364544 |
|---|---|
| author | García, Iago Xabier Vázquez Partanaz, Damla Yetkin, Emrullah Fatih |
| author_facet | García, Iago Xabier Vázquez Partanaz, Damla Yetkin, Emrullah Fatih |
| contents | In this study, we focused on proposing an optimal clustering mechanism for the occupations defined in the well-known US-based occupational database, O*NET. Even though all occupations are defined according to well-conducted surveys in the US, their definitions can vary for different firms and countries. Hence, if one wants to expand the data that is already collected in O*NET for the occupations defined with different tasks, a map between the definitions will be a vital requirement. We proposed a pipeline using several BERT-based techniques with various clustering approaches to obtain such a map. We also examined the effect of dimensionality reduction approaches on several metrics used in measuring performance of clustering algorithms. Finally, we improved our results by using a specialized silhouette approach. This new clustering-based mapping approach with dimensionality reduction may help distinguish the occupations automatically, creating new paths for people wanting to change their careers. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_07582 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Improving Clustering on Occupational Text Data through Dimensionality Reduction García, Iago Xabier Vázquez Partanaz, Damla Yetkin, Emrullah Fatih Machine Learning Computation and Language Computers and Society In this study, we focused on proposing an optimal clustering mechanism for the occupations defined in the well-known US-based occupational database, O*NET. Even though all occupations are defined according to well-conducted surveys in the US, their definitions can vary for different firms and countries. Hence, if one wants to expand the data that is already collected in O*NET for the occupations defined with different tasks, a map between the definitions will be a vital requirement. We proposed a pipeline using several BERT-based techniques with various clustering approaches to obtain such a map. We also examined the effect of dimensionality reduction approaches on several metrics used in measuring performance of clustering algorithms. Finally, we improved our results by using a specialized silhouette approach. This new clustering-based mapping approach with dimensionality reduction may help distinguish the occupations automatically, creating new paths for people wanting to change their careers. |
| title | Improving Clustering on Occupational Text Data through Dimensionality Reduction |
| topic | Machine Learning Computation and Language Computers and Society |
| url | https://arxiv.org/abs/2507.07582 |