Using Word Embeddings for Query Translation for Hindi to English Cross Language Information Retrieval
Fuente:
Redalyc
Guardado en:
| Autor principal: | |
|---|---|
| Formato: | Artículo científico |
| Lenguaje: | en |
| Publicado: |
Instituto Politécnico Nacional
2016
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1876426428238528512 |
|---|---|
| author | Paheli Bhattacharya |
| author_facet | Paheli Bhattacharya |
| contents | Using Word Embeddings for Query Translation for Hindi to English Cross Language Information Retrieval Paheli Bhattacharya Pawan Goyal Sudeshna Sarkar Computación Cross query translation crosslingual word embeddings Language information retrieval Cross-Language Information Retrieval (CLIR) has become an important problem to solve in the recent years due to the growth of content in multiple languages in the Web. One of the standard methods is to use query translation from source to target language. In this paper, we propose an approach based on word embeddings, a method that captures contextual clues for a particular word in the source language and gives those words as translations that occur in a similar context in the target language. Once we obtain the word embeddings of the source and target language pairs, we learn a projection from source to target word embeddings, making use of a dictionary with word translation pairs. We then propose various methods of query translation and aggregation. The advantage of this approach is that it does not require the corpora to be aligned (which is difficult to obtain for resource-scarce languages), a dictionary with word translation pairs is enough to train the word vectors for translation. We experiment with Forum for Information Retrieval and Evaluation (FIRE) 2008 and 2012 datasets for Hindi to English CLIR. The proposed word embedding based approach outperforms the basic dictionary based approach by 70% and when the word embeddings are combined with the dictionary, the hybrid approach beats the baseline dictionary based method by 77%. It outperforms the English monolingual baseline by 15%, when combined with the translations obtained from Google Translate and Dictionary. 2016 artículo científico 1405-5546 https://www.redalyc.org/articulo.oa?id=61547469012 en http://www.redalyc.org/revista.oa?id=615 Computación y Sistemas application/pdf Instituto Politécnico Nacional Computación y Sistemas (México) Num.3 Vol.20 |
| format | Artículo científico |
| id | redalyc_61547469012 |
| institution | Redalyc |
| language | en |
| publishDate | 2016 |
| publisher | Instituto Politécnico Nacional |
| spellingShingle | Using Word Embeddings for Query Translation for Hindi to English Cross Language Information Retrieval Paheli Bhattacharya Computación Cross query translation crosslingual word embeddings Language information retrieval Using Word Embeddings for Query Translation for Hindi to English Cross Language Information Retrieval Paheli Bhattacharya Pawan Goyal Sudeshna Sarkar Computación Cross query translation crosslingual word embeddings Language information retrieval Cross-Language Information Retrieval (CLIR) has become an important problem to solve in the recent years due to the growth of content in multiple languages in the Web. One of the standard methods is to use query translation from source to target language. In this paper, we propose an approach based on word embeddings, a method that captures contextual clues for a particular word in the source language and gives those words as translations that occur in a similar context in the target language. Once we obtain the word embeddings of the source and target language pairs, we learn a projection from source to target word embeddings, making use of a dictionary with word translation pairs. We then propose various methods of query translation and aggregation. The advantage of this approach is that it does not require the corpora to be aligned (which is difficult to obtain for resource-scarce languages), a dictionary with word translation pairs is enough to train the word vectors for translation. We experiment with Forum for Information Retrieval and Evaluation (FIRE) 2008 and 2012 datasets for Hindi to English CLIR. The proposed word embedding based approach outperforms the basic dictionary based approach by 70% and when the word embeddings are combined with the dictionary, the hybrid approach beats the baseline dictionary based method by 77%. It outperforms the English monolingual baseline by 15%, when combined with the translations obtained from Google Translate and Dictionary. 2016 artículo científico 1405-5546 https://www.redalyc.org/articulo.oa?id=61547469012 en http://www.redalyc.org/revista.oa?id=615 Computación y Sistemas application/pdf Instituto Politécnico Nacional Computación y Sistemas (México) Num.3 Vol.20 |
| title | Using Word Embeddings for Query Translation for Hindi to English Cross Language Information Retrieval |
| topic | Computación Cross query translation crosslingual word embeddings Language information retrieval |
| url | https://www.redalyc.org/articulo.oa?id=61547469012 |