Imagen de Portada

EmbeddingGemma: Powerful and Lightweight Text Representations

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Vera, Henrique Schechter, Dua, Sahil, Zhang, Biao, Salz, Daniel, Mullins, Ryan, Panyam, Sindhu Raghuram, Smoot, Sara, Naim, Iftekhar, Zou, Joe, Chen, Feiyang, Cer, Daniel, Lisak, Alice, Choi, Min, Gonzalez, Lucas, Sanseviero, Omar, Cameron, Glenn, Ballantyne, Ian, Black, Kat, Chen, Kaifeng, Wang, Weiyi, Li, Zhe, Martins, Gus, Lee, Jinhyuk, Sherwood, Mark, Ji, Juyeong, Wu, Renjie, Zheng, Jingxiao, Singh, Jyotinder, Sharma, Abheesht, Sreepathihalli, Divyashree, Jain, Aashi, Elarabawy, Adham, Co, AJ, Doumanoglou, Andreas, Samari, Babak, Hora, Ben, Potetz, Brian, Kim, Dahun, Alfonseca, Enrique, Moiseev, Fedor, Han, Feng, Gomez, Frank Palma, Ábrego, Gustavo Hernández, Zhang, Hesen, Hui, Hui, Han, Jay, Gill, Karan, Chen, Ke, Chen, Koert, Shanbhogue, Madhuri, Boratko, Michael, Suganthan, Paul, Duddu, Sai Meher Karthik, Mariserla, Sandeep, Ariafar, Setareh, Zhang, Shanfeng, Zhang, Shijie, Baumgartner, Simon, Goenka, Sonam, Qiu, Steve, Dabral, Tanmaya, Walker, Trevor, Rao, Vikram, Khawaja, Waleed, Zhou, Wenlei, Ren, Xiaoqi, Xia, Ye, Chen, Yichang, Chen, Yi-Ting, Dong, Zhe, Ding, Zhongli, Visin, Francesco, Liu, Gaël, Zhang, Jiageng, Kenealy, Kathleen, Casbon, Michelle, Kumar, Ravin, Mesnard, Thomas, Gleicher, Zach, Brick, Cormac, Lacombe, Olivier, Roberts, Adam, Yin, Qin, Sung, Yunhsuan, Hoffmann, Raphael, Warkentin, Tris, Joulin, Armand, Duerig, Tom, Seyedhosseini, Mojtaba
Formato:	Preprint
Publicado:	2025
Materias:	Computation and Language Artificial Intelligence
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

_version_	1866911245667401728
author	Vera, Henrique Schechter Dua, Sahil Zhang, Biao Salz, Daniel Mullins, Ryan Panyam, Sindhu Raghuram Smoot, Sara Naim, Iftekhar Zou, Joe Chen, Feiyang Cer, Daniel Lisak, Alice Choi, Min Gonzalez, Lucas Sanseviero, Omar Cameron, Glenn Ballantyne, Ian Black, Kat Chen, Kaifeng Wang, Weiyi Li, Zhe Martins, Gus Lee, Jinhyuk Sherwood, Mark Ji, Juyeong Wu, Renjie Zheng, Jingxiao Singh, Jyotinder Sharma, Abheesht Sreepathihalli, Divyashree Jain, Aashi Elarabawy, Adham Co, AJ Doumanoglou, Andreas Samari, Babak Hora, Ben Potetz, Brian Kim, Dahun Alfonseca, Enrique Moiseev, Fedor Han, Feng Gomez, Frank Palma Ábrego, Gustavo Hernández Zhang, Hesen Hui, Hui Han, Jay Gill, Karan Chen, Ke Chen, Koert Shanbhogue, Madhuri Boratko, Michael Suganthan, Paul Duddu, Sai Meher Karthik Mariserla, Sandeep Ariafar, Setareh Zhang, Shanfeng Zhang, Shijie Baumgartner, Simon Goenka, Sonam Qiu, Steve Dabral, Tanmaya Walker, Trevor Rao, Vikram Khawaja, Waleed Zhou, Wenlei Ren, Xiaoqi Xia, Ye Chen, Yichang Chen, Yi-Ting Dong, Zhe Ding, Zhongli Visin, Francesco Liu, Gaël Zhang, Jiageng Kenealy, Kathleen Casbon, Michelle Kumar, Ravin Mesnard, Thomas Gleicher, Zach Brick, Cormac Lacombe, Olivier Roberts, Adam Yin, Qin Sung, Yunhsuan Hoffmann, Raphael Warkentin, Tris Joulin, Armand Duerig, Tom Seyedhosseini, Mojtaba
author_facet	Vera, Henrique Schechter Dua, Sahil Zhang, Biao Salz, Daniel Mullins, Ryan Panyam, Sindhu Raghuram Smoot, Sara Naim, Iftekhar Zou, Joe Chen, Feiyang Cer, Daniel Lisak, Alice Choi, Min Gonzalez, Lucas Sanseviero, Omar Cameron, Glenn Ballantyne, Ian Black, Kat Chen, Kaifeng Wang, Weiyi Li, Zhe Martins, Gus Lee, Jinhyuk Sherwood, Mark Ji, Juyeong Wu, Renjie Zheng, Jingxiao Singh, Jyotinder Sharma, Abheesht Sreepathihalli, Divyashree Jain, Aashi Elarabawy, Adham Co, AJ Doumanoglou, Andreas Samari, Babak Hora, Ben Potetz, Brian Kim, Dahun Alfonseca, Enrique Moiseev, Fedor Han, Feng Gomez, Frank Palma Ábrego, Gustavo Hernández Zhang, Hesen Hui, Hui Han, Jay Gill, Karan Chen, Ke Chen, Koert Shanbhogue, Madhuri Boratko, Michael Suganthan, Paul Duddu, Sai Meher Karthik Mariserla, Sandeep Ariafar, Setareh Zhang, Shanfeng Zhang, Shijie Baumgartner, Simon Goenka, Sonam Qiu, Steve Dabral, Tanmaya Walker, Trevor Rao, Vikram Khawaja, Waleed Zhou, Wenlei Ren, Xiaoqi Xia, Ye Chen, Yichang Chen, Yi-Ting Dong, Zhe Ding, Zhongli Visin, Francesco Liu, Gaël Zhang, Jiageng Kenealy, Kathleen Casbon, Michelle Kumar, Ravin Mesnard, Thomas Gleicher, Zach Brick, Cormac Lacombe, Olivier Roberts, Adam Yin, Qin Sung, Yunhsuan Hoffmann, Raphael Warkentin, Tris Joulin, Armand Duerig, Tom Seyedhosseini, Mojtaba
contents	We introduce EmbeddingGemma, a new lightweight, open text embedding model based on the Gemma 3 language model family. Our innovative training recipe strategically captures knowledge from larger models via encoder-decoder initialization and geometric embedding distillation. We improve model robustness and expressiveness with a spread-out regularizer, and ensure generalizability by merging checkpoints from varied, optimized mixtures. Evaluated on the Massive Text Embedding Benchmark (MTEB) across multilingual, English, and code domains, EmbeddingGemma (300M) achieves state-of-the-art results. Notably, it outperforms prior top models, both proprietary and open, with fewer than 500M parameters, and provides performance comparable to models double its size, offering an exceptional performance-to-cost ratio. Remarkably, this lead persists when quantizing model weights or truncating embedding outputs. This makes EmbeddingGemma particularly well-suited for low-latency and high-throughput use cases such as on-device applications. We provide ablation studies exploring our key design choices. We release EmbeddingGemma to the community to promote further research.
format	Preprint
id	arxiv_https___arxiv_org_abs_2509_20354
institution	arXiv
publishDate	2025
record_format	arxiv
spellingShingle	EmbeddingGemma: Powerful and Lightweight Text Representations Vera, Henrique Schechter Dua, Sahil Zhang, Biao Salz, Daniel Mullins, Ryan Panyam, Sindhu Raghuram Smoot, Sara Naim, Iftekhar Zou, Joe Chen, Feiyang Cer, Daniel Lisak, Alice Choi, Min Gonzalez, Lucas Sanseviero, Omar Cameron, Glenn Ballantyne, Ian Black, Kat Chen, Kaifeng Wang, Weiyi Li, Zhe Martins, Gus Lee, Jinhyuk Sherwood, Mark Ji, Juyeong Wu, Renjie Zheng, Jingxiao Singh, Jyotinder Sharma, Abheesht Sreepathihalli, Divyashree Jain, Aashi Elarabawy, Adham Co, AJ Doumanoglou, Andreas Samari, Babak Hora, Ben Potetz, Brian Kim, Dahun Alfonseca, Enrique Moiseev, Fedor Han, Feng Gomez, Frank Palma Ábrego, Gustavo Hernández Zhang, Hesen Hui, Hui Han, Jay Gill, Karan Chen, Ke Chen, Koert Shanbhogue, Madhuri Boratko, Michael Suganthan, Paul Duddu, Sai Meher Karthik Mariserla, Sandeep Ariafar, Setareh Zhang, Shanfeng Zhang, Shijie Baumgartner, Simon Goenka, Sonam Qiu, Steve Dabral, Tanmaya Walker, Trevor Rao, Vikram Khawaja, Waleed Zhou, Wenlei Ren, Xiaoqi Xia, Ye Chen, Yichang Chen, Yi-Ting Dong, Zhe Ding, Zhongli Visin, Francesco Liu, Gaël Zhang, Jiageng Kenealy, Kathleen Casbon, Michelle Kumar, Ravin Mesnard, Thomas Gleicher, Zach Brick, Cormac Lacombe, Olivier Roberts, Adam Yin, Qin Sung, Yunhsuan Hoffmann, Raphael Warkentin, Tris Joulin, Armand Duerig, Tom Seyedhosseini, Mojtaba Computation and Language Artificial Intelligence We introduce EmbeddingGemma, a new lightweight, open text embedding model based on the Gemma 3 language model family. Our innovative training recipe strategically captures knowledge from larger models via encoder-decoder initialization and geometric embedding distillation. We improve model robustness and expressiveness with a spread-out regularizer, and ensure generalizability by merging checkpoints from varied, optimized mixtures. Evaluated on the Massive Text Embedding Benchmark (MTEB) across multilingual, English, and code domains, EmbeddingGemma (300M) achieves state-of-the-art results. Notably, it outperforms prior top models, both proprietary and open, with fewer than 500M parameters, and provides performance comparable to models double its size, offering an exceptional performance-to-cost ratio. Remarkably, this lead persists when quantizing model weights or truncating embedding outputs. This makes EmbeddingGemma particularly well-suited for low-latency and high-throughput use cases such as on-device applications. We provide ablation studies exploring our key design choices. We release EmbeddingGemma to the community to promote further research.
title	EmbeddingGemma: Powerful and Lightweight Text Representations
topic	Computation and Language Artificial Intelligence
url	https://arxiv.org/abs/2509.20354