Cover Image

Teuken-7B-Base & Teuken-7B-Instruct: Towards European LLMs

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Ali, Mehdi, Fromm, Michael, Thellmann, Klaudia, Ebert, Jan, Weber, Alexander Arno, Rutmann, Richard, Jain, Charvi, Lübbering, Max, Steinigen, Daniel, Leveling, Johannes, Klug, Katrin, Buschhoff, Jasper Schulze, Jurkschat, Lena, Abdelwahab, Hammam, Stein, Benny Jörg, Sylla, Karl-Heinz, Denisov, Pavel, Brandizzi, Nicolo', Saleem, Qasid, Bhowmick, Anirban, Helmer, Lennard, John, Chelsea, Suarez, Pedro Ortiz, Ostendorff, Malte, Jude, Alex, Manjunath, Lalith, Weinbach, Samuel, Penke, Carolin, Filatov, Oleg, Barth, Fabio, Mirza, Paramita, Weber, Lucas, Wendler, Ines, Sifa, Rafet, Küch, Fabian, Herten, Andreas, Jäkel, René, Rehm, Georg, Kesselheim, Stefan, Köhler, Joachim, Flores-Herr, Nicolas
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Artificial Intelligence Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

_version_	1866913998842101760
author	Ali, Mehdi Fromm, Michael Thellmann, Klaudia Ebert, Jan Weber, Alexander Arno Rutmann, Richard Jain, Charvi Lübbering, Max Steinigen, Daniel Leveling, Johannes Klug, Katrin Buschhoff, Jasper Schulze Jurkschat, Lena Abdelwahab, Hammam Stein, Benny Jörg Sylla, Karl-Heinz Denisov, Pavel Brandizzi, Nicolo' Saleem, Qasid Bhowmick, Anirban Helmer, Lennard John, Chelsea Suarez, Pedro Ortiz Ostendorff, Malte Jude, Alex Manjunath, Lalith Weinbach, Samuel Penke, Carolin Filatov, Oleg Barth, Fabio Mirza, Paramita Weber, Lucas Wendler, Ines Sifa, Rafet Küch, Fabian Herten, Andreas Jäkel, René Rehm, Georg Kesselheim, Stefan Köhler, Joachim Flores-Herr, Nicolas
author_facet	Ali, Mehdi Fromm, Michael Thellmann, Klaudia Ebert, Jan Weber, Alexander Arno Rutmann, Richard Jain, Charvi Lübbering, Max Steinigen, Daniel Leveling, Johannes Klug, Katrin Buschhoff, Jasper Schulze Jurkschat, Lena Abdelwahab, Hammam Stein, Benny Jörg Sylla, Karl-Heinz Denisov, Pavel Brandizzi, Nicolo' Saleem, Qasid Bhowmick, Anirban Helmer, Lennard John, Chelsea Suarez, Pedro Ortiz Ostendorff, Malte Jude, Alex Manjunath, Lalith Weinbach, Samuel Penke, Carolin Filatov, Oleg Barth, Fabio Mirza, Paramita Weber, Lucas Wendler, Ines Sifa, Rafet Küch, Fabian Herten, Andreas Jäkel, René Rehm, Georg Kesselheim, Stefan Köhler, Joachim Flores-Herr, Nicolas
contents	We present two multilingual LLMs, Teuken 7B-base and Teuken 7B-instruct, designed to embrace Europe's linguistic diversity by supporting all 24 official languages of the European Union. Trained on a dataset comprising around 60% non-English data and utilizing a custom multilingual tokenizer, our models address the limitations of existing LLMs that predominantly focus on English or a few high-resource languages. We detail the models' development principles, i.e., data composition, tokenizer optimization, and training methodologies. The models demonstrate strong performance across multilingual benchmarks, as evidenced by their performance on European versions of ARC, HellaSwag, and TruthfulQA.
format	Preprint
id	arxiv_https___arxiv_org_abs_2410_03730
institution	arXiv
publishDate	2024
record_format	arxiv
spellingShingle	Teuken-7B-Base & Teuken-7B-Instruct: Towards European LLMs Ali, Mehdi Fromm, Michael Thellmann, Klaudia Ebert, Jan Weber, Alexander Arno Rutmann, Richard Jain, Charvi Lübbering, Max Steinigen, Daniel Leveling, Johannes Klug, Katrin Buschhoff, Jasper Schulze Jurkschat, Lena Abdelwahab, Hammam Stein, Benny Jörg Sylla, Karl-Heinz Denisov, Pavel Brandizzi, Nicolo' Saleem, Qasid Bhowmick, Anirban Helmer, Lennard John, Chelsea Suarez, Pedro Ortiz Ostendorff, Malte Jude, Alex Manjunath, Lalith Weinbach, Samuel Penke, Carolin Filatov, Oleg Barth, Fabio Mirza, Paramita Weber, Lucas Wendler, Ines Sifa, Rafet Küch, Fabian Herten, Andreas Jäkel, René Rehm, Georg Kesselheim, Stefan Köhler, Joachim Flores-Herr, Nicolas Computation and Language Artificial Intelligence Machine Learning We present two multilingual LLMs, Teuken 7B-base and Teuken 7B-instruct, designed to embrace Europe's linguistic diversity by supporting all 24 official languages of the European Union. Trained on a dataset comprising around 60% non-English data and utilizing a custom multilingual tokenizer, our models address the limitations of existing LLMs that predominantly focus on English or a few high-resource languages. We detail the models' development principles, i.e., data composition, tokenizer optimization, and training methodologies. The models demonstrate strong performance across multilingual benchmarks, as evidenced by their performance on European versions of ARC, HellaSwag, and TruthfulQA.
title	Teuken-7B-Base & Teuken-7B-Instruct: Towards European LLMs
topic	Computation and Language Artificial Intelligence Machine Learning
url	https://arxiv.org/abs/2410.03730