Staff View: :: Library Catalog

Saved in:

Bibliographic Details
Main Authors:	Grattafiori, Aaron, Dubey, Abhimanyu, Jauhri, Abhinav, Pandey, Abhinav, Kadian, Abhishek, Al-Dahle, Ahmad, Letman, Aiesha, Mathur, Akhil, Schelten, Alan, Vaughan, Alex, Yang, Amy, Fan, Angela, Goyal, Anirudh, Hartshorn, Anthony, Yang, Aobo, Mitra, Archi, Sravankumar, Archie, Korenev, Artem, Hinsvark, Arthur, Rao, Arun, Zhang, Aston, Rodriguez, Aurelien, Gregerson, Austen, Spataru, Ava, Roziere, Baptiste, Biron, Bethany, Tang, Binh, Chern, Bobbie, Caucheteux, Charlotte, Nayak, Chaya, Bi, Chloe, Marra, Chris, McConnell, Chris, Keller, Christian, Touret, Christophe, Wu, Chunyang, Wong, Corinne, Ferrer, Cristian Canton, Nikolaidis, Cyrus, Allonsius, Damien, Song, Daniel, Pintz, Danielle, Livshits, Danny, Wyatt, Danny, Esiobu, David, Choudhary, Dhruv, Mahajan, Dhruv, Garcia-Olano, Diego, Perino, Diego, Hupkes, Dieuwke, Lakomkin, Egor, AlBadawy, Ehab, Lobanova, Elina, Dinan, Emily, Smith, Eric Michael, Radenovic, Filip, Guzmán, Francisco, Zhang, Frank, Synnaeve, Gabriel, Lee, Gabrielle, Anderson, Georgia Lewis, Thattai, Govind, Nail, Graeme, Mialon, Gregoire, Pang, Guan, Cucurell, Guillem, Nguyen, Hailey, Korevaar, Hannah, Xu, Hu, Touvron, Hugo, Zarov, Iliyan, Ibarra, Imanol Arrieta, Kloumann, Isabel, Misra, Ishan, Evtimov, Ivan, Zhang, Jack, Copet, Jade, Lee, Jaewon, Geffert, Jan, Vranes, Jana, Park, Jason, Mahadeokar, Jay, Shah, Jeet, van der Linde, Jelmer, Billock, Jennifer, Hong, Jenny, Lee, Jenya, Fu, Jeremy, Chi, Jianfeng, Huang, Jianyu, Liu, Jiawen, Wang, Jie, Yu, Jiecao, Bitton, Joanna, Spisak, Joe, Park, Jongsoo, Rocca, Joseph, Johnstun, Joshua, Saxe, Joshua, Jia, Junteng, Alwala, Kalyan Vasuden, Prasad, Karthik, Upasani, Kartikeya, Plawiak, Kate, Li, Ke, Heafield, Kenneth, Stone, Kevin, El-Arini, Khalid, Iyer, Krithika, Malik, Kshitiz, Chiu, Kuenley, Bhalla, Kunal, Lakhotia, Kushal, Rantala-Yeary, Lauren, van der Maaten, Laurens, Chen, Lawrence, Tan, Liang, Jenkins, Liz, Martin, Louis, Madaan, Lovish, Malo, Lubo, Blecher, Lukas, Landzaat, Lukas, de Oliveira, Luke, Muzzi, Madeline, Pasupuleti, Mahesh, Singh, Mannat, Paluri, Manohar, Kardas, Marcin, Tsimpoukelli, Maria, Oldham, Mathew, Rita, Mathieu, Pavlova, Maya, Kambadur, Melanie, Lewis, Mike, Si, Min, Singh, Mitesh Kumar, Hassan, Mona, Goyal, Naman, Torabi, Narjes, Bashlykov, Nikolay, Bogoychev, Nikolay, Chatterji, Niladri, Zhang, Ning, Duchenne, Olivier, Çelebi, Onur, Alrassy, Patrick, Zhang, Pengchuan, Li, Pengwei, Vasic, Petar, Weng, Peter, Bhargava, Prajjwal, Dubal, Pratik, Krishnan, Praveen, Koura, Punit Singh, Xu, Puxin, He, Qing, Dong, Qingxiao, Srinivasan, Ragavan, Ganapathy, Raj, Calderer, Ramon, Cabral, Ricardo Silveira, Stojnic, Robert, Raileanu, Roberta, Maheswari, Rohan, Girdhar, Rohit, Patel, Rohit, Sauvestre, Romain, Polidoro, Ronnie, Sumbaly, Roshan, Taylor, Ross, Silva, Ruan, Hou, Rui, Wang, Rui, Hosseini, Saghar, Chennabasappa, Sahana, Singh, Sanjay, Bell, Sean, Kim, Seohyun Sonia, Edunov, Sergey, Nie, Shaoliang, Narang, Sharan, Raparthy, Sharath, Shen, Sheng, Wan, Shengye, Bhosale, Shruti, Zhang, Shun, Vandenhende, Simon, Batra, Soumya, Whitman, Spencer, Sootla, Sten, Collot, Stephane, Gururangan, Suchin, Borodinsky, Sydney, Herman, Tamar, Fowler, Tara, Sheasha, Tarek, Georgiou, Thomas, Scialom, Thomas, Speckbacher, Tobias, Mihaylov, Todor, Xiao, Tong, Karn, Ujjwal, Goswami, Vedanuj, Gupta, Vibhor, Ramanathan, Vignesh, Kerkez, Viktor, Gonguet, Vincent, Do, Virginie, Vogeti, Vish, Albiero, Vítor, Petrovic, Vladan, Chu, Weiwei, Xiong, Wenhan, Fu, Wenyin, Meers, Whitney, Martinet, Xavier, Wang, Xiaodong, Wang, Xiaofang, Tan, Xiaoqing Ellen, Xia, Xide, Xie, Xinfeng, Jia, Xuchao, Wang, Xuewei, Goldschlag, Yaelle, Gaur, Yashesh, Babaei, Yasmine, Wen, Yi, Song, Yiwen, Zhang, Yuchen, Li, Yue, Mao, Yuning, Coudert, Zacharie Delpierre, Yan, Zheng, Chen, Zhengxing, Papakipos, Zoe, Singh, Aaditya, Srivastava, Aayushi, Jain, Abha, Kelsey, Adam, Shajnfeld, Adam, Gangidi, Adithya, Victoria, Adolfo, Goldstand, Ahuva, Menon, Ajay, Sharma, Ajay, Boesenberg, Alex, Baevski, Alexei, Feinstein, Allie, Kallet, Amanda, Sangani, Amit, Teo, Amos, Yunus, Anam, Lupu, Andrei, Alvarado, Andres, Caples, Andrew, Gu, Andrew, Ho, Andrew, Poulton, Andrew, Ryan, Andrew, Ramchandani, Ankit, Dong, Annie, Franco, Annie, Goyal, Anuj, Saraf, Aparajita, Chowdhury, Arkabandhu, Gabriel, Ashley, Bharambe, Ashwin, Eisenman, Assaf, Yazdan, Azadeh, James, Beau, Maurer, Ben, Leonhardi, Benjamin, Huang, Bernie, Loyd, Beth, De Paola, Beto, Paranjape, Bhargavi, Liu, Bing, Wu, Bo, Ni, Boyu, Hancock, Braden, Wasti, Bram, Spence, Brandon, Stojkovic, Brani, Gamido, Brian, Montalvo, Britt, Parker, Carl, Burton, Carly, Mejia, Catalina, Liu, Ce, Wang, Changhan, Kim, Changkyu, Zhou, Chao, Hu, Chester, Chu, Ching-Hsiang, Cai, Chris, Tindal, Chris, Feichtenhofer, Christoph, Gao, Cynthia, Civin, Damon, Beaty, Dana, Kreymer, Daniel, Li, Daniel, Adkins, David, Xu, David, Testuggine, Davide, David, Delia, Parikh, Devi, Liskovich, Diana, Foss, Didem, Wang, Dingkang, Le, Duc, Holland, Dustin, Dowling, Edward, Jamil, Eissa, Montgomery, Elaine, Presani, Eleonora, Hahn, Emily, Wood, Emily, Le, Eric-Tuan, Brinkman, Erik, Arcaute, Esteban, Dunbar, Evan, Smothers, Evan, Sun, Fei, Kreuk, Felix, Tian, Feng, Kokkinos, Filippos, Ozgenel, Firat, Caggioni, Francesco, Kanayet, Frank, Seide, Frank, Florez, Gabriela Medina, Schwarz, Gabriella, Badeer, Gada, Swee, Georgia, Halpern, Gil, Herman, Grant, Sizov, Grigory, Guangyi, Zhang, Lakshminarayanan, Guna, Inan, Hakan, Shojanazeri, Hamid, Zou, Han, Wang, Hannah, Zha, Hanwen, Habeeb, Haroun, Rudolph, Harrison, Suk, Helen, Aspegren, Henry, Goldman, Hunter, Zhan, Hongyuan, Damlaj, Ibrahim, Molybog, Igor, Tufanov, Igor, Leontiadis, Ilias, Veliche, Irina-Elena, Gat, Itai, Weissman, Jake, Geboski, James, Kohli, James, Lam, Janice, Asher, Japhet, Gaya, Jean-Baptiste, Marcus, Jeff, Tang, Jeff, Chan, Jennifer, Zhen, Jenny, Reizenstein, Jeremy, Teboul, Jeremy, Zhong, Jessica, Jin, Jian, Yang, Jingyi, Cummings, Joe, Carvill, Jon, Shepard, Jon, McPhie, Jonathan, Torres, Jonathan, Ginsburg, Josh, Wang, Junjie, Wu, Kai, U, Kam Hou, Saxena, Karan, Khandelwal, Kartikay, Zand, Katayoun, Matosich, Kathy, Veeraraghavan, Kaushik, Michelena, Kelly, Li, Keqian, Jagadeesh, Kiran, Huang, Kun, Chawla, Kunal, Huang, Kyle, Chen, Lailin, Garg, Lakshya, A, Lavender, Silva, Leandro, Bell, Lee, Zhang, Lei, Guo, Liangpeng, Yu, Licheng, Moshkovich, Liron, Wehrstedt, Luca, Khabsa, Madian, Avalani, Manav, Bhatt, Manish, Mankus, Martynas, Hasson, Matan, Lennie, Matthew, Reso, Matthias, Groshev, Maxim, Naumov, Maxim, Lathi, Maya, Keneally, Meghan, Liu, Miao, Seltzer, Michael L., Valko, Michal, Restrepo, Michelle, Patel, Mihir, Vyatskov, Mik, Samvelyan, Mikayel, Clark, Mike, Macey, Mike, Wang, Mike, Hermoso, Miquel Jubert, Metanat, Mo, Rastegari, Mohammad, Bansal, Munish, Santhanam, Nandhini, Parks, Natascha, White, Natasha, Bawa, Navyata, Singhal, Nayan, Egebo, Nick, Usunier, Nicolas, Mehta, Nikhil, Laptev, Nikolay Pavlovich, Dong, Ning, Cheng, Norman, Chernoguz, Oleg, Hart, Olivia, Salpekar, Omkar, Kalinli, Ozlem, Kent, Parkin, Parekh, Parth, Saab, Paul, Balaji, Pavan, Rittner, Pedro, Bontrager, Philip, Roux, Pierre, Dollar, Piotr, Zvyagina, Polina, Ratanchandani, Prashant, Yuvraj, Pritish, Liang, Qian, Alao, Rachad, Rodriguez, Rachel, Ayub, Rafi, Murthy, Raghotham, Nayani, Raghu, Mitra, Rahul, Parthasarathy, Rangaprabhu, Li, Raymond, Hogan, Rebekkah, Battey, Robin, Wang, Rocky, Howes, Russ, Rinott, Ruty, Mehta, Sachin, Siby, Sachin, Bondu, Sai Jayesh, Datta, Samyak, Chugh, Sara, Hunt, Sara, Dhillon, Sargun, Sidorov, Sasha, Pan, Satadru, Mahajan, Saurabh, Verma, Saurabh, Yamamoto, Seiji, Ramaswamy, Sharadh, Lindsay, Shaun, Feng, Sheng, Lin, Shenghao, Zha, Shengxin Cindy, Patil, Shishir, Shankar, Shiva, Zhang, Shuqiang, Wang, Sinong, Agarwal, Sneha, Sajuyigbe, Soji, Chintala, Soumith, Max, Stephanie, Chen, Stephen, Kehoe, Steve, Satterfield, Steve, Govindaprasad, Sudarshan, Gupta, Sumit, Deng, Summer, Cho, Sungmin, Virk, Sunny, Subramanian, Suraj, Choudhury, Sy, Goldman, Sydney, Remez, Tal, Glaser, Tamar, Best, Tamara, Koehler, Thilo, Robinson, Thomas, Li, Tianhe, Zhang, Tianjun, Matthews, Tim, Chou, Timothy, Shaked, Tzook, Vontimitta, Varun, Ajayi, Victoria, Montanez, Victoria, Mohan, Vijai, Kumar, Vinay Satish, Mangla, Vishal, Ionescu, Vlad, Poenaru, Vlad, Mihailescu, Vlad Tiberiu, Ivanov, Vladimir, Li, Wei, Wang, Wenchen, Jiang, Wenwen, Bouaziz, Wes, Constable, Will, Tang, Xiaocheng, Wu, Xiaojian, Wang, Xiaolan, Wu, Xilun, Gao, Xinbo, Kleinman, Yaniv, Chen, Yanjun, Hu, Ye, Jia, Ye, Qi, Ye, Li, Yenda, Zhang, Yilin, Zhang, Ying, Adi, Yossi, Nam, Youngjin, Yu, Wang, Zhao, Yu, Hao, Yuchen, Qian, Yundi, Li, Yunlu, He, Yuzi, Rait, Zach, DeVito, Zachary, Rosnbrick, Zef, Wen, Zhaoduo, Yang, Zhenyu, Zhao, Zhiwei, Ma, Zhiyu
Format:	Preprint
Published:	2024
Subjects:	Artificial Intelligence Computation and Language Computer Vision and Pattern Recognition
Online Access:	https://arxiv.org/abs/2407.21783
Tags:	Add Tag No Tags, Be the first to tag this record!

_version_	1866909402297008128
author	Grattafiori, Aaron Dubey, Abhimanyu Jauhri, Abhinav Pandey, Abhinav Kadian, Abhishek Al-Dahle, Ahmad Letman, Aiesha Mathur, Akhil Schelten, Alan Vaughan, Alex Yang, Amy Fan, Angela Goyal, Anirudh Hartshorn, Anthony Yang, Aobo Mitra, Archi Sravankumar, Archie Korenev, Artem Hinsvark, Arthur Rao, Arun Zhang, Aston Rodriguez, Aurelien Gregerson, Austen Spataru, Ava Roziere, Baptiste Biron, Bethany Tang, Binh Chern, Bobbie Caucheteux, Charlotte Nayak, Chaya Bi, Chloe Marra, Chris McConnell, Chris Keller, Christian Touret, Christophe Wu, Chunyang Wong, Corinne Ferrer, Cristian Canton Nikolaidis, Cyrus Allonsius, Damien Song, Daniel Pintz, Danielle Livshits, Danny Wyatt, Danny Esiobu, David Choudhary, Dhruv Mahajan, Dhruv Garcia-Olano, Diego Perino, Diego Hupkes, Dieuwke Lakomkin, Egor AlBadawy, Ehab Lobanova, Elina Dinan, Emily Smith, Eric Michael Radenovic, Filip Guzmán, Francisco Zhang, Frank Synnaeve, Gabriel Lee, Gabrielle Anderson, Georgia Lewis Thattai, Govind Nail, Graeme Mialon, Gregoire Pang, Guan Cucurell, Guillem Nguyen, Hailey Korevaar, Hannah Xu, Hu Touvron, Hugo Zarov, Iliyan Ibarra, Imanol Arrieta Kloumann, Isabel Misra, Ishan Evtimov, Ivan Zhang, Jack Copet, Jade Lee, Jaewon Geffert, Jan Vranes, Jana Park, Jason Mahadeokar, Jay Shah, Jeet van der Linde, Jelmer Billock, Jennifer Hong, Jenny Lee, Jenya Fu, Jeremy Chi, Jianfeng Huang, Jianyu Liu, Jiawen Wang, Jie Yu, Jiecao Bitton, Joanna Spisak, Joe Park, Jongsoo Rocca, Joseph Johnstun, Joshua Saxe, Joshua Jia, Junteng Alwala, Kalyan Vasuden Prasad, Karthik Upasani, Kartikeya Plawiak, Kate Li, Ke Heafield, Kenneth Stone, Kevin El-Arini, Khalid Iyer, Krithika Malik, Kshitiz Chiu, Kuenley Bhalla, Kunal Lakhotia, Kushal Rantala-Yeary, Lauren van der Maaten, Laurens Chen, Lawrence Tan, Liang Jenkins, Liz Martin, Louis Madaan, Lovish Malo, Lubo Blecher, Lukas Landzaat, Lukas de Oliveira, Luke Muzzi, Madeline Pasupuleti, Mahesh Singh, Mannat Paluri, Manohar Kardas, Marcin Tsimpoukelli, Maria Oldham, Mathew Rita, Mathieu Pavlova, Maya Kambadur, Melanie Lewis, Mike Si, Min Singh, Mitesh Kumar Hassan, Mona Goyal, Naman Torabi, Narjes Bashlykov, Nikolay Bogoychev, Nikolay Chatterji, Niladri Zhang, Ning Duchenne, Olivier Çelebi, Onur Alrassy, Patrick Zhang, Pengchuan Li, Pengwei Vasic, Petar Weng, Peter Bhargava, Prajjwal Dubal, Pratik Krishnan, Praveen Koura, Punit Singh Xu, Puxin He, Qing Dong, Qingxiao Srinivasan, Ragavan Ganapathy, Raj Calderer, Ramon Cabral, Ricardo Silveira Stojnic, Robert Raileanu, Roberta Maheswari, Rohan Girdhar, Rohit Patel, Rohit Sauvestre, Romain Polidoro, Ronnie Sumbaly, Roshan Taylor, Ross Silva, Ruan Hou, Rui Wang, Rui Hosseini, Saghar Chennabasappa, Sahana Singh, Sanjay Bell, Sean Kim, Seohyun Sonia Edunov, Sergey Nie, Shaoliang Narang, Sharan Raparthy, Sharath Shen, Sheng Wan, Shengye Bhosale, Shruti Zhang, Shun Vandenhende, Simon Batra, Soumya Whitman, Spencer Sootla, Sten Collot, Stephane Gururangan, Suchin Borodinsky, Sydney Herman, Tamar Fowler, Tara Sheasha, Tarek Georgiou, Thomas Scialom, Thomas Speckbacher, Tobias Mihaylov, Todor Xiao, Tong Karn, Ujjwal Goswami, Vedanuj Gupta, Vibhor Ramanathan, Vignesh Kerkez, Viktor Gonguet, Vincent Do, Virginie Vogeti, Vish Albiero, Vítor Petrovic, Vladan Chu, Weiwei Xiong, Wenhan Fu, Wenyin Meers, Whitney Martinet, Xavier Wang, Xiaodong Wang, Xiaofang Tan, Xiaoqing Ellen Xia, Xide Xie, Xinfeng Jia, Xuchao Wang, Xuewei Goldschlag, Yaelle Gaur, Yashesh Babaei, Yasmine Wen, Yi Song, Yiwen Zhang, Yuchen Li, Yue Mao, Yuning Coudert, Zacharie Delpierre Yan, Zheng Chen, Zhengxing Papakipos, Zoe Singh, Aaditya Srivastava, Aayushi Jain, Abha Kelsey, Adam Shajnfeld, Adam Gangidi, Adithya Victoria, Adolfo Goldstand, Ahuva Menon, Ajay Sharma, Ajay Boesenberg, Alex Baevski, Alexei Feinstein, Allie Kallet, Amanda Sangani, Amit Teo, Amos Yunus, Anam Lupu, Andrei Alvarado, Andres Caples, Andrew Gu, Andrew Ho, Andrew Poulton, Andrew Ryan, Andrew Ramchandani, Ankit Dong, Annie Franco, Annie Goyal, Anuj Saraf, Aparajita Chowdhury, Arkabandhu Gabriel, Ashley Bharambe, Ashwin Eisenman, Assaf Yazdan, Azadeh James, Beau Maurer, Ben Leonhardi, Benjamin Huang, Bernie Loyd, Beth De Paola, Beto Paranjape, Bhargavi Liu, Bing Wu, Bo Ni, Boyu Hancock, Braden Wasti, Bram Spence, Brandon Stojkovic, Brani Gamido, Brian Montalvo, Britt Parker, Carl Burton, Carly Mejia, Catalina Liu, Ce Wang, Changhan Kim, Changkyu Zhou, Chao Hu, Chester Chu, Ching-Hsiang Cai, Chris Tindal, Chris Feichtenhofer, Christoph Gao, Cynthia Civin, Damon Beaty, Dana Kreymer, Daniel Li, Daniel Adkins, David Xu, David Testuggine, Davide David, Delia Parikh, Devi Liskovich, Diana Foss, Didem Wang, Dingkang Le, Duc Holland, Dustin Dowling, Edward Jamil, Eissa Montgomery, Elaine Presani, Eleonora Hahn, Emily Wood, Emily Le, Eric-Tuan Brinkman, Erik Arcaute, Esteban Dunbar, Evan Smothers, Evan Sun, Fei Kreuk, Felix Tian, Feng Kokkinos, Filippos Ozgenel, Firat Caggioni, Francesco Kanayet, Frank Seide, Frank Florez, Gabriela Medina Schwarz, Gabriella Badeer, Gada Swee, Georgia Halpern, Gil Herman, Grant Sizov, Grigory Guangyi Zhang Lakshminarayanan, Guna Inan, Hakan Shojanazeri, Hamid Zou, Han Wang, Hannah Zha, Hanwen Habeeb, Haroun Rudolph, Harrison Suk, Helen Aspegren, Henry Goldman, Hunter Zhan, Hongyuan Damlaj, Ibrahim Molybog, Igor Tufanov, Igor Leontiadis, Ilias Veliche, Irina-Elena Gat, Itai Weissman, Jake Geboski, James Kohli, James Lam, Janice Asher, Japhet Gaya, Jean-Baptiste Marcus, Jeff Tang, Jeff Chan, Jennifer Zhen, Jenny Reizenstein, Jeremy Teboul, Jeremy Zhong, Jessica Jin, Jian Yang, Jingyi Cummings, Joe Carvill, Jon Shepard, Jon McPhie, Jonathan Torres, Jonathan Ginsburg, Josh Wang, Junjie Wu, Kai U, Kam Hou Saxena, Karan Khandelwal, Kartikay Zand, Katayoun Matosich, Kathy Veeraraghavan, Kaushik Michelena, Kelly Li, Keqian Jagadeesh, Kiran Huang, Kun Chawla, Kunal Huang, Kyle Chen, Lailin Garg, Lakshya A, Lavender Silva, Leandro Bell, Lee Zhang, Lei Guo, Liangpeng Yu, Licheng Moshkovich, Liron Wehrstedt, Luca Khabsa, Madian Avalani, Manav Bhatt, Manish Mankus, Martynas Hasson, Matan Lennie, Matthew Reso, Matthias Groshev, Maxim Naumov, Maxim Lathi, Maya Keneally, Meghan Liu, Miao Seltzer, Michael L. Valko, Michal Restrepo, Michelle Patel, Mihir Vyatskov, Mik Samvelyan, Mikayel Clark, Mike Macey, Mike Wang, Mike Hermoso, Miquel Jubert Metanat, Mo Rastegari, Mohammad Bansal, Munish Santhanam, Nandhini Parks, Natascha White, Natasha Bawa, Navyata Singhal, Nayan Egebo, Nick Usunier, Nicolas Mehta, Nikhil Laptev, Nikolay Pavlovich Dong, Ning Cheng, Norman Chernoguz, Oleg Hart, Olivia Salpekar, Omkar Kalinli, Ozlem Kent, Parkin Parekh, Parth Saab, Paul Balaji, Pavan Rittner, Pedro Bontrager, Philip Roux, Pierre Dollar, Piotr Zvyagina, Polina Ratanchandani, Prashant Yuvraj, Pritish Liang, Qian Alao, Rachad Rodriguez, Rachel Ayub, Rafi Murthy, Raghotham Nayani, Raghu Mitra, Rahul Parthasarathy, Rangaprabhu Li, Raymond Hogan, Rebekkah Battey, Robin Wang, Rocky Howes, Russ Rinott, Ruty Mehta, Sachin Siby, Sachin Bondu, Sai Jayesh Datta, Samyak Chugh, Sara Hunt, Sara Dhillon, Sargun Sidorov, Sasha Pan, Satadru Mahajan, Saurabh Verma, Saurabh Yamamoto, Seiji Ramaswamy, Sharadh Lindsay, Shaun Lindsay, Shaun Feng, Sheng Lin, Shenghao Zha, Shengxin Cindy Patil, Shishir Shankar, Shiva Zhang, Shuqiang Zhang, Shuqiang Wang, Sinong Agarwal, Sneha Sajuyigbe, Soji Chintala, Soumith Max, Stephanie Chen, Stephen Kehoe, Steve Satterfield, Steve Govindaprasad, Sudarshan Gupta, Sumit Deng, Summer Cho, Sungmin Virk, Sunny Subramanian, Suraj Choudhury, Sy Goldman, Sydney Remez, Tal Glaser, Tamar Best, Tamara Koehler, Thilo Robinson, Thomas Li, Tianhe Zhang, Tianjun Matthews, Tim Chou, Timothy Shaked, Tzook Vontimitta, Varun Ajayi, Victoria Montanez, Victoria Mohan, Vijai Kumar, Vinay Satish Mangla, Vishal Ionescu, Vlad Poenaru, Vlad Mihailescu, Vlad Tiberiu Ivanov, Vladimir Li, Wei Wang, Wenchen Jiang, Wenwen Bouaziz, Wes Constable, Will Tang, Xiaocheng Wu, Xiaojian Wang, Xiaolan Wu, Xilun Gao, Xinbo Kleinman, Yaniv Chen, Yanjun Hu, Ye Jia, Ye Qi, Ye Li, Yenda Zhang, Yilin Zhang, Ying Adi, Yossi Nam, Youngjin Yu Wang Zhao, Yu Hao, Yuchen Qian, Yundi Li, Yunlu He, Yuzi Rait, Zach DeVito, Zachary Rosnbrick, Zef Wen, Zhaoduo Yang, Zhenyu Zhao, Zhiwei Ma, Zhiyu
author_facet	Grattafiori, Aaron Dubey, Abhimanyu Jauhri, Abhinav Pandey, Abhinav Kadian, Abhishek Al-Dahle, Ahmad Letman, Aiesha Mathur, Akhil Schelten, Alan Vaughan, Alex Yang, Amy Fan, Angela Goyal, Anirudh Hartshorn, Anthony Yang, Aobo Mitra, Archi Sravankumar, Archie Korenev, Artem Hinsvark, Arthur Rao, Arun Zhang, Aston Rodriguez, Aurelien Gregerson, Austen Spataru, Ava Roziere, Baptiste Biron, Bethany Tang, Binh Chern, Bobbie Caucheteux, Charlotte Nayak, Chaya Bi, Chloe Marra, Chris McConnell, Chris Keller, Christian Touret, Christophe Wu, Chunyang Wong, Corinne Ferrer, Cristian Canton Nikolaidis, Cyrus Allonsius, Damien Song, Daniel Pintz, Danielle Livshits, Danny Wyatt, Danny Esiobu, David Choudhary, Dhruv Mahajan, Dhruv Garcia-Olano, Diego Perino, Diego Hupkes, Dieuwke Lakomkin, Egor AlBadawy, Ehab Lobanova, Elina Dinan, Emily Smith, Eric Michael Radenovic, Filip Guzmán, Francisco Zhang, Frank Synnaeve, Gabriel Lee, Gabrielle Anderson, Georgia Lewis Thattai, Govind Nail, Graeme Mialon, Gregoire Pang, Guan Cucurell, Guillem Nguyen, Hailey Korevaar, Hannah Xu, Hu Touvron, Hugo Zarov, Iliyan Ibarra, Imanol Arrieta Kloumann, Isabel Misra, Ishan Evtimov, Ivan Zhang, Jack Copet, Jade Lee, Jaewon Geffert, Jan Vranes, Jana Park, Jason Mahadeokar, Jay Shah, Jeet van der Linde, Jelmer Billock, Jennifer Hong, Jenny Lee, Jenya Fu, Jeremy Chi, Jianfeng Huang, Jianyu Liu, Jiawen Wang, Jie Yu, Jiecao Bitton, Joanna Spisak, Joe Park, Jongsoo Rocca, Joseph Johnstun, Joshua Saxe, Joshua Jia, Junteng Alwala, Kalyan Vasuden Prasad, Karthik Upasani, Kartikeya Plawiak, Kate Li, Ke Heafield, Kenneth Stone, Kevin El-Arini, Khalid Iyer, Krithika Malik, Kshitiz Chiu, Kuenley Bhalla, Kunal Lakhotia, Kushal Rantala-Yeary, Lauren van der Maaten, Laurens Chen, Lawrence Tan, Liang Jenkins, Liz Martin, Louis Madaan, Lovish Malo, Lubo Blecher, Lukas Landzaat, Lukas de Oliveira, Luke Muzzi, Madeline Pasupuleti, Mahesh Singh, Mannat Paluri, Manohar Kardas, Marcin Tsimpoukelli, Maria Oldham, Mathew Rita, Mathieu Pavlova, Maya Kambadur, Melanie Lewis, Mike Si, Min Singh, Mitesh Kumar Hassan, Mona Goyal, Naman Torabi, Narjes Bashlykov, Nikolay Bogoychev, Nikolay Chatterji, Niladri Zhang, Ning Duchenne, Olivier Çelebi, Onur Alrassy, Patrick Zhang, Pengchuan Li, Pengwei Vasic, Petar Weng, Peter Bhargava, Prajjwal Dubal, Pratik Krishnan, Praveen Koura, Punit Singh Xu, Puxin He, Qing Dong, Qingxiao Srinivasan, Ragavan Ganapathy, Raj Calderer, Ramon Cabral, Ricardo Silveira Stojnic, Robert Raileanu, Roberta Maheswari, Rohan Girdhar, Rohit Patel, Rohit Sauvestre, Romain Polidoro, Ronnie Sumbaly, Roshan Taylor, Ross Silva, Ruan Hou, Rui Wang, Rui Hosseini, Saghar Chennabasappa, Sahana Singh, Sanjay Bell, Sean Kim, Seohyun Sonia Edunov, Sergey Nie, Shaoliang Narang, Sharan Raparthy, Sharath Shen, Sheng Wan, Shengye Bhosale, Shruti Zhang, Shun Vandenhende, Simon Batra, Soumya Whitman, Spencer Sootla, Sten Collot, Stephane Gururangan, Suchin Borodinsky, Sydney Herman, Tamar Fowler, Tara Sheasha, Tarek Georgiou, Thomas Scialom, Thomas Speckbacher, Tobias Mihaylov, Todor Xiao, Tong Karn, Ujjwal Goswami, Vedanuj Gupta, Vibhor Ramanathan, Vignesh Kerkez, Viktor Gonguet, Vincent Do, Virginie Vogeti, Vish Albiero, Vítor Petrovic, Vladan Chu, Weiwei Xiong, Wenhan Fu, Wenyin Meers, Whitney Martinet, Xavier Wang, Xiaodong Wang, Xiaofang Tan, Xiaoqing Ellen Xia, Xide Xie, Xinfeng Jia, Xuchao Wang, Xuewei Goldschlag, Yaelle Gaur, Yashesh Babaei, Yasmine Wen, Yi Song, Yiwen Zhang, Yuchen Li, Yue Mao, Yuning Coudert, Zacharie Delpierre Yan, Zheng Chen, Zhengxing Papakipos, Zoe Singh, Aaditya Srivastava, Aayushi Jain, Abha Kelsey, Adam Shajnfeld, Adam Gangidi, Adithya Victoria, Adolfo Goldstand, Ahuva Menon, Ajay Sharma, Ajay Boesenberg, Alex Baevski, Alexei Feinstein, Allie Kallet, Amanda Sangani, Amit Teo, Amos Yunus, Anam Lupu, Andrei Alvarado, Andres Caples, Andrew Gu, Andrew Ho, Andrew Poulton, Andrew Ryan, Andrew Ramchandani, Ankit Dong, Annie Franco, Annie Goyal, Anuj Saraf, Aparajita Chowdhury, Arkabandhu Gabriel, Ashley Bharambe, Ashwin Eisenman, Assaf Yazdan, Azadeh James, Beau Maurer, Ben Leonhardi, Benjamin Huang, Bernie Loyd, Beth De Paola, Beto Paranjape, Bhargavi Liu, Bing Wu, Bo Ni, Boyu Hancock, Braden Wasti, Bram Spence, Brandon Stojkovic, Brani Gamido, Brian Montalvo, Britt Parker, Carl Burton, Carly Mejia, Catalina Liu, Ce Wang, Changhan Kim, Changkyu Zhou, Chao Hu, Chester Chu, Ching-Hsiang Cai, Chris Tindal, Chris Feichtenhofer, Christoph Gao, Cynthia Civin, Damon Beaty, Dana Kreymer, Daniel Li, Daniel Adkins, David Xu, David Testuggine, Davide David, Delia Parikh, Devi Liskovich, Diana Foss, Didem Wang, Dingkang Le, Duc Holland, Dustin Dowling, Edward Jamil, Eissa Montgomery, Elaine Presani, Eleonora Hahn, Emily Wood, Emily Le, Eric-Tuan Brinkman, Erik Arcaute, Esteban Dunbar, Evan Smothers, Evan Sun, Fei Kreuk, Felix Tian, Feng Kokkinos, Filippos Ozgenel, Firat Caggioni, Francesco Kanayet, Frank Seide, Frank Florez, Gabriela Medina Schwarz, Gabriella Badeer, Gada Swee, Georgia Halpern, Gil Herman, Grant Sizov, Grigory Guangyi Zhang Lakshminarayanan, Guna Inan, Hakan Shojanazeri, Hamid Zou, Han Wang, Hannah Zha, Hanwen Habeeb, Haroun Rudolph, Harrison Suk, Helen Aspegren, Henry Goldman, Hunter Zhan, Hongyuan Damlaj, Ibrahim Molybog, Igor Tufanov, Igor Leontiadis, Ilias Veliche, Irina-Elena Gat, Itai Weissman, Jake Geboski, James Kohli, James Lam, Janice Asher, Japhet Gaya, Jean-Baptiste Marcus, Jeff Tang, Jeff Chan, Jennifer Zhen, Jenny Reizenstein, Jeremy Teboul, Jeremy Zhong, Jessica Jin, Jian Yang, Jingyi Cummings, Joe Carvill, Jon Shepard, Jon McPhie, Jonathan Torres, Jonathan Ginsburg, Josh Wang, Junjie Wu, Kai U, Kam Hou Saxena, Karan Khandelwal, Kartikay Zand, Katayoun Matosich, Kathy Veeraraghavan, Kaushik Michelena, Kelly Li, Keqian Jagadeesh, Kiran Huang, Kun Chawla, Kunal Huang, Kyle Chen, Lailin Garg, Lakshya A, Lavender Silva, Leandro Bell, Lee Zhang, Lei Guo, Liangpeng Yu, Licheng Moshkovich, Liron Wehrstedt, Luca Khabsa, Madian Avalani, Manav Bhatt, Manish Mankus, Martynas Hasson, Matan Lennie, Matthew Reso, Matthias Groshev, Maxim Naumov, Maxim Lathi, Maya Keneally, Meghan Liu, Miao Seltzer, Michael L. Valko, Michal Restrepo, Michelle Patel, Mihir Vyatskov, Mik Samvelyan, Mikayel Clark, Mike Macey, Mike Wang, Mike Hermoso, Miquel Jubert Metanat, Mo Rastegari, Mohammad Bansal, Munish Santhanam, Nandhini Parks, Natascha White, Natasha Bawa, Navyata Singhal, Nayan Egebo, Nick Usunier, Nicolas Mehta, Nikhil Laptev, Nikolay Pavlovich Dong, Ning Cheng, Norman Chernoguz, Oleg Hart, Olivia Salpekar, Omkar Kalinli, Ozlem Kent, Parkin Parekh, Parth Saab, Paul Balaji, Pavan Rittner, Pedro Bontrager, Philip Roux, Pierre Dollar, Piotr Zvyagina, Polina Ratanchandani, Prashant Yuvraj, Pritish Liang, Qian Alao, Rachad Rodriguez, Rachel Ayub, Rafi Murthy, Raghotham Nayani, Raghu Mitra, Rahul Parthasarathy, Rangaprabhu Li, Raymond Hogan, Rebekkah Battey, Robin Wang, Rocky Howes, Russ Rinott, Ruty Mehta, Sachin Siby, Sachin Bondu, Sai Jayesh Datta, Samyak Chugh, Sara Hunt, Sara Dhillon, Sargun Sidorov, Sasha Pan, Satadru Mahajan, Saurabh Verma, Saurabh Yamamoto, Seiji Ramaswamy, Sharadh Lindsay, Shaun Lindsay, Shaun Feng, Sheng Lin, Shenghao Zha, Shengxin Cindy Patil, Shishir Shankar, Shiva Zhang, Shuqiang Zhang, Shuqiang Wang, Sinong Agarwal, Sneha Sajuyigbe, Soji Chintala, Soumith Max, Stephanie Chen, Stephen Kehoe, Steve Satterfield, Steve Govindaprasad, Sudarshan Gupta, Sumit Deng, Summer Cho, Sungmin Virk, Sunny Subramanian, Suraj Choudhury, Sy Goldman, Sydney Remez, Tal Glaser, Tamar Best, Tamara Koehler, Thilo Robinson, Thomas Li, Tianhe Zhang, Tianjun Matthews, Tim Chou, Timothy Shaked, Tzook Vontimitta, Varun Ajayi, Victoria Montanez, Victoria Mohan, Vijai Kumar, Vinay Satish Mangla, Vishal Ionescu, Vlad Poenaru, Vlad Mihailescu, Vlad Tiberiu Ivanov, Vladimir Li, Wei Wang, Wenchen Jiang, Wenwen Bouaziz, Wes Constable, Will Tang, Xiaocheng Wu, Xiaojian Wang, Xiaolan Wu, Xilun Gao, Xinbo Kleinman, Yaniv Chen, Yanjun Hu, Ye Jia, Ye Qi, Ye Li, Yenda Zhang, Yilin Zhang, Ying Adi, Yossi Nam, Youngjin Yu Wang Zhao, Yu Hao, Yuchen Qian, Yundi Li, Yunlu He, Yuzi Rait, Zach DeVito, Zachary Rosnbrick, Zef Wen, Zhaoduo Yang, Zhenyu Zhao, Zhiwei Ma, Zhiyu
contents	Modern artificial intelligence (AI) systems are powered by foundation models. This paper presents a new set of foundation models, called Llama 3. It is a herd of language models that natively support multilinguality, coding, reasoning, and tool usage. Our largest model is a dense Transformer with 405B parameters and a context window of up to 128K tokens. This paper presents an extensive empirical evaluation of Llama 3. We find that Llama 3 delivers comparable quality to leading language models such as GPT-4 on a plethora of tasks. We publicly release Llama 3, including pre-trained and post-trained versions of the 405B parameter language model and our Llama Guard 3 model for input and output safety. The paper also presents the results of experiments in which we integrate image, video, and speech capabilities into Llama 3 via a compositional approach. We observe this approach performs competitively with the state-of-the-art on image, video, and speech recognition tasks. The resulting models are not yet being broadly released as they are still under development.
format	Preprint
id	arxiv_https___arxiv_org_abs_2407_21783
institution	arXiv
publishDate	2024
record_format	arxiv
spellingShingle	The Llama 3 Herd of Models Grattafiori, Aaron Dubey, Abhimanyu Jauhri, Abhinav Pandey, Abhinav Kadian, Abhishek Al-Dahle, Ahmad Letman, Aiesha Mathur, Akhil Schelten, Alan Vaughan, Alex Yang, Amy Fan, Angela Goyal, Anirudh Hartshorn, Anthony Yang, Aobo Mitra, Archi Sravankumar, Archie Korenev, Artem Hinsvark, Arthur Rao, Arun Zhang, Aston Rodriguez, Aurelien Gregerson, Austen Spataru, Ava Roziere, Baptiste Biron, Bethany Tang, Binh Chern, Bobbie Caucheteux, Charlotte Nayak, Chaya Bi, Chloe Marra, Chris McConnell, Chris Keller, Christian Touret, Christophe Wu, Chunyang Wong, Corinne Ferrer, Cristian Canton Nikolaidis, Cyrus Allonsius, Damien Song, Daniel Pintz, Danielle Livshits, Danny Wyatt, Danny Esiobu, David Choudhary, Dhruv Mahajan, Dhruv Garcia-Olano, Diego Perino, Diego Hupkes, Dieuwke Lakomkin, Egor AlBadawy, Ehab Lobanova, Elina Dinan, Emily Smith, Eric Michael Radenovic, Filip Guzmán, Francisco Zhang, Frank Synnaeve, Gabriel Lee, Gabrielle Anderson, Georgia Lewis Thattai, Govind Nail, Graeme Mialon, Gregoire Pang, Guan Cucurell, Guillem Nguyen, Hailey Korevaar, Hannah Xu, Hu Touvron, Hugo Zarov, Iliyan Ibarra, Imanol Arrieta Kloumann, Isabel Misra, Ishan Evtimov, Ivan Zhang, Jack Copet, Jade Lee, Jaewon Geffert, Jan Vranes, Jana Park, Jason Mahadeokar, Jay Shah, Jeet van der Linde, Jelmer Billock, Jennifer Hong, Jenny Lee, Jenya Fu, Jeremy Chi, Jianfeng Huang, Jianyu Liu, Jiawen Wang, Jie Yu, Jiecao Bitton, Joanna Spisak, Joe Park, Jongsoo Rocca, Joseph Johnstun, Joshua Saxe, Joshua Jia, Junteng Alwala, Kalyan Vasuden Prasad, Karthik Upasani, Kartikeya Plawiak, Kate Li, Ke Heafield, Kenneth Stone, Kevin El-Arini, Khalid Iyer, Krithika Malik, Kshitiz Chiu, Kuenley Bhalla, Kunal Lakhotia, Kushal Rantala-Yeary, Lauren van der Maaten, Laurens Chen, Lawrence Tan, Liang Jenkins, Liz Martin, Louis Madaan, Lovish Malo, Lubo Blecher, Lukas Landzaat, Lukas de Oliveira, Luke Muzzi, Madeline Pasupuleti, Mahesh Singh, Mannat Paluri, Manohar Kardas, Marcin Tsimpoukelli, Maria Oldham, Mathew Rita, Mathieu Pavlova, Maya Kambadur, Melanie Lewis, Mike Si, Min Singh, Mitesh Kumar Hassan, Mona Goyal, Naman Torabi, Narjes Bashlykov, Nikolay Bogoychev, Nikolay Chatterji, Niladri Zhang, Ning Duchenne, Olivier Çelebi, Onur Alrassy, Patrick Zhang, Pengchuan Li, Pengwei Vasic, Petar Weng, Peter Bhargava, Prajjwal Dubal, Pratik Krishnan, Praveen Koura, Punit Singh Xu, Puxin He, Qing Dong, Qingxiao Srinivasan, Ragavan Ganapathy, Raj Calderer, Ramon Cabral, Ricardo Silveira Stojnic, Robert Raileanu, Roberta Maheswari, Rohan Girdhar, Rohit Patel, Rohit Sauvestre, Romain Polidoro, Ronnie Sumbaly, Roshan Taylor, Ross Silva, Ruan Hou, Rui Wang, Rui Hosseini, Saghar Chennabasappa, Sahana Singh, Sanjay Bell, Sean Kim, Seohyun Sonia Edunov, Sergey Nie, Shaoliang Narang, Sharan Raparthy, Sharath Shen, Sheng Wan, Shengye Bhosale, Shruti Zhang, Shun Vandenhende, Simon Batra, Soumya Whitman, Spencer Sootla, Sten Collot, Stephane Gururangan, Suchin Borodinsky, Sydney Herman, Tamar Fowler, Tara Sheasha, Tarek Georgiou, Thomas Scialom, Thomas Speckbacher, Tobias Mihaylov, Todor Xiao, Tong Karn, Ujjwal Goswami, Vedanuj Gupta, Vibhor Ramanathan, Vignesh Kerkez, Viktor Gonguet, Vincent Do, Virginie Vogeti, Vish Albiero, Vítor Petrovic, Vladan Chu, Weiwei Xiong, Wenhan Fu, Wenyin Meers, Whitney Martinet, Xavier Wang, Xiaodong Wang, Xiaofang Tan, Xiaoqing Ellen Xia, Xide Xie, Xinfeng Jia, Xuchao Wang, Xuewei Goldschlag, Yaelle Gaur, Yashesh Babaei, Yasmine Wen, Yi Song, Yiwen Zhang, Yuchen Li, Yue Mao, Yuning Coudert, Zacharie Delpierre Yan, Zheng Chen, Zhengxing Papakipos, Zoe Singh, Aaditya Srivastava, Aayushi Jain, Abha Kelsey, Adam Shajnfeld, Adam Gangidi, Adithya Victoria, Adolfo Goldstand, Ahuva Menon, Ajay Sharma, Ajay Boesenberg, Alex Baevski, Alexei Feinstein, Allie Kallet, Amanda Sangani, Amit Teo, Amos Yunus, Anam Lupu, Andrei Alvarado, Andres Caples, Andrew Gu, Andrew Ho, Andrew Poulton, Andrew Ryan, Andrew Ramchandani, Ankit Dong, Annie Franco, Annie Goyal, Anuj Saraf, Aparajita Chowdhury, Arkabandhu Gabriel, Ashley Bharambe, Ashwin Eisenman, Assaf Yazdan, Azadeh James, Beau Maurer, Ben Leonhardi, Benjamin Huang, Bernie Loyd, Beth De Paola, Beto Paranjape, Bhargavi Liu, Bing Wu, Bo Ni, Boyu Hancock, Braden Wasti, Bram Spence, Brandon Stojkovic, Brani Gamido, Brian Montalvo, Britt Parker, Carl Burton, Carly Mejia, Catalina Liu, Ce Wang, Changhan Kim, Changkyu Zhou, Chao Hu, Chester Chu, Ching-Hsiang Cai, Chris Tindal, Chris Feichtenhofer, Christoph Gao, Cynthia Civin, Damon Beaty, Dana Kreymer, Daniel Li, Daniel Adkins, David Xu, David Testuggine, Davide David, Delia Parikh, Devi Liskovich, Diana Foss, Didem Wang, Dingkang Le, Duc Holland, Dustin Dowling, Edward Jamil, Eissa Montgomery, Elaine Presani, Eleonora Hahn, Emily Wood, Emily Le, Eric-Tuan Brinkman, Erik Arcaute, Esteban Dunbar, Evan Smothers, Evan Sun, Fei Kreuk, Felix Tian, Feng Kokkinos, Filippos Ozgenel, Firat Caggioni, Francesco Kanayet, Frank Seide, Frank Florez, Gabriela Medina Schwarz, Gabriella Badeer, Gada Swee, Georgia Halpern, Gil Herman, Grant Sizov, Grigory Guangyi Zhang Lakshminarayanan, Guna Inan, Hakan Shojanazeri, Hamid Zou, Han Wang, Hannah Zha, Hanwen Habeeb, Haroun Rudolph, Harrison Suk, Helen Aspegren, Henry Goldman, Hunter Zhan, Hongyuan Damlaj, Ibrahim Molybog, Igor Tufanov, Igor Leontiadis, Ilias Veliche, Irina-Elena Gat, Itai Weissman, Jake Geboski, James Kohli, James Lam, Janice Asher, Japhet Gaya, Jean-Baptiste Marcus, Jeff Tang, Jeff Chan, Jennifer Zhen, Jenny Reizenstein, Jeremy Teboul, Jeremy Zhong, Jessica Jin, Jian Yang, Jingyi Cummings, Joe Carvill, Jon Shepard, Jon McPhie, Jonathan Torres, Jonathan Ginsburg, Josh Wang, Junjie Wu, Kai U, Kam Hou Saxena, Karan Khandelwal, Kartikay Zand, Katayoun Matosich, Kathy Veeraraghavan, Kaushik Michelena, Kelly Li, Keqian Jagadeesh, Kiran Huang, Kun Chawla, Kunal Huang, Kyle Chen, Lailin Garg, Lakshya A, Lavender Silva, Leandro Bell, Lee Zhang, Lei Guo, Liangpeng Yu, Licheng Moshkovich, Liron Wehrstedt, Luca Khabsa, Madian Avalani, Manav Bhatt, Manish Mankus, Martynas Hasson, Matan Lennie, Matthew Reso, Matthias Groshev, Maxim Naumov, Maxim Lathi, Maya Keneally, Meghan Liu, Miao Seltzer, Michael L. Valko, Michal Restrepo, Michelle Patel, Mihir Vyatskov, Mik Samvelyan, Mikayel Clark, Mike Macey, Mike Wang, Mike Hermoso, Miquel Jubert Metanat, Mo Rastegari, Mohammad Bansal, Munish Santhanam, Nandhini Parks, Natascha White, Natasha Bawa, Navyata Singhal, Nayan Egebo, Nick Usunier, Nicolas Mehta, Nikhil Laptev, Nikolay Pavlovich Dong, Ning Cheng, Norman Chernoguz, Oleg Hart, Olivia Salpekar, Omkar Kalinli, Ozlem Kent, Parkin Parekh, Parth Saab, Paul Balaji, Pavan Rittner, Pedro Bontrager, Philip Roux, Pierre Dollar, Piotr Zvyagina, Polina Ratanchandani, Prashant Yuvraj, Pritish Liang, Qian Alao, Rachad Rodriguez, Rachel Ayub, Rafi Murthy, Raghotham Nayani, Raghu Mitra, Rahul Parthasarathy, Rangaprabhu Li, Raymond Hogan, Rebekkah Battey, Robin Wang, Rocky Howes, Russ Rinott, Ruty Mehta, Sachin Siby, Sachin Bondu, Sai Jayesh Datta, Samyak Chugh, Sara Hunt, Sara Dhillon, Sargun Sidorov, Sasha Pan, Satadru Mahajan, Saurabh Verma, Saurabh Yamamoto, Seiji Ramaswamy, Sharadh Lindsay, Shaun Lindsay, Shaun Feng, Sheng Lin, Shenghao Zha, Shengxin Cindy Patil, Shishir Shankar, Shiva Zhang, Shuqiang Zhang, Shuqiang Wang, Sinong Agarwal, Sneha Sajuyigbe, Soji Chintala, Soumith Max, Stephanie Chen, Stephen Kehoe, Steve Satterfield, Steve Govindaprasad, Sudarshan Gupta, Sumit Deng, Summer Cho, Sungmin Virk, Sunny Subramanian, Suraj Choudhury, Sy Goldman, Sydney Remez, Tal Glaser, Tamar Best, Tamara Koehler, Thilo Robinson, Thomas Li, Tianhe Zhang, Tianjun Matthews, Tim Chou, Timothy Shaked, Tzook Vontimitta, Varun Ajayi, Victoria Montanez, Victoria Mohan, Vijai Kumar, Vinay Satish Mangla, Vishal Ionescu, Vlad Poenaru, Vlad Mihailescu, Vlad Tiberiu Ivanov, Vladimir Li, Wei Wang, Wenchen Jiang, Wenwen Bouaziz, Wes Constable, Will Tang, Xiaocheng Wu, Xiaojian Wang, Xiaolan Wu, Xilun Gao, Xinbo Kleinman, Yaniv Chen, Yanjun Hu, Ye Jia, Ye Qi, Ye Li, Yenda Zhang, Yilin Zhang, Ying Adi, Yossi Nam, Youngjin Yu Wang Zhao, Yu Hao, Yuchen Qian, Yundi Li, Yunlu He, Yuzi Rait, Zach DeVito, Zachary Rosnbrick, Zef Wen, Zhaoduo Yang, Zhenyu Zhao, Zhiwei Ma, Zhiyu Artificial Intelligence Computation and Language Computer Vision and Pattern Recognition Modern artificial intelligence (AI) systems are powered by foundation models. This paper presents a new set of foundation models, called Llama 3. It is a herd of language models that natively support multilinguality, coding, reasoning, and tool usage. Our largest model is a dense Transformer with 405B parameters and a context window of up to 128K tokens. This paper presents an extensive empirical evaluation of Llama 3. We find that Llama 3 delivers comparable quality to leading language models such as GPT-4 on a plethora of tasks. We publicly release Llama 3, including pre-trained and post-trained versions of the 405B parameter language model and our Llama Guard 3 model for input and output safety. The paper also presents the results of experiments in which we integrate image, video, and speech capabilities into Llama 3 via a compositional approach. We observe this approach performs competitively with the state-of-the-art on image, video, and speech recognition tasks. The resulting models are not yet being broadly released as they are still under development.
title	The Llama 3 Herd of Models
topic	Artificial Intelligence Computation and Language Computer Vision and Pattern Recognition
url	https://arxiv.org/abs/2407.21783

Similar Items