Saved in:
Bibliographic Details
Main Authors: NVIDIA, :, Chandiramani, Aakshita, Blakeman, Aaron, Olaoye, Abdullahi, Gupta, Abhibha, Somasamudramath, Abhilash, Khattar, Abhinav, Adesoba, Adeola, Renduchintala, Adi, Asif, Adil, Agrawal, Aditya, Vavre, Aditya, Kiswani, Ahmad, Padmakumar, Aishwarya, Hotchandani, Ajay, Shukla, Akanksha, Bercovich, Akhiad, Ficek, Aleksander, Shaposhnikov, Aleksandr, Gronskiy, Alex, Kondratenko, Alex, Neefus, Alex, Steiner, Alex, Yang, Alex, Bukharin, Alexander, Young, Alexander, Hatamizadeh, Ali, Taghibakhshi, Ali, Galiautdinova, Alina, Liu, Alisa, Kumar, Alok, Mahabaleshwarkar, Ameya Sunil, Klein, Amir, Zuker, Amit, Geifman, Amnon, Bhiwandiwalla, Anahita, Subramaniam, Ananth, Tao, Andrew, Shrivastava, Anjaney, Agrusa, Anjulie, Srivastava, Ankur, Verma, Ankur, Guan, Ann, Shors, Anna, Chockalingam, Annamalai, Mandarwal, Anubhav, Ramani, Aparnaa, Mehta, Arham, Jain, Arti, Venkatesan, Arun, Anoosheh, Asha, Aithal, Ashwath, Poojary, Ashwin, Ahamed, Asif, Mishra, Asit, Demiroz, Asli Sabanci, Thekkumpate, Asma Kuriparambil, Sohrabizadeh, Atefeh, Kaur, Avinash, Dattagupta, Ayush, Anandan, Barath Subramaniam, Sadeghi, Bardiya, Simkin, Barnaby, Lanir, Ben, Schifferer, Benedikt, Chislett, Benjamin, Nushi, Besmira, Kartal, Bilal, Thiede, Bill, Rouhani, Bita Darvish, Chen, Bobby, Ginsburg, Boris, Norick, Brandon, Kisacanin, Branislav, Yu, Brian, Catanzaro, Bryan, Mani, Buvaneswari, del Mundo, Carlo, Lee, Chankyu, Kim, Chanran, Hwang, Chantal, Ni, Chao, Wang, Charles, Truong, Charlie, Hsieh, Cheng-Ping, Yu, Chenhan, Luo, Chenjie, Wang, Cherie, Mungekar, Chetan, Patel, Chintan, Alexiuk, Chris, Holguin, Chris, Wing, Chris, Munley, Christian, Parisien, Christopher, Desai, Chuck, Sheng, Chunyang, Neale, Collin, Meurillon, Cyril, Kumar, Dakshi, Gil, Dan, Su, Dan, Corneil, Dane, Afrimi, Daniel, Triana, Daniel Burkhardt Eliuth, Egert, Daniel, Fatade, Daniel, Lo, Daniel, Rohrer, Daniel, Serebrenik, Daniel, Sorokin, Daniil, Gitman, Daria, Levy, Daria, Stosic, Darko, Edelsohn, David, Messina, David, Mosallanezhad, David, Tamok, David, Donia, Deena, Narayanan, Deepak, O'Kelly, Devin, Peri, Dheeraj, Nathawani, Dhruv, Wu, Di, Rekesh, Dima, Yared, Dina, Kakwani, Divyanshu, Tuttle, Dmitry Konyagin Brandon, Ahn, Dong, Jiang, Dongfu, Poorkay, Dorrin, O'Flaherty, Douglas, Riach, Duncan, Stosic, Dusan, Van Stee, Dustin, Minasyan, Edgar, Lin, Edward, Long, Eileen Peters, Segal, Elad, Lantz, Elena, Lewis, Elena, Evans, Ellie, Ning, Elliott, Chung, Eric, Harper, Eric, Pham-Hung, Eric, Tramel, Eric W., Galinkin, Erick, Pounds, Erik, Etrog, Esti, Briones, Evan, Wu, Evan, Bakhturina, Evelina, Tsykunov, Evgeny, Dobrowolska, Ewa, Movahed, Farshad Saberi, Memarian, Farzan, Wang, Fay, Jia, Fei, Soares, Felipe, Frujeri, Felipe Vieira, Chen, Feng, Lin, Fengguang, Galko, Ferenc, Zhang, Fortuna, Siino, Frankie, Hou, Frida, Bhatt, Gantavya, Prasad, Gargi, Venkataramani, Geethapriya, Gupta, Geetika, Armstrong, George, Shen, Gerald, Borghesi, Giulio, Neskovic, Gordana, Batmaz, Gorkem, Lam, Grace, Wu, Grace, Pauloski, Greg, Davis, Greyson, Nalbandyan, Grigor, Zhang, Guoming, Farber, Guy, Huang, Guyue, Qian, Haifeng, Kumar, Haran Kumar Shiv, Kim, Harry, Sharma, Harsh, Iso, Hayate, Ross, Hayley, Hum, Herbert, Sahota, Herman, Wang, Hexin, Soni, Himanshu, Upadhyay, Hiren, Nguyen, Huy, Cunningham, Iain, Galil, Ido, Shahaf, Ido, Padovani, Igino, Gitman, Igor, Shovkun, Igor, Dhillon, Ikroop, Loshchilov, Ilya, Kelly, Ingrid, Schen, Itamar, Levy, Itay, Moshkov, Ivan, Golan, Izik, Putterman, Izzy, Tu, Jain, Baczek, Jan, Kautz, Jan, Scowcroft, Jane Polak, Rosenberg, Janica, Casper, Jared, Pflum, Jarrod, Grant, Jason, Sewall, Jason, Mitra, Jatin, Glick, Jeffrey, Chen, Jenny, Oliver, Jesse, Xu, Jiacheng, Zhu, Jiafan, Song, Jialin, Zhang, Jian, Zeng, Jiaqi, Lou, Jie, Milton, Jill, Chow, Jim, Zhang, Jimmy, Choi, Jinhang, Huang, Jining, Huang, Jocelyn, Caruso, Joel, Conway, Joey, Guman, Joey, Jatko, Johan, Kamalu, John, Greco, Johnny, Cohen, Jonathan, Raiman, Jonathan, Jennings, Joseph, Daw, Joyjit, Yu, Juan, Tapia, Julio, Yi, Junkeun, Parmar, Jupinder, Achar, Jyothi, Briski, Kari, Mattoo, Kartik, Cheung, Katherine, Luna, Katherine, Wyss, Keith, Shih, Kevin, Kong, Kezhi, Nguyen, Khanh, Bhardwaj, Khushi, Buryak, Kirill, Sivamani, Kirthi Shankar, Krommydas, Konstantinos, Murphy, Kris, Puvvada, Krishna C., Pawelec, Krzysztof, Anik, Kumar, Tewari, Laikh, Sleiman, Laya, Du, Leo, Derczynski, Leon, Ding, Li, Ilan, Lilach, Wu, Lingjie, Wei, Lizzie, Vega, Luis, Su, Lun, Van Segbroeck, Maarten, de Melo, Maer Rodrigues, Zhang, Magaret, Fathi, Mahan, Sreedhar, Makesh Narsimhan, Sreedhar, Makesh, Chandran, Makesh Tarun, Gomez, Manuel Reyes, Ashkenazi, Maor, Cuevas, Marc, Romeijn, Marc, Zhang, Margaret, Cai, Mark, Gabel, Mark, Kliegl, Markus, Patelka, Martyna, Moosaei, Maryam, Varacalli, Matthew, Novikov, Matvei, Ferrato, Mauricio, Samadi, Mehrzad, Corpuz, Melissa, Xin, Meng, Wang, Mengdi, Wang, Mengru, Price, Meredith, Schaffer, Micah, Andersch, Michael, Boone, Michael, Evans, Michael, Wang, Michael Z, Martinez, Miguel, Khona, Mikail, Chrzanowski, Mike, Hollinger, Mike, Ma, Mingyuan, Lee, Minseok, Dabbah, Mohammad, Shoeybi, Mohammad, Patwary, Mostofa, Mulepati, Nabin, Khalil, Nader, Nabwani, Najeeb, Agarwal, Nancy, Balasubramaniam, Nanthini, Hennouni, Narimane, Kodukula, Narsi, Hereth, Natalie, Pinckney, Nathaniel, Assaf, Nave, Habibi, Negar, Qin, Nestor, Zmora, Neta, Haber, Netanel, Reamaroon, Nick, Quak, Nickson, Bhatia, Nidhi, Jukar, Nikhil, Pope, Nikki, Ludwig, Nikolai, Tajbakhsh, Nima, Ailon, Nir, Juluru, Nirmal, De, Nirmalya, Pitt, Nowel, Rybakov, Oleg, Hrinchuk, Oleksii, Kuchaiev, Oleksii, Delalleau, Olivier, Olabiyi, Oluwatobi, Argov, Omer Ullman, Almog, Omri, Puny, Omri, Tropp, Oren, Padovani, Otavio, Xie, Ouye, Chadha, Parth, Shamis, Pasha, Gibbons, Paul, Molchanov, Pavlo, Belcak, Peter, Jin, Peter, Xu, Pinky, Januszewski, Piotr, Jannaty, Pooya, Shevate, Prachi, Thalasta, Pradeep, Thombre, Pranav Prashant, Varshney, Prasoon, Gambhir, Prerana, Gundecha, Pritam, Tredak, Przemek, Miao, Qing, Wan, Qiyu, Minh, Quan Tran, Mahabadi, Rabeeh Karimi, Oberman, Rachel, Garg, Rachit, Kandu, Rahul, Zhong, Raina, El-Yaniv, Ran, Zilberstein, Ran, Shafipour, Rasoul, Yao, Renee, Pi, Renjie, Mazzarese, Richard, Wang, Richard, Izzo, Rick, Singla, Ridhima, Shahbazyan, Rima, Garg, Rishabh, Borkar, Ritika, Gala, Ritu, Islam, Riyad, Clark, Robert, Hesse, Robert, Waleffe, Roger, Kalidindi, Rohit Varma, Watve, Rohit, Koren, Roi, Fan, Ron, Kharwar, Ruchika, Cai, Ruisi, Zhang, Ruoxi, Hewett, Russell J., Prenger, Ryan, Timbrook, Ryan, Egashira, Ryota, Mahdavi, Sadegh, Joshi, Sagar Singh Ashutosh, Modi, Sahil, Kriman, Samuel, Pombra, Sandeep, Kariyappa, Sanjay, Satheesh, Sanjeev, Pombo, Santiago, Kaji, Saori, Pasumarthi, Satish, Mishra, Saurav, Muralidharan, Saurav, Hara, Scott, Narenthiran, Sean, Rogawski, Sebastian, Na, Seonjin, Bak, Seonmyeong, Sameni, Sepehr, Poulos, Seth, Mor, Shahar, Acharya, Shantanu, Lord, Shaona Ghosh Adam, Sreenivas, Sharath Turuvekere, Kotek, Shaun, Gharghabi, Shaya, Thomas, Shelby, Lin, Sheng-Chieh, Likhite, Shibani, Fan, Shiqing, Chen, Shiyang, Gopal, Shreya, Prabhumoye, Shrimai, Pachori, Shubham, Toshniwal, Shubham, Zhang, Shuo, Ding, Shuoyang, Renjith, Shyam, Prayaga, Shyamala, Jain, Siddhartha, Sun, Simeng, Rella, Sirisha, Das, Sirshak, Ithape, Smita, S, Sneha Harishchandra, Majumdar, Somshubra, Singhal, Soumye, Singudasu, Sri Harsha, Niverty, Sriharsha, Sergienko, Stas, Gloginic, Stefana, Alborghetti, Stefania, Ge, Stephen, McCullough, Stephen, Devare, Sugam Dipak, Velury, Suguna Varshini, Rao, Sukrit, Barua, Sumeet Kumar, Gai, Sunny, Panguluri, Suseella, Koundinyan, Sushil, Patnam, Swathi, Priyadarshi, Sweta, Bhendigeri, Swetha, Akter, Syeda Nahida, Arunagiri, Sylendran, Yuan, Tailling, Abramovich, Talor, Bui, Tan, Yu, Tan, Kong, Terry, Do, Thanh, Gburek, Thomas, Marques, Thorgane, Moore, Tiffany, Blankevoort, Tijmen, Moon, Tim, Ma, Timothy, Mitra, Tiyasa, Grzegorzek, Tomasz, Asida, Tomer, Natan, Tomer Bar, Keren, Tomer, Ronen, Tomer, Rebedea, Traian, Starkey, Trenton, Konuk, Tugrul, Vashishth, Twinkle, Condensa, Tyler, Karpas, Udi, De, Ushnish, Noorozi, Vahid, Noroozi, Vahid, Shah, Vanshil Atul, Vaidyanathan, Veena, Srinivasan, Venkat, Elango, Venmugil, Cui, Victor, Korthikanti, Vijay, Mehta, Vikas, Adams, Virginia, Wu, Virginia, Kurin, Vitaly, Lavrukhin, Vitaly, Anisimov, Vladimir, Seo, Wan, Jiang, Wanli, Ahmad, Wasi Uddin, Du, Wei, Ping, Wei, Chen, Wei-Ming, Quan, Wendy, Dai, Wenliang, Gao, Wenwen, Jennings, Will, Zhang, William, Ren, Xiaowei, Xin, Xiaowen, Li, Xin, Yu, Yang, Chen, Yangyi, Galron, Yaniv, Karnati, Yashaswi, Choi, Yejin, Meyer, Yev, Wu, Yi-Fu, Zhang, Yian, Lin, Ying, Geifman, Yonatan, Fu, Yonggan, Suhara, Yoshi, Kwon, Youngeun, Zhang, Yuan, Huang, Yuki, Moshe, Zach, Wang, Zhilin, Cheng, Zhiyu, Zhu, Zhongbo, Yang, Zhuolin, Liu, Zihan, Chen, Zijia, Yan, Zijie, Ahmed, Zuhair
Format: Preprint
Published: 2026
Subjects:
Online Access:https://arxiv.org/abs/2604.12374
Tags: Add Tag
No Tags, Be the first to tag this record!
Table of Contents:
  • We describe the pre-training, post-training, and quantization of Nemotron 3 Super, a 120 billion (active 12 billion) parameter hybrid Mamba-Attention Mixture-of-Experts model. Nemotron 3 Super is the first model in the Nemotron 3 family to 1) be pre-trained in NVFP4, 2) leverage LatentMoE, a new Mixture-of-Experts architecture that optimizes for both accuracy per FLOP and accuracy per parameter, and 3) include MTP layers for inference acceleration through native speculative decoding. We pre-trained Nemotron 3 Super on 25 trillion tokens followed by post-training using supervised fine tuning (SFT) and reinforcement learning (RL). The final model supports up to 1M context length and achieves comparable accuracy on common benchmarks, while also achieving up to 2.2x and 7.5x higher inference throughput compared to GPT-OSS-120B and Qwen3.5-122B, respectively. Nemotron 3 Super datasets, along with the base, post-trained, and quantized checkpoints, are open-sourced on HuggingFace.