Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | https://arxiv.org/abs/2507.13575 |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866912556455559168 |
|---|---|
| author | Li, Ethan Larsen, Anders Boesen Lindbo Zhang, Chen Zhou, Xiyou Qin, Jun Yap, Dian Ang Raghavan, Narendran Chang, Xuankai Bowler, Margit Yildiz, Eray Peebles, John Coleman, Hannah Gillis Ronchi, Matteo Gray, Peter You, Keen Spalvieri-Kruse, Anthony Pang, Ruoming Li, Reed Yang, Yuli Soroush, Emad Lu, Zhiyun Xiao, Crystal Situ, Rong Huffaker, Jordan Griffiths, David Ahmed, Zaid Zhang, Peng Parilla, Daniel Liberman, Asaf Mallalieu, Jennifer Mazaheri, Parsa Chen, Qibin Bilkhu, Manjot Zhang, Aonan Wang, Eric Nelson, Dave FitzMaurice, Michael Voice, Thomas Liu, Jeremy Shaffer, Josh Zhao, Shiwen Yadla, Prasanth Rasteh, Farzin Guo, Pengsheng Farooq, Arsalan Snow, Jeremy Murphy, Stephen Lei, Tao Cho, Minsik Horrell, George Dodge, Sam Hislop, Lindsay Singh, Sumeet Dombrowski, Alex Raghavan, Aiswarya Sirovica, Sasha Saebi, Mandana Lao, Faye Lam, Max Lu, TJ Xu, Zhaoyang Singh, Karanjeet Kirchner, Marc Mizrahi, David Arora, Rajat Zhang, Haotian Mason, Henry Zhou, Lawrence Hua, Yi Jain, Ankur Bai, Felix Astrauskas, Joseph Weers, Floris Gardner, Josh Chiang, Mira Zhang, Yi Agrawal, Pulkit Sun, Tony Keunebroek, Quentin Hopkins, Matthew Wu, Bugu Jia, Tao Chen, Chen Zhou, Xingyu Wang, Nanzhu Liu, Peng Hou, Ruixuan Rauch, Rene Gao, Yuan Dehghan, Afshin Janke, Jonathan Wang, Zirui Chen, Cha Ren, Xiaoyi Nan, Feng Elman, Josh Yin, Dong Goren, Yusuf Lai, Jeff Fei, Yiran Evans, Syd Yu, Muyang Yin, Guoli Qin, Yi Feldman, Erin Garg, Isha Rajamani, Aparna Vega, Karla Cheng, Walker Collins, TJ Han, Hans Menacho, Raul Rea Yeung, Simon Lee, Sophy Mutyala, Phani Cheng, Ying-Chang Gan, Zhe Chu, Sprite Lazarow, Justin Pappalardo, Alessandro Scozzafava, Federico Lu, Jing Daxberger, Erik Duchesne, Laurent Liu, Jen Güera, David Ligas, Stefano Kery, Mary Beth Ramerth, Brent Sannino, Ciro Eichner, Marcin Huang, Haoshuo Qian, Rui Schwarzer-Becker, Moritz Riazati, David Gao, Mingfei Wang, Bailin Cackler, Jack Lu, Yang Niu, Ransen Dennison, John Klein, Guillaume Bigham, Jeffrey Gopinath, Deepak Shiee, Navid Botten, Darren Tartavel, Guillaume Garcia, Alex Guillen Xu, Sam Haladjian, Victoria MönchJuan Dou, Zi-Yi Paulik, Matthias Mendez, Adolfo Lopez Li, Zhen Chen, Hong-You Jia, Chao Doshi, Dhaval Zhang, Zhengdong Manjani, Raunak Franklin, Aaron Ren, Zhile Chen, David Peshko, Artsiom Raghuram, Nandhitha Hao, Hans Shan, Jiulong Nerella, Kavya Tantawi, Ramsey Kumar, Vivek Wang, Saiwen Wershing, Brycen Dhingra, Bhuwan Shah, Dhruti Adaranijo, Ob Zheng, Xin Madsen, Tait Kotek, Hadas Liu, Chang Xia, Yin Li, Hanli Jayaram, Suma Sun, Yanchao Fakhry, Ahmed Saveris, Vasileios Withers, Dustin Li, Yanghao Aygar, Alp Teran, Andres Romero Mier Y Huang, Kaiwei Lee, Mark Li, Xiujun Li, Yuhong Johnson, Tyler Tang, Jay Cheng, Joseph Yitan Peng, Futang Walkingshaw, Andrew Guibert, Lucas Sharma, Abhishek Shen, Cheng Maj, Piotr Tanaka, Yasutaka Jhang, You-Cyuan Ma, Vivian Vehvilainen, Tommi Zou, Kelvin Nichols, Jeff Lei, Matthew Qiu, David Qian, Yihao Santhanam, Gokul Wu, Wentao Han, Yena Moritz, Dominik Fu, Haijing Xu, Mingze Rathod, Vivek Liu, Jian D'hauwe, Louis Ba, Qin Sun, Haitian Yan, Haoran Dufter, Philipp Nguyen, Anh Feng, Yihao Wang, Emma He, Keyu Nair, Rahul Shah, Sanskruti Lu, Jiarui Sonnenberg, Patrick Warner, Jeremy Li, Yuanzhi Pan, Bowen Zhong, Ziyi Zhou, Joe Davarnia, Sam Saarikivi, Olli Belousova, Irina Burger, Rachel Wu, Shang-Chen Feng, Di Straathof, Bas Chou, James Zhang, Yuanyang Zuliani, Marco Jimenez, Eduardo Sundararajan, Abhishek Du, Xianzhi Lan, Chang Shahdadpuri, Nilesh Grasch, Peter Sima, Sergiu Newnham, Josh Paidi, Varsha Wang, Jianyu Haag, Kaelen Braunstein, Alex Molinari, Daniele Wei, Richard Yang, Brenda Lusskin, Nicholas Arreaza-Taylor, Joanna Cao, Meng Seidl, Nicholas Wang, Simon Hu, Jiaming Ma, Yiping Li, Mengyu Liu, Kieran Su, Hang Ravi, Sachin Wang, Chong Wang, Xin Smith, Kevin You, Haoxuan Karimzadeh, Binazir Li, Rui Lei, Jinhao Fang, Wei Doane, Alec Wiseman, Sam Fernandez, Ismael Li, Jane Hansen, Andrew Movellan, Javier Neubauer, Christopher Zhou, Hanzhi Chaney, Chris Kamaldin, Nazir Wolf, Valentin Bermúdez-Medina, Fernando Pelemans, Joris Fu, Peter Xing, Howard Kong, Xiang Shan, Wayne Jacoby-Cooper, Gabriel Shen, Dongcai Gunter, Tom Seguin, Guillaume Shi, Fangping Li, Shiyu Xu, Yang Kamal, Areeba Masi, Dan Guha, Saptarshi Zhu, Qi Thibodeau, Jenna Zhang, Changyuan Callahan, Rebecca Maalouf, Charles Tsao, Wilson Li, Boyue Cao, Qingqing Sabo, Naomy Leong, Cheng Wang, Yi Anupama, Anupama Mann Reed, Colorado Jung, Kenneth Chen, Zhifeng Moorthy, Mohana Prasad Sathya He, Yifei Hornberger, Erik Krishna, Devi Tong, Senyu Michael Lee Haldimann, David Zhao, Yang Zhang, Bowen Gao, Chang Bartels, Chris Rao, Sushma Tran, Nathalie Lehnerer, Simon Giang, Co Dong, Patrick Pan, Junting Wang, Biyao Li, Dongxu Farajtabar, Mehrdad Hwang, Dongseong Duanmu, Grace Verma, Eshan Reddy, Sujeeth Shan, Qi Gao, Hongbin Du, Nan Sridhar, Pragnya Huang, Forrest Wang, Yingbo Bhendawade, Nikhil Zhu, Diane Aitharaju, Sai Hohman, Fred Gardiner, Lauren Chiu, Chung-Cheng Yang, Yinfei Kokmen, Alper Chu, Frank Ye, Ke Elgin, Kaan Levy, Oron Park, John Zhang, Donald Schoop, Eldon Wenzel, Nina Booker, Michael Kim, Hyunjik Erdenebileg, Chinguun Dun, Nan Yang, Eric Liang Chhatrapati, Priyal Mahtani, Vishaal Gang, Haiming Chia, Kohen Seshadri, Deepa Yu, Donghan Meng, Yan Peterson, Kelsey Yang, Zhen Wang, Yongqiang Peng, Carina Kang, Doug Agarwal, Anuva Antony, Albert Tebar, Juan Lao Jose, Albin Madappally Poston, Regan De Wang, Andy Casamayor, Gerard Amirloo, Elmira Yao, Violet Kryscinski, Wojciech Duan, Kun L, Lezhi |
| author_facet | Li, Ethan Larsen, Anders Boesen Lindbo Zhang, Chen Zhou, Xiyou Qin, Jun Yap, Dian Ang Raghavan, Narendran Chang, Xuankai Bowler, Margit Yildiz, Eray Peebles, John Coleman, Hannah Gillis Ronchi, Matteo Gray, Peter You, Keen Spalvieri-Kruse, Anthony Pang, Ruoming Li, Reed Yang, Yuli Soroush, Emad Lu, Zhiyun Xiao, Crystal Situ, Rong Huffaker, Jordan Griffiths, David Ahmed, Zaid Zhang, Peng Parilla, Daniel Liberman, Asaf Mallalieu, Jennifer Mazaheri, Parsa Chen, Qibin Bilkhu, Manjot Zhang, Aonan Wang, Eric Nelson, Dave FitzMaurice, Michael Voice, Thomas Liu, Jeremy Shaffer, Josh Zhao, Shiwen Yadla, Prasanth Rasteh, Farzin Guo, Pengsheng Farooq, Arsalan Snow, Jeremy Murphy, Stephen Lei, Tao Cho, Minsik Horrell, George Dodge, Sam Hislop, Lindsay Singh, Sumeet Dombrowski, Alex Raghavan, Aiswarya Sirovica, Sasha Saebi, Mandana Lao, Faye Lam, Max Lu, TJ Xu, Zhaoyang Singh, Karanjeet Kirchner, Marc Mizrahi, David Arora, Rajat Zhang, Haotian Mason, Henry Zhou, Lawrence Hua, Yi Jain, Ankur Bai, Felix Astrauskas, Joseph Weers, Floris Gardner, Josh Chiang, Mira Zhang, Yi Agrawal, Pulkit Sun, Tony Keunebroek, Quentin Hopkins, Matthew Wu, Bugu Jia, Tao Chen, Chen Zhou, Xingyu Wang, Nanzhu Liu, Peng Hou, Ruixuan Rauch, Rene Gao, Yuan Dehghan, Afshin Janke, Jonathan Wang, Zirui Chen, Cha Ren, Xiaoyi Nan, Feng Elman, Josh Yin, Dong Goren, Yusuf Lai, Jeff Fei, Yiran Evans, Syd Yu, Muyang Yin, Guoli Qin, Yi Feldman, Erin Garg, Isha Rajamani, Aparna Vega, Karla Cheng, Walker Collins, TJ Han, Hans Menacho, Raul Rea Yeung, Simon Lee, Sophy Mutyala, Phani Cheng, Ying-Chang Gan, Zhe Chu, Sprite Lazarow, Justin Pappalardo, Alessandro Scozzafava, Federico Lu, Jing Daxberger, Erik Duchesne, Laurent Liu, Jen Güera, David Ligas, Stefano Kery, Mary Beth Ramerth, Brent Sannino, Ciro Eichner, Marcin Huang, Haoshuo Qian, Rui Schwarzer-Becker, Moritz Riazati, David Gao, Mingfei Wang, Bailin Cackler, Jack Lu, Yang Niu, Ransen Dennison, John Klein, Guillaume Bigham, Jeffrey Gopinath, Deepak Shiee, Navid Botten, Darren Tartavel, Guillaume Garcia, Alex Guillen Xu, Sam Haladjian, Victoria MönchJuan Dou, Zi-Yi Paulik, Matthias Mendez, Adolfo Lopez Li, Zhen Chen, Hong-You Jia, Chao Doshi, Dhaval Zhang, Zhengdong Manjani, Raunak Franklin, Aaron Ren, Zhile Chen, David Peshko, Artsiom Raghuram, Nandhitha Hao, Hans Shan, Jiulong Nerella, Kavya Tantawi, Ramsey Kumar, Vivek Wang, Saiwen Wershing, Brycen Dhingra, Bhuwan Shah, Dhruti Adaranijo, Ob Zheng, Xin Madsen, Tait Kotek, Hadas Liu, Chang Xia, Yin Li, Hanli Jayaram, Suma Sun, Yanchao Fakhry, Ahmed Saveris, Vasileios Withers, Dustin Li, Yanghao Aygar, Alp Teran, Andres Romero Mier Y Huang, Kaiwei Lee, Mark Li, Xiujun Li, Yuhong Johnson, Tyler Tang, Jay Cheng, Joseph Yitan Peng, Futang Walkingshaw, Andrew Guibert, Lucas Sharma, Abhishek Shen, Cheng Maj, Piotr Tanaka, Yasutaka Jhang, You-Cyuan Ma, Vivian Vehvilainen, Tommi Zou, Kelvin Nichols, Jeff Lei, Matthew Qiu, David Qian, Yihao Santhanam, Gokul Wu, Wentao Han, Yena Moritz, Dominik Fu, Haijing Xu, Mingze Rathod, Vivek Liu, Jian D'hauwe, Louis Ba, Qin Sun, Haitian Yan, Haoran Dufter, Philipp Nguyen, Anh Feng, Yihao Wang, Emma He, Keyu Nair, Rahul Shah, Sanskruti Lu, Jiarui Sonnenberg, Patrick Warner, Jeremy Li, Yuanzhi Pan, Bowen Zhong, Ziyi Zhou, Joe Davarnia, Sam Saarikivi, Olli Belousova, Irina Burger, Rachel Wu, Shang-Chen Feng, Di Straathof, Bas Chou, James Zhang, Yuanyang Zuliani, Marco Jimenez, Eduardo Sundararajan, Abhishek Du, Xianzhi Lan, Chang Shahdadpuri, Nilesh Grasch, Peter Sima, Sergiu Newnham, Josh Paidi, Varsha Wang, Jianyu Haag, Kaelen Braunstein, Alex Molinari, Daniele Wei, Richard Yang, Brenda Lusskin, Nicholas Arreaza-Taylor, Joanna Cao, Meng Seidl, Nicholas Wang, Simon Hu, Jiaming Ma, Yiping Li, Mengyu Liu, Kieran Su, Hang Ravi, Sachin Wang, Chong Wang, Xin Smith, Kevin You, Haoxuan Karimzadeh, Binazir Li, Rui Lei, Jinhao Fang, Wei Doane, Alec Wiseman, Sam Fernandez, Ismael Li, Jane Hansen, Andrew Movellan, Javier Neubauer, Christopher Zhou, Hanzhi Chaney, Chris Kamaldin, Nazir Wolf, Valentin Bermúdez-Medina, Fernando Pelemans, Joris Fu, Peter Xing, Howard Kong, Xiang Shan, Wayne Jacoby-Cooper, Gabriel Shen, Dongcai Gunter, Tom Seguin, Guillaume Shi, Fangping Li, Shiyu Xu, Yang Kamal, Areeba Masi, Dan Guha, Saptarshi Zhu, Qi Thibodeau, Jenna Zhang, Changyuan Callahan, Rebecca Maalouf, Charles Tsao, Wilson Li, Boyue Cao, Qingqing Sabo, Naomy Leong, Cheng Wang, Yi Anupama, Anupama Mann Reed, Colorado Jung, Kenneth Chen, Zhifeng Moorthy, Mohana Prasad Sathya He, Yifei Hornberger, Erik Krishna, Devi Tong, Senyu Michael Lee Haldimann, David Zhao, Yang Zhang, Bowen Gao, Chang Bartels, Chris Rao, Sushma Tran, Nathalie Lehnerer, Simon Giang, Co Dong, Patrick Pan, Junting Wang, Biyao Li, Dongxu Farajtabar, Mehrdad Hwang, Dongseong Duanmu, Grace Verma, Eshan Reddy, Sujeeth Shan, Qi Gao, Hongbin Du, Nan Sridhar, Pragnya Huang, Forrest Wang, Yingbo Bhendawade, Nikhil Zhu, Diane Aitharaju, Sai Hohman, Fred Gardiner, Lauren Chiu, Chung-Cheng Yang, Yinfei Kokmen, Alper Chu, Frank Ye, Ke Elgin, Kaan Levy, Oron Park, John Zhang, Donald Schoop, Eldon Wenzel, Nina Booker, Michael Kim, Hyunjik Erdenebileg, Chinguun Dun, Nan Yang, Eric Liang Chhatrapati, Priyal Mahtani, Vishaal Gang, Haiming Chia, Kohen Seshadri, Deepa Yu, Donghan Meng, Yan Peterson, Kelsey Yang, Zhen Wang, Yongqiang Peng, Carina Kang, Doug Agarwal, Anuva Antony, Albert Tebar, Juan Lao Jose, Albin Madappally Poston, Regan De Wang, Andy Casamayor, Gerard Amirloo, Elmira Yao, Violet Kryscinski, Wojciech Duan, Kun L, Lezhi |
| contents | We introduce two multilingual, multimodal foundation language models that power Apple Intelligence features across Apple devices and services: i a 3B-parameter on-device model optimized for Apple silicon through architectural innovations such as KV-cache sharing and 2-bit quantization-aware training; and ii a scalable server model built on a novel Parallel-Track Mixture-of-Experts PT-MoE transformer that combines track parallelism, mixture-of-experts sparse computation, and interleaved global-local attention to deliver high quality with competitive cost on Apple's Private Cloud Compute platform. Both models are trained on large-scale multilingual and multimodal datasets sourced via responsible web crawling, licensed corpora, and high-quality synthetic data, then further refined with supervised fine-tuning and reinforcement learning on a new asynchronous platform. The resulting models support several additional languages while understanding images and executing tool calls. In public benchmarks and human evaluations, both the server model and the on-device model match or surpass comparably sized open baselines.
A new Swift-centric Foundation Models framework exposes guided generation, constrained tool calling, and LoRA adapter fine-tuning, allowing developers to integrate these capabilities with a few lines of code. The latest advancements in Apple Intelligence models are grounded in our Responsible AI approach with safeguards like content filtering and locale-specific evaluation, as well as our commitment to protecting our users' privacy with innovations like Private Cloud Compute. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_13575 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Apple Intelligence Foundation Language Models: Tech Report 2025 Li, Ethan Larsen, Anders Boesen Lindbo Zhang, Chen Zhou, Xiyou Qin, Jun Yap, Dian Ang Raghavan, Narendran Chang, Xuankai Bowler, Margit Yildiz, Eray Peebles, John Coleman, Hannah Gillis Ronchi, Matteo Gray, Peter You, Keen Spalvieri-Kruse, Anthony Pang, Ruoming Li, Reed Yang, Yuli Soroush, Emad Lu, Zhiyun Xiao, Crystal Situ, Rong Huffaker, Jordan Griffiths, David Ahmed, Zaid Zhang, Peng Parilla, Daniel Liberman, Asaf Mallalieu, Jennifer Mazaheri, Parsa Chen, Qibin Bilkhu, Manjot Zhang, Aonan Wang, Eric Nelson, Dave FitzMaurice, Michael Voice, Thomas Liu, Jeremy Shaffer, Josh Zhao, Shiwen Yadla, Prasanth Rasteh, Farzin Guo, Pengsheng Farooq, Arsalan Snow, Jeremy Murphy, Stephen Lei, Tao Cho, Minsik Horrell, George Dodge, Sam Hislop, Lindsay Singh, Sumeet Dombrowski, Alex Raghavan, Aiswarya Sirovica, Sasha Saebi, Mandana Lao, Faye Lam, Max Lu, TJ Xu, Zhaoyang Singh, Karanjeet Kirchner, Marc Mizrahi, David Arora, Rajat Zhang, Haotian Mason, Henry Zhou, Lawrence Hua, Yi Jain, Ankur Bai, Felix Astrauskas, Joseph Weers, Floris Gardner, Josh Chiang, Mira Zhang, Yi Agrawal, Pulkit Sun, Tony Keunebroek, Quentin Hopkins, Matthew Wu, Bugu Jia, Tao Chen, Chen Zhou, Xingyu Wang, Nanzhu Liu, Peng Hou, Ruixuan Rauch, Rene Gao, Yuan Dehghan, Afshin Janke, Jonathan Wang, Zirui Chen, Cha Ren, Xiaoyi Nan, Feng Elman, Josh Yin, Dong Goren, Yusuf Lai, Jeff Fei, Yiran Evans, Syd Yu, Muyang Yin, Guoli Qin, Yi Feldman, Erin Garg, Isha Rajamani, Aparna Vega, Karla Cheng, Walker Collins, TJ Han, Hans Menacho, Raul Rea Yeung, Simon Lee, Sophy Mutyala, Phani Cheng, Ying-Chang Gan, Zhe Chu, Sprite Lazarow, Justin Pappalardo, Alessandro Scozzafava, Federico Lu, Jing Daxberger, Erik Duchesne, Laurent Liu, Jen Güera, David Ligas, Stefano Kery, Mary Beth Ramerth, Brent Sannino, Ciro Eichner, Marcin Huang, Haoshuo Qian, Rui Schwarzer-Becker, Moritz Riazati, David Gao, Mingfei Wang, Bailin Cackler, Jack Lu, Yang Niu, Ransen Dennison, John Klein, Guillaume Bigham, Jeffrey Gopinath, Deepak Shiee, Navid Botten, Darren Tartavel, Guillaume Garcia, Alex Guillen Xu, Sam Haladjian, Victoria MönchJuan Dou, Zi-Yi Paulik, Matthias Mendez, Adolfo Lopez Li, Zhen Chen, Hong-You Jia, Chao Doshi, Dhaval Zhang, Zhengdong Manjani, Raunak Franklin, Aaron Ren, Zhile Chen, David Peshko, Artsiom Raghuram, Nandhitha Hao, Hans Shan, Jiulong Nerella, Kavya Tantawi, Ramsey Kumar, Vivek Wang, Saiwen Wershing, Brycen Dhingra, Bhuwan Shah, Dhruti Adaranijo, Ob Zheng, Xin Madsen, Tait Kotek, Hadas Liu, Chang Xia, Yin Li, Hanli Jayaram, Suma Sun, Yanchao Fakhry, Ahmed Saveris, Vasileios Withers, Dustin Li, Yanghao Aygar, Alp Teran, Andres Romero Mier Y Huang, Kaiwei Lee, Mark Li, Xiujun Li, Yuhong Johnson, Tyler Tang, Jay Cheng, Joseph Yitan Peng, Futang Walkingshaw, Andrew Guibert, Lucas Sharma, Abhishek Shen, Cheng Maj, Piotr Tanaka, Yasutaka Jhang, You-Cyuan Ma, Vivian Vehvilainen, Tommi Zou, Kelvin Nichols, Jeff Lei, Matthew Qiu, David Qian, Yihao Santhanam, Gokul Wu, Wentao Han, Yena Moritz, Dominik Fu, Haijing Xu, Mingze Rathod, Vivek Liu, Jian D'hauwe, Louis Ba, Qin Sun, Haitian Yan, Haoran Dufter, Philipp Nguyen, Anh Feng, Yihao Wang, Emma He, Keyu Nair, Rahul Shah, Sanskruti Lu, Jiarui Sonnenberg, Patrick Warner, Jeremy Li, Yuanzhi Pan, Bowen Zhong, Ziyi Zhou, Joe Davarnia, Sam Saarikivi, Olli Belousova, Irina Burger, Rachel Wu, Shang-Chen Feng, Di Straathof, Bas Chou, James Zhang, Yuanyang Zuliani, Marco Jimenez, Eduardo Sundararajan, Abhishek Du, Xianzhi Lan, Chang Shahdadpuri, Nilesh Grasch, Peter Sima, Sergiu Newnham, Josh Paidi, Varsha Wang, Jianyu Haag, Kaelen Braunstein, Alex Molinari, Daniele Wei, Richard Yang, Brenda Lusskin, Nicholas Arreaza-Taylor, Joanna Cao, Meng Seidl, Nicholas Wang, Simon Hu, Jiaming Ma, Yiping Li, Mengyu Liu, Kieran Su, Hang Ravi, Sachin Wang, Chong Wang, Xin Smith, Kevin You, Haoxuan Karimzadeh, Binazir Li, Rui Lei, Jinhao Fang, Wei Doane, Alec Wiseman, Sam Fernandez, Ismael Li, Jane Hansen, Andrew Movellan, Javier Neubauer, Christopher Zhou, Hanzhi Chaney, Chris Kamaldin, Nazir Wolf, Valentin Bermúdez-Medina, Fernando Pelemans, Joris Fu, Peter Xing, Howard Kong, Xiang Shan, Wayne Jacoby-Cooper, Gabriel Shen, Dongcai Gunter, Tom Seguin, Guillaume Shi, Fangping Li, Shiyu Xu, Yang Kamal, Areeba Masi, Dan Guha, Saptarshi Zhu, Qi Thibodeau, Jenna Zhang, Changyuan Callahan, Rebecca Maalouf, Charles Tsao, Wilson Li, Boyue Cao, Qingqing Sabo, Naomy Leong, Cheng Wang, Yi Anupama, Anupama Mann Reed, Colorado Jung, Kenneth Chen, Zhifeng Moorthy, Mohana Prasad Sathya He, Yifei Hornberger, Erik Krishna, Devi Tong, Senyu Michael Lee Haldimann, David Zhao, Yang Zhang, Bowen Gao, Chang Bartels, Chris Rao, Sushma Tran, Nathalie Lehnerer, Simon Giang, Co Dong, Patrick Pan, Junting Wang, Biyao Li, Dongxu Farajtabar, Mehrdad Hwang, Dongseong Duanmu, Grace Verma, Eshan Reddy, Sujeeth Shan, Qi Gao, Hongbin Du, Nan Sridhar, Pragnya Huang, Forrest Wang, Yingbo Bhendawade, Nikhil Zhu, Diane Aitharaju, Sai Hohman, Fred Gardiner, Lauren Chiu, Chung-Cheng Yang, Yinfei Kokmen, Alper Chu, Frank Ye, Ke Elgin, Kaan Levy, Oron Park, John Zhang, Donald Schoop, Eldon Wenzel, Nina Booker, Michael Kim, Hyunjik Erdenebileg, Chinguun Dun, Nan Yang, Eric Liang Chhatrapati, Priyal Mahtani, Vishaal Gang, Haiming Chia, Kohen Seshadri, Deepa Yu, Donghan Meng, Yan Peterson, Kelsey Yang, Zhen Wang, Yongqiang Peng, Carina Kang, Doug Agarwal, Anuva Antony, Albert Tebar, Juan Lao Jose, Albin Madappally Poston, Regan De Wang, Andy Casamayor, Gerard Amirloo, Elmira Yao, Violet Kryscinski, Wojciech Duan, Kun L, Lezhi Machine Learning Artificial Intelligence We introduce two multilingual, multimodal foundation language models that power Apple Intelligence features across Apple devices and services: i a 3B-parameter on-device model optimized for Apple silicon through architectural innovations such as KV-cache sharing and 2-bit quantization-aware training; and ii a scalable server model built on a novel Parallel-Track Mixture-of-Experts PT-MoE transformer that combines track parallelism, mixture-of-experts sparse computation, and interleaved global-local attention to deliver high quality with competitive cost on Apple's Private Cloud Compute platform. Both models are trained on large-scale multilingual and multimodal datasets sourced via responsible web crawling, licensed corpora, and high-quality synthetic data, then further refined with supervised fine-tuning and reinforcement learning on a new asynchronous platform. The resulting models support several additional languages while understanding images and executing tool calls. In public benchmarks and human evaluations, both the server model and the on-device model match or surpass comparably sized open baselines. A new Swift-centric Foundation Models framework exposes guided generation, constrained tool calling, and LoRA adapter fine-tuning, allowing developers to integrate these capabilities with a few lines of code. The latest advancements in Apple Intelligence models are grounded in our Responsible AI approach with safeguards like content filtering and locale-specific evaluation, as well as our commitment to protecting our users' privacy with innovations like Private Cloud Compute. |
| title | Apple Intelligence Foundation Language Models: Tech Report 2025 |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2507.13575 |