Rethinking Software Engineering in the Foundation Model Era: A Curated Catalogue of Challenges in the Development of Trustworthy FMware
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866914700575375360 |
|---|---|
| author | Hassan, Ahmed E. Lin, Dayi Rajbahadur, Gopi Krishnan Gallaba, Keheliya Cogo, Filipe R. Chen, Boyuan Zhang, Haoxiang Thangarajah, Kishanthan Oliva, Gustavo Ansaldi Lin, Jiahuei Abdullah, Wali Mohammad Jiang, Zhen Ming |
| author_facet | Hassan, Ahmed E. Lin, Dayi Rajbahadur, Gopi Krishnan Gallaba, Keheliya Cogo, Filipe R. Chen, Boyuan Zhang, Haoxiang Thangarajah, Kishanthan Oliva, Gustavo Ansaldi Lin, Jiahuei Abdullah, Wali Mohammad Jiang, Zhen Ming |
| contents | Foundation models (FMs), such as Large Language Models (LLMs), have revolutionized software development by enabling new use cases and business models. We refer to software built using FMs as FMware. The unique properties of FMware (e.g., prompts, agents, and the need for orchestration), coupled with the intrinsic limitations of FMs (e.g., hallucination) lead to a completely new set of software engineering challenges. Based on our industrial experience, we identified 10 key SE4FMware challenges that have caused enterprise FMware development to be unproductive, costly, and risky. In this paper, we discuss these challenges in detail and state the path for innovation that we envision. Next, we present FMArts, which is our long-term effort towards creating a cradle-to-grave platform for the engineering of trustworthy FMware. Finally, we (i) show how the unique properties of FMArts enabled us to design and develop a complex FMware for a large customer in a timely manner and (ii) discuss the lessons that we learned in doing so. We hope that the disclosure of the aforementioned challenges and our associated efforts to tackle them will not only raise awareness but also promote deeper and further discussions, knowledge sharing, and innovative solutions across the software engineering discipline. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_15943 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Rethinking Software Engineering in the Foundation Model Era: A Curated Catalogue of Challenges in the Development of Trustworthy FMware Hassan, Ahmed E. Lin, Dayi Rajbahadur, Gopi Krishnan Gallaba, Keheliya Cogo, Filipe R. Chen, Boyuan Zhang, Haoxiang Thangarajah, Kishanthan Oliva, Gustavo Ansaldi Lin, Jiahuei Abdullah, Wali Mohammad Jiang, Zhen Ming Software Engineering Artificial Intelligence Foundation models (FMs), such as Large Language Models (LLMs), have revolutionized software development by enabling new use cases and business models. We refer to software built using FMs as FMware. The unique properties of FMware (e.g., prompts, agents, and the need for orchestration), coupled with the intrinsic limitations of FMs (e.g., hallucination) lead to a completely new set of software engineering challenges. Based on our industrial experience, we identified 10 key SE4FMware challenges that have caused enterprise FMware development to be unproductive, costly, and risky. In this paper, we discuss these challenges in detail and state the path for innovation that we envision. Next, we present FMArts, which is our long-term effort towards creating a cradle-to-grave platform for the engineering of trustworthy FMware. Finally, we (i) show how the unique properties of FMArts enabled us to design and develop a complex FMware for a large customer in a timely manner and (ii) discuss the lessons that we learned in doing so. We hope that the disclosure of the aforementioned challenges and our associated efforts to tackle them will not only raise awareness but also promote deeper and further discussions, knowledge sharing, and innovative solutions across the software engineering discipline. |
| title | Rethinking Software Engineering in the Foundation Model Era: A Curated Catalogue of Challenges in the Development of Trustworthy FMware |
| topic | Software Engineering Artificial Intelligence |
| url | https://arxiv.org/abs/2402.15943 |