Beyond the Comfort Zone: Emerging Solutions to Overcome Challenges in Integrating LLMs into Software Products
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866912144783572992 |
|---|---|
| author | Nahar, Nadia Kästner, Christian Butler, Jenna Parnin, Chris Zimmermann, Thomas Bird, Christian |
| author_facet | Nahar, Nadia Kästner, Christian Butler, Jenna Parnin, Chris Zimmermann, Thomas Bird, Christian |
| contents | Large Language Models (LLMs) are increasingly embedded into software products across diverse industries, enhancing user experiences, but at the same time introducing numerous challenges for developers. Unique characteristics of LLMs force developers, who are accustomed to traditional software development and evaluation, out of their comfort zones as the LLM components shatter standard assumptions about software systems. This study explores the emerging solutions that software developers are adopting to navigate the encountered challenges. Leveraging a mixed-method research, including 26 interviews and a survey with 332 responses, the study identifies 19 emerging solutions regarding quality assurance that practitioners across several product teams at Microsoft are exploring. The findings provide valuable insights that can guide the development and evaluation of LLM-based products more broadly in the face of these challenges. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_12071 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Beyond the Comfort Zone: Emerging Solutions to Overcome Challenges in Integrating LLMs into Software Products Nahar, Nadia Kästner, Christian Butler, Jenna Parnin, Chris Zimmermann, Thomas Bird, Christian Software Engineering Machine Learning Large Language Models (LLMs) are increasingly embedded into software products across diverse industries, enhancing user experiences, but at the same time introducing numerous challenges for developers. Unique characteristics of LLMs force developers, who are accustomed to traditional software development and evaluation, out of their comfort zones as the LLM components shatter standard assumptions about software systems. This study explores the emerging solutions that software developers are adopting to navigate the encountered challenges. Leveraging a mixed-method research, including 26 interviews and a survey with 332 responses, the study identifies 19 emerging solutions regarding quality assurance that practitioners across several product teams at Microsoft are exploring. The findings provide valuable insights that can guide the development and evaluation of LLM-based products more broadly in the face of these challenges. |
| title | Beyond the Comfort Zone: Emerging Solutions to Overcome Challenges in Integrating LLMs into Software Products |
| topic | Software Engineering Machine Learning |
| url | https://arxiv.org/abs/2410.12071 |