Adobe Summit Concierge Evaluation with Human in the Loop
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866915598847442944 |
|---|---|
| author | Chen, Yiru Fang, Sally Harsha, Sai Sree Luo, Dan Muppala, Vaishnavi Wu, Fei Jiang, Shun Qian, Kun Li, Yunyao |
| author_facet | Chen, Yiru Fang, Sally Harsha, Sai Sree Luo, Dan Muppala, Vaishnavi Wu, Fei Jiang, Shun Qian, Kun Li, Yunyao |
| contents | Generative AI assistants offer significant potential to enhance productivity, streamline information access, and improve user experience in enterprise contexts. In this work, we present Summit Concierge, a domain-specific AI assistant developed for Adobe Summit. The assistant handles a wide range of event-related queries and operates under real-world constraints such as data sparsity, quality assurance, and rapid deployment. To address these challenges, we adopt a human-in-the-loop development workflow that combines prompt engineering, retrieval grounding, and lightweight human validation. We describe the system architecture, development process, and real-world deployment outcomes. Our experience shows that agile, feedback-driven development enables scalable and reliable AI assistants, even in cold-start scenarios. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_03186 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Adobe Summit Concierge Evaluation with Human in the Loop Chen, Yiru Fang, Sally Harsha, Sai Sree Luo, Dan Muppala, Vaishnavi Wu, Fei Jiang, Shun Qian, Kun Li, Yunyao Artificial Intelligence Generative AI assistants offer significant potential to enhance productivity, streamline information access, and improve user experience in enterprise contexts. In this work, we present Summit Concierge, a domain-specific AI assistant developed for Adobe Summit. The assistant handles a wide range of event-related queries and operates under real-world constraints such as data sparsity, quality assurance, and rapid deployment. To address these challenges, we adopt a human-in-the-loop development workflow that combines prompt engineering, retrieval grounding, and lightweight human validation. We describe the system architecture, development process, and real-world deployment outcomes. Our experience shows that agile, feedback-driven development enables scalable and reliable AI assistants, even in cold-start scenarios. |
| title | Adobe Summit Concierge Evaluation with Human in the Loop |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2511.03186 |