Salvato in:
| Autori principali: | , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | https://arxiv.org/abs/2402.11723 |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866929247884410880 |
|---|---|
| author | Dhillon, Paramveer S. Molaei, Somayeh Li, Jiaqi Golub, Maximilian Zheng, Shaochun Robert, Lionel P. |
| author_facet | Dhillon, Paramveer S. Molaei, Somayeh Li, Jiaqi Golub, Maximilian Zheng, Shaochun Robert, Lionel P. |
| contents | Advances in language modeling have paved the way for novel human-AI co-writing experiences. This paper explores how varying levels of scaffolding from large language models (LLMs) shape the co-writing process. Employing a within-subjects field experiment with a Latin square design, we asked participants (N=131) to respond to argumentative writing prompts under three randomly sequenced conditions: no AI assistance (control), next-sentence suggestions (low scaffolding), and next-paragraph suggestions (high scaffolding). Our findings reveal a U-shaped impact of scaffolding on writing quality and productivity (words/time). While low scaffolding did not significantly improve writing quality or productivity, high scaffolding led to significant improvements, especially benefiting non-regular writers and less tech-savvy users. No significant cognitive burden was observed while using the scaffolded writing tools, but a moderate decrease in text ownership and satisfaction was noted. Our results have broad implications for the design of AI-powered writing tools, including the need for personalized scaffolding mechanisms. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_11723 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Shaping Human-AI Collaboration: Varied Scaffolding Levels in Co-writing with Language Models Dhillon, Paramveer S. Molaei, Somayeh Li, Jiaqi Golub, Maximilian Zheng, Shaochun Robert, Lionel P. Human-Computer Interaction Computation and Language Advances in language modeling have paved the way for novel human-AI co-writing experiences. This paper explores how varying levels of scaffolding from large language models (LLMs) shape the co-writing process. Employing a within-subjects field experiment with a Latin square design, we asked participants (N=131) to respond to argumentative writing prompts under three randomly sequenced conditions: no AI assistance (control), next-sentence suggestions (low scaffolding), and next-paragraph suggestions (high scaffolding). Our findings reveal a U-shaped impact of scaffolding on writing quality and productivity (words/time). While low scaffolding did not significantly improve writing quality or productivity, high scaffolding led to significant improvements, especially benefiting non-regular writers and less tech-savvy users. No significant cognitive burden was observed while using the scaffolded writing tools, but a moderate decrease in text ownership and satisfaction was noted. Our results have broad implications for the design of AI-powered writing tools, including the need for personalized scaffolding mechanisms. |
| title | Shaping Human-AI Collaboration: Varied Scaffolding Levels in Co-writing with Language Models |
| topic | Human-Computer Interaction Computation and Language |
| url | https://arxiv.org/abs/2402.11723 |