From Task to Tutorial: An Automated GUI Framework for Excel Tutorial Document and Video Creation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910150947766272 |
|---|---|
| author | Xie, Yuhang Mu, Jian Ma, Xiaojun Zhang, Chaoyun Wang, Lu Zhou, Mengyu Liu, Mugeng Qin, Si Lin, Qingwei Rajmohan, Saravan Han, Shi Zhang, Dongmei |
| author_facet | Xie, Yuhang Mu, Jian Ma, Xiaojun Zhang, Chaoyun Wang, Lu Zhou, Mengyu Liu, Mugeng Qin, Si Lin, Qingwei Rajmohan, Saravan Han, Shi Zhang, Dongmei |
| contents | Excel is one of the most widely used productivity tools across domains, offering rich functionality but also overwhelming users with its complexity. This creates a persistent demand for tutorials to support effective usage. However, while building and maintaining the Microsoft tutorial corpus, we observed that existing tutorials are manually created by experts, need frequent updates with each software release, and involve substantial human labor. Moreover, prior work has not achieved fully automated tutorial generation. In this paper, we present the first framework for automatically generating Excel tutorials directly from natural language task descriptions. Our framework first instantiates the task. Then a central component of this framework, Execution Agent, plans and executes the solution in Excel, and collects the intermediate artifacts required for tutorial construction. These artifacts are then transformed into both structured Excel documents and video demonstrations. To build a comprehensive tutorial corpus, we collected 1,559 task descriptions from real-world scenarios. In addition, we designed a systematic evaluation framework that integrates assessments from both large language models (LLMs) and human reviewers. Experimental results show that our framework improves task execution success rates by 8.5% over state-of-the-art baselines. Moreover, the generated tutorials demonstrate superior readability and instructional effectiveness, often approaching or surpassing expert-authored materials. Importantly, the automated pipeline eliminates manual labor and reduces time costs to 1/20 of expert authoring, making scalable and high-quality tutorial generation practical for the first time. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_21816 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | From Task to Tutorial: An Automated GUI Framework for Excel Tutorial Document and Video Creation Xie, Yuhang Mu, Jian Ma, Xiaojun Zhang, Chaoyun Wang, Lu Zhou, Mengyu Liu, Mugeng Qin, Si Lin, Qingwei Rajmohan, Saravan Han, Shi Zhang, Dongmei Software Engineering Excel is one of the most widely used productivity tools across domains, offering rich functionality but also overwhelming users with its complexity. This creates a persistent demand for tutorials to support effective usage. However, while building and maintaining the Microsoft tutorial corpus, we observed that existing tutorials are manually created by experts, need frequent updates with each software release, and involve substantial human labor. Moreover, prior work has not achieved fully automated tutorial generation. In this paper, we present the first framework for automatically generating Excel tutorials directly from natural language task descriptions. Our framework first instantiates the task. Then a central component of this framework, Execution Agent, plans and executes the solution in Excel, and collects the intermediate artifacts required for tutorial construction. These artifacts are then transformed into both structured Excel documents and video demonstrations. To build a comprehensive tutorial corpus, we collected 1,559 task descriptions from real-world scenarios. In addition, we designed a systematic evaluation framework that integrates assessments from both large language models (LLMs) and human reviewers. Experimental results show that our framework improves task execution success rates by 8.5% over state-of-the-art baselines. Moreover, the generated tutorials demonstrate superior readability and instructional effectiveness, often approaching or surpassing expert-authored materials. Importantly, the automated pipeline eliminates manual labor and reduces time costs to 1/20 of expert authoring, making scalable and high-quality tutorial generation practical for the first time. |
| title | From Task to Tutorial: An Automated GUI Framework for Excel Tutorial Document and Video Creation |
| topic | Software Engineering |
| url | https://arxiv.org/abs/2509.21816 |