CodeWatcher: IDE Telemetry Data Extraction Tool for Understanding Coding Interactions with LLMs
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866911207670153216 |
|---|---|
| author | Basha, Manaal Ribeiro, Aimeê M. Javahar, Jeena de Souza, Cleidson R. B. Rodríguez-Pérez, Gema |
| author_facet | Basha, Manaal Ribeiro, Aimeê M. Javahar, Jeena de Souza, Cleidson R. B. Rodríguez-Pérez, Gema |
| contents | Understanding how developers interact with code generation tools (CGTs) requires detailed, real-time data on programming behavior which is often difficult to collect without disrupting workflow. We present \textit{CodeWatcher}, a lightweight, unobtrusive client-server system designed to capture fine-grained interaction events from within the Visual Studio Code (VS Code) editor. \textit{CodeWatcher} logs semantically meaningful events such as insertions made by CGTs, deletions, copy-paste actions, and focus shifts, enabling continuous monitoring of developer activity without modifying user workflows. The system comprises a VS Code plugin, a Python-based RESTful API, and a MongoDB backend, all containerized for scalability and ease of deployment. By structuring and timestamping each event, \textit{CodeWatcher} enables post-hoc reconstruction of coding sessions and facilitates rich behavioral analyses, including how and when CGTs are used during development. This infrastructure is crucial for supporting research on responsible AI, developer productivity, and the human-centered evaluation of CGTs. Please find the demo, diagrams, and tool here: https://osf.io/j2kru/overview. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_11536 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | CodeWatcher: IDE Telemetry Data Extraction Tool for Understanding Coding Interactions with LLMs Basha, Manaal Ribeiro, Aimeê M. Javahar, Jeena de Souza, Cleidson R. B. Rodríguez-Pérez, Gema Software Engineering Artificial Intelligence Understanding how developers interact with code generation tools (CGTs) requires detailed, real-time data on programming behavior which is often difficult to collect without disrupting workflow. We present \textit{CodeWatcher}, a lightweight, unobtrusive client-server system designed to capture fine-grained interaction events from within the Visual Studio Code (VS Code) editor. \textit{CodeWatcher} logs semantically meaningful events such as insertions made by CGTs, deletions, copy-paste actions, and focus shifts, enabling continuous monitoring of developer activity without modifying user workflows. The system comprises a VS Code plugin, a Python-based RESTful API, and a MongoDB backend, all containerized for scalability and ease of deployment. By structuring and timestamping each event, \textit{CodeWatcher} enables post-hoc reconstruction of coding sessions and facilitates rich behavioral analyses, including how and when CGTs are used during development. This infrastructure is crucial for supporting research on responsible AI, developer productivity, and the human-centered evaluation of CGTs. Please find the demo, diagrams, and tool here: https://osf.io/j2kru/overview. |
| title | CodeWatcher: IDE Telemetry Data Extraction Tool for Understanding Coding Interactions with LLMs |
| topic | Software Engineering Artificial Intelligence |
| url | https://arxiv.org/abs/2510.11536 |