Does Context Size Matter? Context Structure and Integration Effort in Agent Pull Requests
Fuente:
Zenodo
Salvato in:
| Autore principale: | |
|---|---|
| Natura: | Recurso digital |
| Pubblicazione: |
Zenodo
2026
|
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866902157339394048 |
|---|---|
| author | Anonymous, Anonymous |
| author_facet | Anonymous, Anonymous |
| contents | <p>This is the replication package for the paper: Does Context Size Matter? An Empirical Study of Agent-Authored Pull Requests</p> <p>All experiments are provided as Python notebooks and were run on Google Colaboratory.</p> <p>Requires:</p> <ul> <li>Huggingface API key</li> <li>OpenAI API key</li> <li>Connection to Google Drive with required files (to run the categorization and evaluations)</li> <li>GitHub token (to fetch commit dates, although we provide the extracted data)</li> </ul> <p>The zip package contains the following files:</p> <h2>Data exploration:</h2> <ul> <li>data_exploration: <ul> <li>AIDev-pop data exploration.ipynb: general data exploration of the dataset, fix PR types.</li> </ul> </li> </ul> <h2>For <strong>RQ1</strong>:</h2> <ul> <li>first_commits (<strong>RQ1</strong>): <ul> <li>AIDev-pop single-bot-commits.ipynb</li> </ul> </li> </ul> <h2>For <strong>RQ2</strong>: </h2> <ul> <li>commits_and_comments_past1st: <ul> <li>AIDev-Fix Review Commits-Comments.ipynb: notebook to explore commits and comments</li> </ul> </li> <li>commits_data: <ul> <li>AIDev-pop fetch commit dates.ipynb: fetch missing commit dates associated with commit details for agent commits.</li> <li>commit_datetime_cache.csv: extracted commit dates with PR ids.</li> <li>failed_commits_humans.csv: extracted commits on rejected agent PRs by human users</li> <li>success_commits_humans.csv: extracted commits on accepted agent PRs by human users</li> </ul> </li> </ul> <ul> <li>comments_data: <ul> <li>pr_comments_failed_full.csv: extraction of comments on rejected PRs</li> <li>pr_comments_success_full.csv: extraction of comments on accepted PRs</li> </ul> </li> <li>llm_categorization: <ul> <li>AIDev comment categorization.ipynb: automated labelling of human review comments by GPT4.1</li> <li>comments-eval.ipynb: results analysis, automated + manual labeling of comments</li> <li>categorize-commits.ipynb: automated labeling of human commits by GPT4.1</li> <li>commits-eval.ipynb: results analysis, automated + manual labeling of commits</li> <li>success_commits_llm.csv: LLM categorization of commits on accepted PRs</li> <li>success_comments_llm.csv: LLM categorization of comments on accepted PRs</li> </ul> </li> <li>manual_categorization: <ul> <li>pr_comments_failed_sample_w_cat.csv: 50 sampled comments for manual evaluation, rejected PRs</li> <li>pr_comments_success_sample_w_cat.csv: 50sampled comments for manual evaluation, accepted PRs</li> <li>sampled_commits_fail_w_cat.csv: 50sampled commits for manual evaluation, rejected PRs</li> <li>sampled_commits_success_w_cat.csv: 50 sampled commits for manual evaluation, accepted PRs</li> </ul> </li> </ul> |
| format | Recurso digital |
| id | zenodo_https___doi_org_10_5281_zenodo_18040248 |
| institution | Zenodo |
| language | |
| publishDate | 2026 |
| publisher | Zenodo |
| record_format | zenodo |
| spellingShingle | Does Context Size Matter? Context Structure and Integration Effort in Agent Pull Requests Anonymous, Anonymous <p>This is the replication package for the paper: Does Context Size Matter? An Empirical Study of Agent-Authored Pull Requests</p> <p>All experiments are provided as Python notebooks and were run on Google Colaboratory.</p> <p>Requires:</p> <ul> <li>Huggingface API key</li> <li>OpenAI API key</li> <li>Connection to Google Drive with required files (to run the categorization and evaluations)</li> <li>GitHub token (to fetch commit dates, although we provide the extracted data)</li> </ul> <p>The zip package contains the following files:</p> <h2>Data exploration:</h2> <ul> <li>data_exploration: <ul> <li>AIDev-pop data exploration.ipynb: general data exploration of the dataset, fix PR types.</li> </ul> </li> </ul> <h2>For <strong>RQ1</strong>:</h2> <ul> <li>first_commits (<strong>RQ1</strong>): <ul> <li>AIDev-pop single-bot-commits.ipynb</li> </ul> </li> </ul> <h2>For <strong>RQ2</strong>: </h2> <ul> <li>commits_and_comments_past1st: <ul> <li>AIDev-Fix Review Commits-Comments.ipynb: notebook to explore commits and comments</li> </ul> </li> <li>commits_data: <ul> <li>AIDev-pop fetch commit dates.ipynb: fetch missing commit dates associated with commit details for agent commits.</li> <li>commit_datetime_cache.csv: extracted commit dates with PR ids.</li> <li>failed_commits_humans.csv: extracted commits on rejected agent PRs by human users</li> <li>success_commits_humans.csv: extracted commits on accepted agent PRs by human users</li> </ul> </li> </ul> <ul> <li>comments_data: <ul> <li>pr_comments_failed_full.csv: extraction of comments on rejected PRs</li> <li>pr_comments_success_full.csv: extraction of comments on accepted PRs</li> </ul> </li> <li>llm_categorization: <ul> <li>AIDev comment categorization.ipynb: automated labelling of human review comments by GPT4.1</li> <li>comments-eval.ipynb: results analysis, automated + manual labeling of comments</li> <li>categorize-commits.ipynb: automated labeling of human commits by GPT4.1</li> <li>commits-eval.ipynb: results analysis, automated + manual labeling of commits</li> <li>success_commits_llm.csv: LLM categorization of commits on accepted PRs</li> <li>success_comments_llm.csv: LLM categorization of comments on accepted PRs</li> </ul> </li> <li>manual_categorization: <ul> <li>pr_comments_failed_sample_w_cat.csv: 50 sampled comments for manual evaluation, rejected PRs</li> <li>pr_comments_success_sample_w_cat.csv: 50sampled comments for manual evaluation, accepted PRs</li> <li>sampled_commits_fail_w_cat.csv: 50sampled commits for manual evaluation, rejected PRs</li> <li>sampled_commits_success_w_cat.csv: 50 sampled commits for manual evaluation, accepted PRs</li> </ul> </li> </ul> |
| title | Does Context Size Matter? Context Structure and Integration Effort in Agent Pull Requests |
| url | https://doi.org/10.5281/zenodo.18040248 |