Table of Contents: :: Library Catalog

Saved in:

Bibliographic Details
Main Author:	Anonymous, Anonymous
Format:	Recurso digital
Language:	English
Published:	Zenodo 2025
Online Access:	https://doi.org/10.5281/zenodo.17666438
Tags:	Add Tag No Tags, Be the first to tag this record!

Table of Contents:

<h2>Dataset Overview</h2> projects_discovery.zip Includes all auxiliary files used during dataset construction and during the generation of the CSV files contained in <code>repositories_dataset.zip</code>. <ul> <li> clone_repos.py – Clones all repositories listed in <code>top_repos.txt</code> into the source directory. The official repository list used in the study is provided in <code>top_repos_official.txt</code>. </li> <li> create_repo_links.py / extract_repo_links.py – Utility scripts responsible for normalizing, cleaning, and generating correct GitHub repository paths for Python projects. </li> <li> filter_repos_annotations.py – Filters repositories based on a minimum threshold of type-annotation coverage. Only projects meeting or exceeding the specified annotation percentage are retained. </li> <li> get_about.py – Extracts the “About” and descriptive metadata from GitHub repositories to support repository categorization. </li> <li> merge_topics.py – Merges topic metadata from multiple CSV sources into a unified structure. </li> </ul> type_hints.zip Contains all scripts responsible for extracting, aggregating, and processing type-annotation information. <ul> <li> type_extractor.py – Recursively scans all Python files inside each repository under <code>repos/</code>. For every file, it records all detected type annotations and outputs a CSV located at <code>output/{repository}.csv</code>. Each row contains: <code>["file", "member_name", "type", "member_type", "context_code"]</code>. </li> <li> type_extractor_metrics.py – Generates a per-repository summary from the extractor output. Produces a <code>summary.csv</code> including: <code>["repo", "total", "typed_total", "total_typed_%", "variables_total", "variables_typed", "variables_typed_%", "parameters_total", "parameters_typed", "parameters_typed_%", "returns_total", "returns_typed", "returns_typed_%"]</code>. A final GLOBAL row aggregates all repositories to provide overall statistics. </li> <li> strip_annotations.py – Produces multi-level code snippets (local, medium, full scope) for all annotated regions. Each repository receives a CSV containing the extracted snippets. These files are used for evaluating small LLMs on automatic type-inference tasks. </li> <li> delimiter.py / drop_duplicates.py – Utility scripts for data cleaning, duplicate removal, and consistent CSV formatting. </li> <li>pipeline.py - Contains a small pipeline where each snippet is inputted into a small size LLM model, deepseek-r1:7b in this case,  for automatic type inference performance.</li> <li>type_summary.py - Type to extract and create a category for each type seem at each repository unique csv file.</li> <li>add_category_column.py - Helper file to append the category column at the summary dataset.</li> </ul> resulting_dataset_sheets/ Contains the primary consolidated CSV files used for analysis in the paper. <ul> <li> repos_merged.csv – Integrates GitHub repository URLs, classification (library vs. non-library), and descriptive metadata such as repository topics and “About” fields. </li> <li> summary.csv – Merges all computed metrics required to answer each research question. </li> </ul> repositories_dataset/ A reduced sample of the full 136-GB dataset. Each file corresponds to the output of <code>type_extractor.py</code> for one repository. Columns are: <ul> <li> file – File path of the Python source where the annotation was found. </li> <li> member_name – Identifier of the function, variable, or member containing the annotation. </li> <li> type – The explicitly declared type annotation. </li> <li> context_code – The code snippet surrounding the annotation. </li> </ul> The complete dataset (not fully uploaded due to size constraints) contains the full set of extracted annotations across all analyzed repositories.

Similar Items