| _version_ | 1866901854253744128 |
|---|---|
| author | Chembakottu, Bhagya P. Robillard, Martin |
| author_facet | Chembakottu, Bhagya P. Robillard, Martin |
| contents | <h3> </h3> <h3>A Survey of Web Application Security Tutorials: Replication Package</h3> <p> </p> <div>Bhagya Chembakottu and Martin P. Robillard </div> <div>School of Computer Science, McGill University, Montréal, Canada</div> <p> </p> <div> </div> <p><br>Abstract</p> <div> <div>This replication package provides the complete infrastructure to reproduce the study "A Survey of Web Application Security Tutorials" submitted in IEEE Software. The package includes automated scripts for data collection, ranking algorithms, text extraction, and statistical analysis. The replication can be executed on Windows, macOS, and Linux systems with minimal setup requirements.</div> <br> <div> <p>1. Introduction</p> </div> <br> <div>This replication package enables researchers and practitioners to reproduce our study on web application security tutorials. The package implements the complete methodology described in the paper, including query-based data collection, rank summation algorithms, manual coding guidelines, and statistical analysis procedures. More details of setting up sample replication study can be found [here](./documentation/methodology.md)</div> <br> <div>The replication is designed to be executed with minimal technical expertise, providing both graphical and command-line interfaces for different user preferences.</div> <div> </div> 2. Package Overview<br> <div> 2.1 Directory Structure</div> <div>```</div> <div>replication-package/</div> <div>├── README.md # This file</div> <div>├── requirements.txt # Python dependencies</div> <div>├── run-replication.sh # Unix/Linux/macOS script</div> <div>├── run-replication.bat # Windows batch script</div> <div>├── config.yaml # Configuration parameters</div> <div>├── data/ # Data storage</div> <div>│ ├── query-set-sample/ # Sample query results</div> <div>│ └── final-dataset/ # Final coded dataset</div> <div>├── scripts/ # Analysis scripts</div> <div>│ ├── 01-extract-queries.py</div> <div>│ ├── 02-rank-summation.py</div> <div>│ ├── 03-text-extraction.py</div> <div>│ ├── 04-statistical-analysis.py</div> <div>│ └── utils/</div> <div>├── documentation/</div> <div>│ ├── coding-guide.md</div> <div>│ └── methodology.md</div> <div>└── results/ # Output files</div> <div>```</div> <br> <div>2.2 System Requirements</div> <div>- Python 3.7 or higher</div> <div>- 2GB available disk space</div> <div>- Internet connection (for initial data collection)</div> <div>- Windows 10/11, macOS 10.14+, or Linux (Ubuntu 18.04+)</div> <br> <div>3 Installation</div> <br> <div>3.1 Automated Installation (Recommended)</div> <br> <div>Windows</div> <div>Double-click installation</div> <div> setup-windows.bat</div> <br><br><br> <div>3.2 Manual Installation</div> <div> Install Python dependencies:</div> <br> <div> pip install -r requirements.txt</div> <br><br><br> <div>4 Execution</div> <br> <div>4.1 Windows Execution</div> <br> <div>Option 1: Batch File (Recommended for beginners)</div> <br> <div>Double-click in File Explorer</div> <div>run-replication.bat</div> <br> <div>Or command prompt</div> <div>run-replication.bat</div> <br> <div>> Note : If any configuration issues try to run `fix-windows-issues.bat`</div> <br><br><br> <div>Option 3: Manual Command Line</div> <br> <div>Step-by-step execution</div> <div>- python scripts\01-extract-queries-sample.py --output data\query-set-sample</div> <div>- python scripts\02-rank-summation.py --input-dir data\query-set-sample --output results\top-25-urls.txt</div> <div>- python scripts\03-text-extraction.py --input data\final-dataset\security-tutorials-sample.csv --output results\text-lengths-sample.csv --simulate</div> <div>- python scripts\04-statistical-analysis.py --input results\text-lengths-sample.csv --output results\statistics-report-sample.txt</div> <br><br> <div>4.2 Unix/Linux/macOS Execution</div> <br> <div>Make executable and run</div> <div>chmod +x run-replication.sh</div> <div>./run-replication.sh</div> <br><br><br> <div>Option 2: Manual Execution</div> <br> <div>Step-by-step execution</div> <div>- python scripts/01-extract-queries-sample.py --output data/query-set-sample</div> <div>- python scripts/02-rank-summation.py --input-dir data/query-set-sample --output results/top-25-urls.txt</div> <div>- python scripts/03-text-extraction.py --input data/final-dataset/security-tutorials-sample.csv --output results/text-lengths-sample.csv --simulate</div> <div>- python scripts/04-statistical-analysis.py --input results/text-lengths-sample.csv --output results/statistics-report-sample.txt</div> <br><br> <div>5 Configuration</div> <br> <div>Edit `config.yaml` to customize:</div> <br> <div>```yaml</div> <div># Analysis parameters</div> <div>sample_settings:</div> <div> tutorial_count: 25</div> <div> statistical_significance: 0.05</div> <div> bonferroni_corrections: 4</div> <br> <div># File paths (auto-adjusted for OS)</div> <div>paths:</div> <div> windows_separator: "\"</div> <div> unix_separator: "/"</div> <div> query_pattern: "query_*.txt"</div> <br> <div># Execution settings</div> <div>execution:</div> <div> delay_between_requests: 2 # seconds</div> <div> timeout: 15 # seconds</div> <div> max_retries: 3</div> <div>```</div> <br> <div>6 Expected Outputs</div> <br> <div>6.1 Primary Results</div> <div>- **Top 25 URLs**: `results/top-25-urls.txt`</div> <div>- **Text lengths**: `results/text-lengths-sample.csv`</div> <div>- **Statistical report**: `results/statistics-report-sample.txt`</div> <br> <div>6.2 Sample Statistical Output</div> <div>```</div> <div>Statistical Analysis Results - Sample Dataset</div> <div>============================================================</div> <div>Sample size: 25 tutorials</div> <div>Bonferroni corrected alpha: 0.0125</div> <br> <div>Tutorial Types:</div> <div> Vendor Learning Resource: 18 (72.0%)</div> <div> Community / Education Project: 7 (28.0%)</div> <br> <div>Code Example Presence:</div> <div> none: 20 (80.0%)</div> <div> snippet: 3 (12.0%)</div> <div> full working code: 2 (8.0%)</div> <br> <div>Mann-Whitney U (Code vs No Code):</div> <div> U-statistic: 45.5</div> <div> p-value: 0.042</div> <div> Significant (α=0.0125): No</div> <div>```</div> <br> <div>7 Troubleshooting</div> <br> <div>7.1 Common Issues</div> <br> <div>| Issue | Windows Solution | Unix/Linux/macOS Solution |</div> <div>|-------|------------------|---------------------------|</div> <div>| Python not found | Install Python and check PATH | `which python` or install via package manager |</div> <div>| Permission denied | Run as Administrator | `chmod +x script.sh` |</div> <div>| Package installation fails | `pip install --user package` | `pip install --user package` or use `sudo` |</div> <div>| Network timeouts | Increase timeout in config.yaml | Same as Windows |</div> <div>| Character encoding errors | Set `chcp 65001` in CMD | Set `export PYTHONIOENCODING=utf-8` |</div> <br> <div>7.2 Platform-Specific Debugging</div> <br> <div>Windows Debugging</div> <div>```batch</div> <div># Check Python installation</div> <div>where python</div> <div>python --version</div> <br> <div># Check package installation</div> <div>pip list | findstr pandas</div> <br> <div># Run with verbose output</div> <div>python -v scripts\01-extract-queries-sample.py --output data\query-set-sample</div> <div>```</div> <br> <div>Unix/Linux/macOS Debugging</div> <div>```bash</div> <div># Check Python installation</div> <div>which python</div> <div>python --version</div> <br> <div># Check package installation</div> <div>pip list | grep pandas</div> <br> <div># Run with verbose output</div> <div>python -v scripts/01-extract-queries-sample.py --output data/query-set-sample</div> <div>```</div> <br> <div>9 Validation</div> <br> <div>9.2 Expected Validation Results</div> <div>```bash</div> <div># Validate installation</div> <div>python -c "import pandas, numpy, scipy; print('Core packages OK')"</div> <br> <div># Validate data integrity</div> <div>python -c "import pandas as pd; df = pd.read_csv('data/final-dataset/security-tutorials-sample.csv'); print(f'Sample size: {len(df)}')"</div> <br> <div># Validate statistical output</div> <div>python -c "with open('results/statistics-report-sample.txt') as f: content = f.read(); print('Statistical report generated' if 'Mann-Whitney U' in content else 'Missing statistics')"</div> <div>```</div> <br><br><br> <div>10. Full Dataset Replication</div> <div>To replicate with the full 132-tutorial dataset:</div> <div>1. Modify `config.yaml` to use full query set</div> <div>2. Run with original query parameters</div> <div>3. Apply manual coding to top 200 URLs</div> <div>4. Execute full statistical analysis</div> </div> |
| format | Recurso digital |
| id | zenodo_https___doi_org_10_5281_zenodo_18905186 |
| institution | Zenodo |
| language | |
| publishDate | 2026 |
| publisher | Zenodo |
| record_format | zenodo |
| spellingShingle | A Survey of Web Application Security Tutorials Chembakottu, Bhagya P. Robillard, Martin <h3> </h3> <h3>A Survey of Web Application Security Tutorials: Replication Package</h3> <p> </p> <div>Bhagya Chembakottu and Martin P. Robillard </div> <div>School of Computer Science, McGill University, Montréal, Canada</div> <p> </p> <div> </div> <p><br>Abstract</p> <div> <div>This replication package provides the complete infrastructure to reproduce the study "A Survey of Web Application Security Tutorials" submitted in IEEE Software. The package includes automated scripts for data collection, ranking algorithms, text extraction, and statistical analysis. The replication can be executed on Windows, macOS, and Linux systems with minimal setup requirements.</div> <br> <div> <p>1. Introduction</p> </div> <br> <div>This replication package enables researchers and practitioners to reproduce our study on web application security tutorials. The package implements the complete methodology described in the paper, including query-based data collection, rank summation algorithms, manual coding guidelines, and statistical analysis procedures. More details of setting up sample replication study can be found [here](./documentation/methodology.md)</div> <br> <div>The replication is designed to be executed with minimal technical expertise, providing both graphical and command-line interfaces for different user preferences.</div> <div> </div> 2. Package Overview<br> <div> 2.1 Directory Structure</div> <div>```</div> <div>replication-package/</div> <div>├── README.md # This file</div> <div>├── requirements.txt # Python dependencies</div> <div>├── run-replication.sh # Unix/Linux/macOS script</div> <div>├── run-replication.bat # Windows batch script</div> <div>├── config.yaml # Configuration parameters</div> <div>├── data/ # Data storage</div> <div>│ ├── query-set-sample/ # Sample query results</div> <div>│ └── final-dataset/ # Final coded dataset</div> <div>├── scripts/ # Analysis scripts</div> <div>│ ├── 01-extract-queries.py</div> <div>│ ├── 02-rank-summation.py</div> <div>│ ├── 03-text-extraction.py</div> <div>│ ├── 04-statistical-analysis.py</div> <div>│ └── utils/</div> <div>├── documentation/</div> <div>│ ├── coding-guide.md</div> <div>│ └── methodology.md</div> <div>└── results/ # Output files</div> <div>```</div> <br> <div>2.2 System Requirements</div> <div>- Python 3.7 or higher</div> <div>- 2GB available disk space</div> <div>- Internet connection (for initial data collection)</div> <div>- Windows 10/11, macOS 10.14+, or Linux (Ubuntu 18.04+)</div> <br> <div>3 Installation</div> <br> <div>3.1 Automated Installation (Recommended)</div> <br> <div>Windows</div> <div>Double-click installation</div> <div> setup-windows.bat</div> <br><br><br> <div>3.2 Manual Installation</div> <div> Install Python dependencies:</div> <br> <div> pip install -r requirements.txt</div> <br><br><br> <div>4 Execution</div> <br> <div>4.1 Windows Execution</div> <br> <div>Option 1: Batch File (Recommended for beginners)</div> <br> <div>Double-click in File Explorer</div> <div>run-replication.bat</div> <br> <div>Or command prompt</div> <div>run-replication.bat</div> <br> <div>> Note : If any configuration issues try to run `fix-windows-issues.bat`</div> <br><br><br> <div>Option 3: Manual Command Line</div> <br> <div>Step-by-step execution</div> <div>- python scripts\01-extract-queries-sample.py --output data\query-set-sample</div> <div>- python scripts\02-rank-summation.py --input-dir data\query-set-sample --output results\top-25-urls.txt</div> <div>- python scripts\03-text-extraction.py --input data\final-dataset\security-tutorials-sample.csv --output results\text-lengths-sample.csv --simulate</div> <div>- python scripts\04-statistical-analysis.py --input results\text-lengths-sample.csv --output results\statistics-report-sample.txt</div> <br><br> <div>4.2 Unix/Linux/macOS Execution</div> <br> <div>Make executable and run</div> <div>chmod +x run-replication.sh</div> <div>./run-replication.sh</div> <br><br><br> <div>Option 2: Manual Execution</div> <br> <div>Step-by-step execution</div> <div>- python scripts/01-extract-queries-sample.py --output data/query-set-sample</div> <div>- python scripts/02-rank-summation.py --input-dir data/query-set-sample --output results/top-25-urls.txt</div> <div>- python scripts/03-text-extraction.py --input data/final-dataset/security-tutorials-sample.csv --output results/text-lengths-sample.csv --simulate</div> <div>- python scripts/04-statistical-analysis.py --input results/text-lengths-sample.csv --output results/statistics-report-sample.txt</div> <br><br> <div>5 Configuration</div> <br> <div>Edit `config.yaml` to customize:</div> <br> <div>```yaml</div> <div># Analysis parameters</div> <div>sample_settings:</div> <div> tutorial_count: 25</div> <div> statistical_significance: 0.05</div> <div> bonferroni_corrections: 4</div> <br> <div># File paths (auto-adjusted for OS)</div> <div>paths:</div> <div> windows_separator: "\"</div> <div> unix_separator: "/"</div> <div> query_pattern: "query_*.txt"</div> <br> <div># Execution settings</div> <div>execution:</div> <div> delay_between_requests: 2 # seconds</div> <div> timeout: 15 # seconds</div> <div> max_retries: 3</div> <div>```</div> <br> <div>6 Expected Outputs</div> <br> <div>6.1 Primary Results</div> <div>- **Top 25 URLs**: `results/top-25-urls.txt`</div> <div>- **Text lengths**: `results/text-lengths-sample.csv`</div> <div>- **Statistical report**: `results/statistics-report-sample.txt`</div> <br> <div>6.2 Sample Statistical Output</div> <div>```</div> <div>Statistical Analysis Results - Sample Dataset</div> <div>============================================================</div> <div>Sample size: 25 tutorials</div> <div>Bonferroni corrected alpha: 0.0125</div> <br> <div>Tutorial Types:</div> <div> Vendor Learning Resource: 18 (72.0%)</div> <div> Community / Education Project: 7 (28.0%)</div> <br> <div>Code Example Presence:</div> <div> none: 20 (80.0%)</div> <div> snippet: 3 (12.0%)</div> <div> full working code: 2 (8.0%)</div> <br> <div>Mann-Whitney U (Code vs No Code):</div> <div> U-statistic: 45.5</div> <div> p-value: 0.042</div> <div> Significant (α=0.0125): No</div> <div>```</div> <br> <div>7 Troubleshooting</div> <br> <div>7.1 Common Issues</div> <br> <div>| Issue | Windows Solution | Unix/Linux/macOS Solution |</div> <div>|-------|------------------|---------------------------|</div> <div>| Python not found | Install Python and check PATH | `which python` or install via package manager |</div> <div>| Permission denied | Run as Administrator | `chmod +x script.sh` |</div> <div>| Package installation fails | `pip install --user package` | `pip install --user package` or use `sudo` |</div> <div>| Network timeouts | Increase timeout in config.yaml | Same as Windows |</div> <div>| Character encoding errors | Set `chcp 65001` in CMD | Set `export PYTHONIOENCODING=utf-8` |</div> <br> <div>7.2 Platform-Specific Debugging</div> <br> <div>Windows Debugging</div> <div>```batch</div> <div># Check Python installation</div> <div>where python</div> <div>python --version</div> <br> <div># Check package installation</div> <div>pip list | findstr pandas</div> <br> <div># Run with verbose output</div> <div>python -v scripts\01-extract-queries-sample.py --output data\query-set-sample</div> <div>```</div> <br> <div>Unix/Linux/macOS Debugging</div> <div>```bash</div> <div># Check Python installation</div> <div>which python</div> <div>python --version</div> <br> <div># Check package installation</div> <div>pip list | grep pandas</div> <br> <div># Run with verbose output</div> <div>python -v scripts/01-extract-queries-sample.py --output data/query-set-sample</div> <div>```</div> <br> <div>9 Validation</div> <br> <div>9.2 Expected Validation Results</div> <div>```bash</div> <div># Validate installation</div> <div>python -c "import pandas, numpy, scipy; print('Core packages OK')"</div> <br> <div># Validate data integrity</div> <div>python -c "import pandas as pd; df = pd.read_csv('data/final-dataset/security-tutorials-sample.csv'); print(f'Sample size: {len(df)}')"</div> <br> <div># Validate statistical output</div> <div>python -c "with open('results/statistics-report-sample.txt') as f: content = f.read(); print('Statistical report generated' if 'Mann-Whitney U' in content else 'Missing statistics')"</div> <div>```</div> <br><br><br> <div>10. Full Dataset Replication</div> <div>To replicate with the full 132-tutorial dataset:</div> <div>1. Modify `config.yaml` to use full query set</div> <div>2. Run with original query parameters</div> <div>3. Apply manual coding to top 200 URLs</div> <div>4. Execute full statistical analysis</div> </div> |
| title | A Survey of Web Application Security Tutorials |
| url | https://doi.org/10.5281/zenodo.18905186 |