| _version_ | 1866901715277578240 |
|---|---|
| author | Reddy, Veerababu Poola, Seetha Ram shaik, musharaf kistaparapu, sai durga sajja, vignesh |
| author_facet | Reddy, Veerababu Poola, Seetha Ram shaik, musharaf kistaparapu, sai durga sajja, vignesh |
| contents | <div> <div> <h2>Description</h2> <p>This release contains code and supporting files for an image captioning project based on transformer decoding, bottom-up visual region features, and LDCAP/SCST-style training. It is provided as a research software package for reproducibility, reuse, and extension.</p> <p>The repository includes:</p> <ul> <li> <p>Streamlit inference application</p> </li> <li> <p>Inference pipeline</p> </li> <li> <p>Project configuration module</p> </li> <li> <p>FiLM-based sifting attention module</p> </li> <li> <p>Transformer model definition used in inference</p> </li> <li> <p>RIN module used in the ASCAP variant</p> </li> <li> <p>Cross-entropy (XE) training script</p> </li> <li> <p>Self-critical sequence training (SCST) script</p> </li> <li> <p>Dataset validation script</p> </li> </ul> <p>The package supports:</p> <ul> <li> <p>Local inference and demonstration</p> </li> <li> <p>Documentation of the training and inference workflow</p> </li> <li> <p>Partial reproduction of the experimental pipeline</p> </li> </ul> <h2>External Resources Required</h2> <p>The following external datasets are referenced by the project:</p> <ol> <li> <p><code>/kaggle/input/datasets/mariofrcrce/coco-bottom-up-features-adaptive-k</code><br>Pre-extracted COCO adaptive-k bottom-up features used during training and validation</p> </li> <li> <p><code>/kaggle/input/datasets/musharaf5/coco-karpathy-split</code><br>Karpathy train/validation/test split and caption references</p> </li> <li> <p><code>/kaggle/input/datasets/nadaibrahim/coco2014</code><br>COCO 2014 images and annotations</p> </li> <li> <p><code>/kaggle/input/datasets/musharaf5/scap-source-code</code><br>External SCAP source code used by <code>ascap_encoder.py</code> and <code>ascap_decoder.py</code></p> </li> <li> <p><code>/kaggle/input/datasets/soumikrakshit/lol-dataset</code><br>Included but not used by the scripts in this repository snapshot</p> </li> </ol> <p>Associated notebook:</p> <ul> <li> <p><code>https://www.kaggle.com/code/musharaf5/caption</code></p> </li> </ul> <p>Runtime assets:</p> <ul> <li> <p><code>https://drive.google.com/drive/folders/1oLuZ2tZIJuFgclueSr0xK92NQsiNMHi0</code><br>(model checkpoint and <code>vocab.json</code>)</p> </li> </ul> <h2>Contents</h2> <pre><code>LDCAP-Caption/ |-- .streamlit/ |-- checkpoints/ |-- models/ |-- venv/ |-- __pycache__/ |-- app.py |-- ascap_decoder.py |-- ascap_encoder.py |-- config.py |-- film_sifting.py |-- inference.py |-- inference (1).py |-- README.md |-- requirements.txt |-- rin.py |-- test_dataset.py |-- test_image.jpg |-- train_scst.py |-- train_xe.py |-- vocab.json </code></pre> <p>Key components:</p> <ul> <li> <p><code>app.py</code>: Streamlit interface for caption generation</p> </li> <li> <p><code>inference.py</code>: Main inference pipeline</p> </li> <li> <p><code>train_xe.py</code>: Stage-1 training</p> </li> <li> <p><code>train_scst.py</code>: Stage-2 training</p> </li> <li> <p><code>film_sifting.py</code>: FiLM-based attention module</p> </li> <li> <p><code>rin.py</code>: RIN module</p> </li> <li> <p><code>config.py</code>: Configuration and paths</p> </li> <li> <p><code>test_dataset.py</code>: Dataset validation</p> </li> <li> <p><code>vocab.json</code>: Vocabulary file</p> </li> </ul> <h2>Software Requirements</h2> <ul> <li> <p>Python 3.10</p> </li> <li> <p>pip 24.x</p> </li> <li> <p>GPU recommended for training</p> </li> <li> <p>CPU supported for inference</p> </li> </ul> <p>Dependencies:</p> <pre><code>streamlit==1.35.0 torch==2.6.0 torchvision==0.21.0 Pillow==10.4.0 numpy==1.26.4 tqdm==4.67.1 </code></pre> <p>Additional:</p> <pre><code>pycocoevalcap </code></pre> <h2>Installation</h2> <h3>Local</h3> <pre><code class="language-bash">python -m venv venv source venv/bin/activate # or use Windows activation pip install --upgrade pip pip install -r requirements.txt pip install pycocoevalcap </code></pre> <h3>Kaggle</h3> <ol> <li> <p>Attach required datasets</p> </li> <li> <p>Place repository in <code>/kaggle/working/</code></p> </li> <li> <p>Verify paths in configuration</p> </li> <li> <p>Run validation and training scripts</p> </li> </ol> <h2>Reproducibility Scope</h2> <p>This release includes:</p> <ul> <li> <p>Inference and Streamlit interface</p> </li> <li> <p>Configuration aligned with Kaggle-based training</p> </li> <li> <p>Dataset validation script</p> </li> <li> <p>XE and SCST training scripts</p> </li> <li> <p>ASCAP-related modules</p> </li> <li> <p>Checkpoint and vocabulary (external download)</p> </li> </ul> <h2>Workflow</h2> <h3>Dataset Validation</h3> <pre><code class="language-bash">python test_dataset.py </code></pre> <h3>XE Training</h3> <pre><code class="language-bash">python train_xe.py </code></pre> <p>Output:</p> <pre><code>checkpoints/xe_best_model.pt </code></pre> <h3>SCST Training</h3> <pre><code class="language-bash">python train_scst.py </code></pre> <p>Outputs:</p> <pre><code>checkpoints/scst_best_model.pt checkpoints/scst_checkpoint.pt </code></pre> <h2>Inference</h2> <p>Download required files and place:</p> <pre><code>checkpoints/scst_best_model.pt vocab.json </code></pre> <p>Run:</p> <pre><code class="language-bash">streamlit run app.py </code></pre> <p>Access:</p> <pre><code>http://localhost:8501 </code></pre> <h2>Limitations</h2> <ul> <li> <p>Path-dependent training workflow</p> </li> <li> <p>Kaggle-specific paths in some scripts</p> </li> <li> <p><code>pycocoevalcap</code> not included in requirements</p> </li> <li> <p>No dedicated evaluation script</p> </li> <li> <p>No environment lock file</p> </li> </ul> <h2>Reuse</h2> <p>This release provides the software, configuration, and workflow as included in this repository snapshot for reproducibility and reuse.</p> </div> </div> |
| format | Recurso digital |
| id | zenodo_https___doi_org_10_5281_zenodo_19842490 |
| institution | Zenodo |
| language | eng |
| publishDate | 2026 |
| publisher | Zenodo |
| record_format | zenodo |
| spellingShingle | Robust Image Captioning via Latent Compression and Dynamic Decoder Conditioning Reddy, Veerababu Poola, Seetha Ram shaik, musharaf kistaparapu, sai durga sajja, vignesh <div> <div> <h2>Description</h2> <p>This release contains code and supporting files for an image captioning project based on transformer decoding, bottom-up visual region features, and LDCAP/SCST-style training. It is provided as a research software package for reproducibility, reuse, and extension.</p> <p>The repository includes:</p> <ul> <li> <p>Streamlit inference application</p> </li> <li> <p>Inference pipeline</p> </li> <li> <p>Project configuration module</p> </li> <li> <p>FiLM-based sifting attention module</p> </li> <li> <p>Transformer model definition used in inference</p> </li> <li> <p>RIN module used in the ASCAP variant</p> </li> <li> <p>Cross-entropy (XE) training script</p> </li> <li> <p>Self-critical sequence training (SCST) script</p> </li> <li> <p>Dataset validation script</p> </li> </ul> <p>The package supports:</p> <ul> <li> <p>Local inference and demonstration</p> </li> <li> <p>Documentation of the training and inference workflow</p> </li> <li> <p>Partial reproduction of the experimental pipeline</p> </li> </ul> <h2>External Resources Required</h2> <p>The following external datasets are referenced by the project:</p> <ol> <li> <p><code>/kaggle/input/datasets/mariofrcrce/coco-bottom-up-features-adaptive-k</code><br>Pre-extracted COCO adaptive-k bottom-up features used during training and validation</p> </li> <li> <p><code>/kaggle/input/datasets/musharaf5/coco-karpathy-split</code><br>Karpathy train/validation/test split and caption references</p> </li> <li> <p><code>/kaggle/input/datasets/nadaibrahim/coco2014</code><br>COCO 2014 images and annotations</p> </li> <li> <p><code>/kaggle/input/datasets/musharaf5/scap-source-code</code><br>External SCAP source code used by <code>ascap_encoder.py</code> and <code>ascap_decoder.py</code></p> </li> <li> <p><code>/kaggle/input/datasets/soumikrakshit/lol-dataset</code><br>Included but not used by the scripts in this repository snapshot</p> </li> </ol> <p>Associated notebook:</p> <ul> <li> <p><code>https://www.kaggle.com/code/musharaf5/caption</code></p> </li> </ul> <p>Runtime assets:</p> <ul> <li> <p><code>https://drive.google.com/drive/folders/1oLuZ2tZIJuFgclueSr0xK92NQsiNMHi0</code><br>(model checkpoint and <code>vocab.json</code>)</p> </li> </ul> <h2>Contents</h2> <pre><code>LDCAP-Caption/ |-- .streamlit/ |-- checkpoints/ |-- models/ |-- venv/ |-- __pycache__/ |-- app.py |-- ascap_decoder.py |-- ascap_encoder.py |-- config.py |-- film_sifting.py |-- inference.py |-- inference (1).py |-- README.md |-- requirements.txt |-- rin.py |-- test_dataset.py |-- test_image.jpg |-- train_scst.py |-- train_xe.py |-- vocab.json </code></pre> <p>Key components:</p> <ul> <li> <p><code>app.py</code>: Streamlit interface for caption generation</p> </li> <li> <p><code>inference.py</code>: Main inference pipeline</p> </li> <li> <p><code>train_xe.py</code>: Stage-1 training</p> </li> <li> <p><code>train_scst.py</code>: Stage-2 training</p> </li> <li> <p><code>film_sifting.py</code>: FiLM-based attention module</p> </li> <li> <p><code>rin.py</code>: RIN module</p> </li> <li> <p><code>config.py</code>: Configuration and paths</p> </li> <li> <p><code>test_dataset.py</code>: Dataset validation</p> </li> <li> <p><code>vocab.json</code>: Vocabulary file</p> </li> </ul> <h2>Software Requirements</h2> <ul> <li> <p>Python 3.10</p> </li> <li> <p>pip 24.x</p> </li> <li> <p>GPU recommended for training</p> </li> <li> <p>CPU supported for inference</p> </li> </ul> <p>Dependencies:</p> <pre><code>streamlit==1.35.0 torch==2.6.0 torchvision==0.21.0 Pillow==10.4.0 numpy==1.26.4 tqdm==4.67.1 </code></pre> <p>Additional:</p> <pre><code>pycocoevalcap </code></pre> <h2>Installation</h2> <h3>Local</h3> <pre><code class="language-bash">python -m venv venv source venv/bin/activate # or use Windows activation pip install --upgrade pip pip install -r requirements.txt pip install pycocoevalcap </code></pre> <h3>Kaggle</h3> <ol> <li> <p>Attach required datasets</p> </li> <li> <p>Place repository in <code>/kaggle/working/</code></p> </li> <li> <p>Verify paths in configuration</p> </li> <li> <p>Run validation and training scripts</p> </li> </ol> <h2>Reproducibility Scope</h2> <p>This release includes:</p> <ul> <li> <p>Inference and Streamlit interface</p> </li> <li> <p>Configuration aligned with Kaggle-based training</p> </li> <li> <p>Dataset validation script</p> </li> <li> <p>XE and SCST training scripts</p> </li> <li> <p>ASCAP-related modules</p> </li> <li> <p>Checkpoint and vocabulary (external download)</p> </li> </ul> <h2>Workflow</h2> <h3>Dataset Validation</h3> <pre><code class="language-bash">python test_dataset.py </code></pre> <h3>XE Training</h3> <pre><code class="language-bash">python train_xe.py </code></pre> <p>Output:</p> <pre><code>checkpoints/xe_best_model.pt </code></pre> <h3>SCST Training</h3> <pre><code class="language-bash">python train_scst.py </code></pre> <p>Outputs:</p> <pre><code>checkpoints/scst_best_model.pt checkpoints/scst_checkpoint.pt </code></pre> <h2>Inference</h2> <p>Download required files and place:</p> <pre><code>checkpoints/scst_best_model.pt vocab.json </code></pre> <p>Run:</p> <pre><code class="language-bash">streamlit run app.py </code></pre> <p>Access:</p> <pre><code>http://localhost:8501 </code></pre> <h2>Limitations</h2> <ul> <li> <p>Path-dependent training workflow</p> </li> <li> <p>Kaggle-specific paths in some scripts</p> </li> <li> <p><code>pycocoevalcap</code> not included in requirements</p> </li> <li> <p>No dedicated evaluation script</p> </li> <li> <p>No environment lock file</p> </li> </ul> <h2>Reuse</h2> <p>This release provides the software, configuration, and workflow as included in this repository snapshot for reproducibility and reuse.</p> </div> </div> |
| title | Robust Image Captioning via Latent Compression and Dynamic Decoder Conditioning |
| url | https://doi.org/10.5281/zenodo.19842490 |