Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Recurso digital |
| Language: | |
| Published: |
Zenodo
2024
|
| Subjects: | |
| Online Access: | https://doi.org/10.5281/zenodo.11080876 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
Table of Contents:
- <p>This repository contains the trained neural network (TensorFlow) models, associated with the paper "<em>Lightweight Multitask Learning for Robust JND Prediction using Latent Space and Reconstructed Frames</em>", published in IEEE T-CSVT 2024. The code associated with these models can be accessed through the <a href="https://github.com/sanaznami/MTL_JND">paper's GitHub page</a>.</p> <div> <h2>Paper</h2> </div> <p>The paper is available on <a href="https://ieeexplore.ieee.org/document/10500870">IEEEXplore </a>and a preprint is available on <a href="https://www.techrxiv.org/doi/full/10.36227/techrxiv.170421899.96243855/v1">TechArxiv</a>.</p> <div> <h2>Requirements</h2> </div> <ul> <li>Tensorflow</li> <li>FFmpeg</li> </ul> <div> <h2>Dataset</h2> </div> <p>Our evaluation is conducted on <a href="https://ieee-dataport.org/documents/videoset" rel="nofollow">VideoSet</a> and <a href="https://mcl.usc.edu/mcl-jci-dataset/" rel="nofollow">MCL-JCI</a> datasets.</p> <div> <h2>Usage</h2> </div> <p>Our pretrained models are capable of predicting JND values, and they can also be employed for training on a custom dataset.</p> <div> <h5>Note: The dataset used for training and testing should have such a structure.</h5> <pre><code>- rootdir/ - train/ - img#1 - ... - JND-Levels.txt (a file containing the 3 JND levels per image: first column for the first JND, second column for the second JND, and third column for the third JND level) - valid/ - img#1 - ... - JND-Levels.txt (a file containing the 3 JND levels per image: first column for the first JND, second column for the second JND, and third column for the third JND level) - test/ - img#1 - ... - jnd1train/ - img#1 - ... - jnd1valid/ - img#1 - ... - jnd2train/ - img#1 - ... - jnd2valid/ - img#1 - ... - jnd3train/ - img#1 - ... - jnd3valid/ - img#1 - ...</code></pre> <div> <h3>Testing</h3> </div> <p>For prediction with LAT or REC model, the following commands can be used.</p> <div> <pre><code>python3 [LAT.py or REC.py] test --jnd_value [JND1 or JND2 or JND3] --data_dir "Path-to-the-rootdir/" --model_weights_path "Path-to-the-pretrained-model/" --result_path "Path-to-save-test-results/" --JND_Recon_Models_Path "Path-to-the-pretrained-JND-Reconstruction-models/"</code></pre> <p>For prediction with E2E-LAT or E2E-REC model, the following commands can be used.</p> <pre><code>python3 [E2ELAT.py or E2EREC.py] test --jnd_value [JND1 or JND2 or JND3] --data_dir "Path-to-the-rootdir/" --model_weights_path "Path-to-the-pretrained-model/" --result_path "Path-to-save-test-results/" --ImgReconstrution_Model_Path "Path-to-the-pretrained-Img-Reconstruction-models/"</code></pre> <p> For prediction with MJ-LAT or MJ-REC model, the following commands can be used.</p> <pre><code>python3 [MJLAT.py or MJREC.py] test --data_dir "Path-to-the-rootdir/" --model_weights_path "Path-to-the-pretrained-model/" --result_path "Path-to-save-test-results/" --JND_Recon_Models_Path "Path-to-the-pretrained-JND-Reconstruction-models/"</code></pre> <p> </p> <p>More details about the associated codes can be found on the github page: <a href="https://github.com/sanaznami/MTL_JND">https://github.com/sanaznami/MTL_JND</a></p> </div> </div>