Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Recurso digital |
| Language: | |
| Published: |
Zenodo
2025
|
| Online Access: | https://doi.org/10.5281/zenodo.17210810 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
Table of Contents:
- <div>The aim of this repository is to centralize the work performed by the authors to write the paper: "<strong>iSubscription: Bridging the Gap Between Contracts and Runtime Access Control in SaaS</strong>", submitted to the International Conference on Service Oriented Computing (ICSOC) 2025.</div> <div> </div> <div>The laboratory package is divided in three main sections: "tools repositories", "usage limits analysis", and "load testing comparative". The first one contains the links to the repositories of all the tools developed to perform the contribution. These include: SPACE, space-node-client, space-react-client and TomatoMeter. Further details will be provided regarding the content that can be found in each one. The second corresponds to the material required to replicate the analysis on the evolution of pricings' usage limits. The third includes both the material and the explanations to replicate the load tests presented in the main paper.</div> <div> </div> <div> <h1>Tools' Repositories</h1> <p>This section provides the repositories where the tools presented in the paper are allocated:</p> <ul> <li>SPACE: <a href="https://github.com/Alex-GF/space">https://github.com/Alex-GF/space</a> – This is the decoupled, pricing-driven feature toggling solution presented in the paper.</li> <li>space-react-client: <a href="https://github.com/Alex-GF/space-react-client">https://github.com/Alex-GF/space-react-client</a> – A react client that can be installed in the managed service to control the rendering of UI elements. It offers a more familiar interface with SPACE and frees developers from implementing ad-hoc modules for communication.</li> <li>space-node-client: <a href="https://github.com/Alex-GF/space-node-client">https://github.com/Alex-GF/space-node-client</a> – An express client that can be installed in the managed service to control the access to feature on the server side. It offers a more familiar interface with SPACE and frees developers from implementing ad-hoc modules for communication.</li> <li>TomatoMeter: <a href="https://github.com/Alex-GF/TomatoMeter">https://github.com/Alex-GF/TomatoMeter</a> – This is the demo app where SPACE can be tested and showcased. In addition, this was the application used to perform the load testing comparative between Pricing4SaaS and SPACE. Because of this, two branches are relevant in this repository: the one integrating SPACE (main); and the one integrating Pricing4SaaS (demo-pricing4saas).</li> </ul> <h1>Usage Limits' Evolution Analysis</h1> <p>This section details the structure and dataset employed to perform the observational study on the evolution of pricings' usage limits. In addition, instructions to replicate the experiment are provided.</p> <h2>Files</h2> <p>All the files required to replicate this experiment can be found within the "SaaS-analysis-main.zip" file provided with this laboratory package.</p> </div> <h2>Available SaaS</h2> <div>The SaaS of the following table are the ones that have been used in the paper. The following symbols are used to indicate the status of the SaaS for the indicated year:</div> <p> </p> <div>- ✅: The SaaS is modeled for the indicated year.</div> <div>- ✖️: There is a snapshot, but a clear feature list cannot be extracted.</div> <div>- ❌: There isn't a snapshot for the SaaS in the indicated year.</div> <p> </p> <div>| SaaS | 2019 | 2020 | 2021 | 2022 | 2023 | 2024 | 2025 |</div> <div>| ------------- | :--: | :--: | :--: | :--: | :--: | :--: | :--: |</div> <div>| Box | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |</div> <div>| Buffer | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |</div> <div>| Canva | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |</div> <div>| CircleCI | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |</div> <div>| ClickUp | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |</div> <div>| Clockify | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |</div> <div>| Crowdcast | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |</div> <div>| Databox | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |</div> <div>| Deskera | ❌ | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ |</div> <div>| Dropbox | ✖️ | ✖️ | ✅ | ✅ | ✅ | ✅ | ✅ |</div> <div>| Evernote | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |</div> <div>| Figma | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |</div> <div>| GitHub | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |</div> <div>| Hypercontext | ❌ | ❌ | ✅ | ✅ | ✅ | ✅ | ❌ |</div> <div>| Jira | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |</div> <div>| MailChimp | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |</div> <div>| Microsoft 365 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |</div> <div>| Notion | ✖️ | ✖️ | ✅ | ✅ | ✅ | ✅ | ✅ |</div> <div>| Okta | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |</div> <div>| OpenPhone | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |</div> <div>| Overleaf | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |</div> <div>| Planable | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |</div> <div>| Postman | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |</div> <div>| Pumble | ❌ | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ |</div> <div>| Quip | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |</div> <div>| Salesforce | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |</div> <div>| Shopify | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |</div> <div>| Slack | ✅ | ✅ | ❌ | ❌ | ✅ | ✅ | ✅ |</div> <div>| Tableau | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |</div> <div>| Trello | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |</div> <div>| Trustmary | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |</div> <div>| UserGuiding | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |</div> <div>| Webflow | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |</div> <div>| Wrike | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |</div> <div>| Zenhub | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |</div> <div>| Zoom | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |</div> <div>| Zapier | ✅ | ✅ | ❌ | ✅ | ✅ | ✅ | ✅ |</div> <div> </div> <div><strong>TOTAL:</strong> 240 pricings - 37 SaaS</div> <p> </p> <h2><strong>Structure</strong></h2> <div>The repository contains three main folders:</div> <div>- `data`</div> <div>- `figures`</div> <div>- `src`</div> <div> </div> <div>In addition to these, there are some files:</div> <div> </div> <div>- `.gitignore`: Contains the files that are ignored by git.</div> <div>- `analysis_and_figures.ipynb`: It is the notebook that contains all the data analysis performed for the paper and generates the figures included in it.</div> <div>- `Dockerfile`: Since having installed MiniZinc is mandatory to run some part of the experiment, a Dockerfile is provided to create a Docker image with MiniZinc installed.</div> <div>- `LICENSE`: Contains the license of the repository.</div> <div>- `package.json` and `package-lock.json`: Contain the dependencies needed to run some scripts that are called from the notebook.</div> <div>- `README.md`: Contains the information of the repository.</div> <div>- `requirements.txt`: Contains the dependencies needed to the `analysis_and_figures.ipynb` notebook.</div> <div>- `tsconfig.json`: Contains the configuration of the TypeScript compiler for development tasks.</div> <p> </p> <h3><strong>Data</strong></h3> <div>The `data` folder initially contains a single subfolder: `pricings`. This subfolder stores the dataset of pricing models used in the study, along with metadata and analytics about them (the latter is generated by the notebook `analysis_and_figures`.ipynb).</div> <div> </div> <div>During the execution of the labpack, a few additional folders may be created:</div> <div> </div> <div>- `CSVs`: Contains CSV files summarizing the relevant variables from both the 2019–2024 and 2019–2025 datasets, as used throughout the study.</div> <div>- `pricings/dzn`: Includes the MiniZinc data models generated from each pricing. Although not used directly in the experiments, they are stored here for transparency, in case users wish to explore them further.</div> <div>- `pricings/json`: Stores the JSON files summarizing all analytics extracted from each pricing using the [Pricing4TS package](https://github.com/Alex-GF/Pricing4TS).</div> <div> </div> <div> </div> <h3>Figures</h3> <div>This folder contains the figures generated by the notebook `analysis_and_figures.ipynb`.</div> <p> </p> <h3>Src</h3> <div>The `src` folder contains the scripts used to extract the analytics from the pricings dataset. Such scripts are written in TypeScript and leverage the [Pricing4TS package](https://github.com/Alex-GF/Pricing4TS) to extract the analytics.</div> <div> </div> <div> </div> <h2>Usage</h2> <h3>Install Dependencies</h3> <div>Dependencies can be installed using the following command:</div> <div> </div> <div>```bash</div> <div>pip install -r requirements.txt</div> <div>```</div> <div> </div> <div>> [!WARNING]</div> <div>> If you don't want to install **MiniZinc** in your machine, you can use the Docker image provided in the repository. To build the image, run the following command:</div> <div>> ```bash</div> <div>> docker build -t minizinc .</div> <div>> ```</div> <p> </p> <h3>Run the Scripts</h3> <div>> [!WARNING]</div> <div>> The notebook is designed to be run with Python 3.9 or higher.</div> <div> </div> <div>The scripts can be run by following the notebook `analysis_and_figures.ipynb`.</div> <p> </p> <h2>License</h2> <div>This work is licensed under a Creative Commons Attribution 4.0 International License [cc-by].</div> <div> </div> <div>[cc-by]: http://creativecommons.org/licenses/by/4.0/</div> <div> </div> <div> <h1>Load Testing Comparative</h1> <p>This section details the instructions required to replicated the load testing comparative between SPACE and Pricing4SaaS presented in the main paper.</p> <h2>Files</h2> <p>All the files required to replicate this experiment can be found within the "LoadTesting-SPACE.zip" and "LoadTesting-Pricing4SaaS.zip" files provided with this laboratory package. </p> <h2>SPACE Load Testing</h2> <p>To replicate the results of the load testing performed on the integration of SPACE with TomatoMeter, download the "LoadTesting-SPACE.zip" and follow these steps:</p> <h3>0. Prerequisites</h3> <ul> <li>Docker</li> <li>To have access to a kubernetes cluster wherein SPACE will be deployed</li> </ul> <h3>1. Deploy SPACE using kubernetes</h3> <p>Before running the demo, you need to have a kubernetes deployment of SPACE running. </p> <p>Considering that you already have a cluster (we recommend using kubeadm from docker-desktop if want to run tests locally), you just need to navigate to the `k8s` folder within the repository available in the "LoadTesting-SPACE.zip" file. Considering that you are in the directory where the ZIP has been decompressed:</p> <p>```bash<br>cd space/k8s<br>```</p> <p>Then, you just need to run the deploy.sh script to deploy SPACE in your kubernetes cluster:</p> <p>```bash<br>chmod +x deploy.sh<br>./deploy.sh<br>```</p> <p>Then wait for the SPACE pods to start. If everything is set up correctly, you must have 3 replicas of the `server` pod running, and should be able to access SPACE at <a href="http://localhost">http://localhost</a>.</p> <p>If you want to explore SPACE, you can log in into the app using the following default credentials (this will be needed for retrieving the API key in the next step):</p> <p>- <strong>Username:</strong> admin<br>- <strong>Password:</strong> space4all</p> <p><strong>NOTE:</strong> If you are interested in learning more about SPACE, you can check out the <a href="https://github.com/Alex-GF/space#readme">SPACE documentation</a> for more details on how to set it up, configure it, and use its features.</p> <h3>2. Configure the project for load testing</h3> <p>This demo project rely on <a href="https://www.artillery.io">artillery</a> to develop, run and evaluate load tests. Therefore, in order to ensure a seamless integration between TomatoMeter, SPACE and artillery.io (the service where artillery insights are stored), you must follow these steps:</p> <p>1. Install the dependencies of the project:</p> <p> ```bash<br> pnpm install<br> # or<br> npm install<br> # or<br> yarn install<br> ```</p> <p>2. Connect to <a href="https://www.artillery.io">artillery.io</a> using your github account and generate an API key. Once logged, you can find your API key in `settings -> API Keys`.</p> <p>3. Generate a `.env` file following the structure of `.env.example`.</p> <p> To retrieve the SPACE API key, you must log in into SPACE and navigate to `Access Control`. Once you are there, you can copy the API key of the admin user.</p> <p> Once you have obtained both the SPACE API key and the Artillery.io API key, your `.env` file should look like this:</p> <p> ```bash<br> VITE_SPACE_URL=http://localhost<br> VITE_SPACE_API_KEY=96ff87ecdf9df3971ea98f51cc37edd301a34391fd89a7b96a7436246d8f0ac6<br> ARTILLERY_KEY=a9_koizwzxadtywh677t9hwg9pvogmr600f<br> ```</p> <h3>3. Run the load tests</h3> <p>After configuring the project, you can either run all tests at once, or run them individually.</p> <p>- <strong>To run all tests at once</strong>, just run on the root folder:</p> <p>```bash<br>pnpm run load-test<br>```</p> <p>- <strong>To run individual tests</strong>, you can use the commands with the `load-test` prefix in the `package.json` file. For example, to run the `tomatometer-renew-only` test, you can run:</p> <p>```bash<br>pnpm run load-test:tomatometer-renew-only<br>```</p> <p><strong>IMPORTANT: </strong>With this setup, you will run the load tests against the SPACE instance running in your kubernetes cluster, which by default has 3 space-server replicas configured. If you want to increase/decrease the number of replicas, you can do so by modifying the <em>replicas</em> field in the "k8s/pods/server-deployment.yaml" file (which can be found on the <a href="https://github.com/Alex-GF/space/blob/main/k8s/pods/server-deployment.yaml">SPACE repository</a>) and applying the changes with `<em>kubectl apply -f k8s/pods/server-deployment.yaml</em>`.</p> <h2>Pricing4SaaS Load Testing</h2> <p>Given that Pricing4SaaS is not deployed as an independent service like SPACE, the configuration of the load testing setup is more straightforward. To replicate the results of the load testing performed on the integration of Pricing4SaaS with TomatoMeter, download the "LoadTesting-Pricing4SaaS.zip" and follow these steps:</p> <h3>0. Prerequisites</h3> <ul> <li>Docker</li> </ul> <h3>1. Configure the project for load testing</h3> <p>This demo project rely on <a href="https://www.artillery.io">artillery</a> to develop, run and evaluate load tests. Therefore, in order to ensure a seamless integration between Tomatometer and artillery.io (the service where artillery insights are stored), you must follow these steps:</p> <p>1. Install the dependencies of the project:</p> <p> ```bash<br> pnpm install<br> # or<br> npm install<br> # or<br> yarn install<br> ```</p> <p>2. Connect to <a href="https://www.artillery.io">artillery.io</a> using your github account and generate an API key. Once logged, you can find your API key in `settings -> API Keys`.</p> <p>3. Generate a `.env` file following the structure of `.env.example`. Considering that you already have your artilleri.io API key, your `.env` file should look like this:</p> <p> ```bash<br> ARTILLERY_KEY=a9_koizwzxadtywh677t9hwg9pvogmr600f # The artillery API key<br> ```</p> <h3>2. Run the load tests</h3> <p>After configuring the project, you can either run all tests at once, or run them individually.</p> <p>- <strong>To run all tests at once</strong>, just run on the root folder:</p> <p>```bash<br>pnpm run load-test<br>```</p> <p>- <strong>To run individual tests</strong>, you can use the commands with the `load-test` prefix in the `package.json` file. For example, to run the `renew-only` test, you can run:</p> <p>```bash<br>pnpm run load-test:renew-only<br>```</p> </div>