Cover Image

A Thorough Investigation of Content-Defined Chunking Algorithms for Data Deduplication

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Gregoriadis, Marcel, Balduf, Leonhard, Scheuermann, Björn, Pouwelse, Johan
Format:	Preprint
Published:	2024
Subjects:	Distributed, Parallel, and Cluster Computing
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

SwarmSearch: Decentralized Search Engine with Self-Funding Economy
by: Gregoriadis, Marcel, et al.
Published: (2025)

A Large-Scale Web Search Dataset for Federated Online Learning to Rank
by: Gregoriadis, Marcel, et al.
Published: (2025)

Vectorized Sequence-Based Chunking for Data Deduplication
by: Udayashankar, Sreeharsha, et al.
Published: (2025)

Accelerating Data Chunking in Deduplication Systems using Vector Instructions
by: Udayashankar, Sreeharsha, et al.
Published: (2025)

MeritRank: Sybil Tolerant Reputation for Merit-based Tokenomics
by: Nasrulin, Bulat, et al.
Published: (2022)

De-DSI: Decentralised Differentiable Search Index
by: Neague, Petru, et al.
Published: (2024)

SmartphoneDemocracy: Privacy-Preserving E-Voting on Decentralized Infrastructure using Novel European Identity
by: Jóźwik, Michał, et al.
Published: (2025)

Failures of public key infrastructure: 53 year survey
by: Dumitrescu, Adrian-Tudor, et al.
Published: (2024)

Low-level I/O Monitoring for Scientific Workflows
by: Witzke, Joel, et al.
Published: (2024)

KS+: Predicting Workflow Task Memory Usage Over Time
by: Bader, Jonathan, et al.
Published: (2024)

Practical Federated Learning without a Server
by: Dhasade, Akash, et al.
Published: (2025)

Decentralized Learning Made Practical with Client Sampling
by: de Vos, Martijn, et al.
Published: (2023)

ZipLLM: Efficient LLM Storage via Model-Aware Synergistic Data Deduplication and Compression
by: Wang, Zirui, et al.
Published: (2025)

FASTEN: Towards a FAult-tolerant and STorage EfficieNt Cloud: Balancing Between Replication and Deduplication
by: Ahmed, Sabbir, et al.
Published: (2023)

Efficient Long Context Fine-tuning with Chunk Flow
by: Yuan, Xiulong, et al.
Published: (2025)

RadixMLP -- Intra-batch Deduplication for Causal Transformers
by: Feil, Michael, et al.
Published: (2026)

Privacy-preserving Chunk Scheduling in a BitTorrent Implementation of Federated Learning
by: Li, Naicheng, et al.
Published: (2026)

ASA -- The Adaptive Scheduling Algorithm
by: Souza, Abel, et al.
Published: (2024)

StarveSpam: Mitigating Spam with Local Reputation in Permissionless Blockchains
by: Chotkan, Rowdy, et al.
Published: (2025)

Syncopate: Efficient Multi-GPU AI Kernels via Automatic Chunk-Centric Compute-Communication Overlap
by: Qiang, Xinwei, et al.
Published: (2026)

Beyond 2-Edge-Connectivity: Algorithms and Impossibility for Content-Oblivious Leader Election
by: Chang, Yi-Jun, et al.
Published: (2025)

ColonyOS -- A Meta-Operating System for Distributed Computing Across Heterogeneous Platform
by: Kristiansson, Johan
Published: (2024)

HierMoE: Accelerating MoE Training with Hierarchical Token Deduplication and Expert Swap
by: Lin, Wenxiang, et al.
Published: (2025)

ChunkFlow: Communication-Aware Chunked Prefetching for Layerwise Offloading in Distributed Diffusion Transformer Inference
by: Meng, Han, et al.
Published: (2026)

Deduplicator: When Computation Reuse Meets Load Balancing at the Network Edge
by: Azad, Md Washik Al, et al.
Published: (2024)

Distributed Ranges: A Model for Distributed Data Structures, Algorithms, and Views
by: Brock, Benjamin, et al.
Published: (2024)

Advancing Blockchain Scalability: A Linear Optimization Framework for Diversified Node Allocation in Shards
by: Assmann, Björn, et al.
Published: (2024)

CoGenT: A Content-oriented Generative-hit Framework for Content Delivery Networks
by: Wang, Peng, et al.
Published: (2024)

A Chunked-Object Pattern for Multi-Region Large Payload Storage in Managed NoSQL Databases
by: Chinthareddy, Manideep Reddy
Published: (2025)

A Survey on Adversarial Contention Resolution
by: Banicescu, Ioana, et al.
Published: (2024)

Energy Efficiency Support for Software Defined Networks: a Serverless Computing Approach
by: Banaie, Fatemeh, et al.
Published: (2024)

Optimizing the Longhorn Cloud-native Software Defined Storage Engine for High Performance
by: Kampadais, Konstantinos, et al.
Published: (2025)

A HPC Co-Scheduler with Reinforcement Learning
by: Souza, Abel, et al.
Published: (2024)

AutoChunk: Automated Activation Chunk for Memory-Efficient Long Sequence Inference
by: Zhao, Xuanlei, et al.
Published: (2024)

A Contention-Free Model for Converged Kubernetes on HPC
by: Sochat, Vanessa, et al.
Published: (2024)

Analysis of Blockchain Assisted Energy Sharing Algorithms with Realistic Data Across Microgrids
by: Zekiye, Abdulrezzak, et al.
Published: (2024)

Softening the Impact of Collisions in Contention Resolution
by: Biswas, Umesh, et al.
Published: (2024)

Semantica: Decentralized Search using a LLM-Guided Semantic Tree Overlay
by: Neague, Petru, et al.
Published: (2025)

CB-SpMV:A Data Aggregating and Balance Algorithm for Cache-Friendly Block-Based SpMV on GPUs
by: Cong, Xing, et al.
Published: (2026)

bigMICE: Multiple Imputation of Big Data
by: Morvan, Hugo, et al.
Published: (2026)