Cover Image

Vectorized Sequence-Based Chunking for Data Deduplication

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Udayashankar, Sreeharsha, Al-Kiswany, Samer
Format:	Preprint
Published:	2025
Subjects:	Distributed, Parallel, and Cluster Computing
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Accelerating Data Chunking in Deduplication Systems using Vector Instructions
by: Udayashankar, Sreeharsha, et al.
Published: (2025)

A Thorough Investigation of Content-Defined Chunking Algorithms for Data Deduplication
by: Gregoriadis, Marcel, et al.
Published: (2024)

Message-Oriented Middleware Systems: Technology Overview
by: Al-Manasrah, Wael, et al.
Published: (2026)

ZipLLM: Efficient LLM Storage via Model-Aware Synergistic Data Deduplication and Compression
by: Wang, Zirui, et al.
Published: (2025)

FASTEN: Towards a FAult-tolerant and STorage EfficieNt Cloud: Balancing Between Replication and Deduplication
by: Ahmed, Sabbir, et al.
Published: (2023)

Deduplicator: When Computation Reuse Meets Load Balancing at the Network Edge
by: Azad, Md Washik Al, et al.
Published: (2024)

AutoChunk: Automated Activation Chunk for Memory-Efficient Long Sequence Inference
by: Zhao, Xuanlei, et al.
Published: (2024)

Efficient Long Context Fine-tuning with Chunk Flow
by: Yuan, Xiulong, et al.
Published: (2025)

RadixMLP -- Intra-batch Deduplication for Causal Transformers
by: Feil, Michael, et al.
Published: (2026)

Privacy-preserving Chunk Scheduling in a BitTorrent Implementation of Federated Learning
by: Li, Naicheng, et al.
Published: (2026)

Syncopate: Efficient Multi-GPU AI Kernels via Automatic Chunk-Centric Compute-Communication Overlap
by: Qiang, Xinwei, et al.
Published: (2026)

Minimizing Communication for Parallel Symmetric Tensor Times Same Vector Computation
by: Daas, Hussam Al, et al.
Published: (2025)

HierMoE: Accelerating MoE Training with Hierarchical Token Deduplication and Expert Swap
by: Lin, Wenxiang, et al.
Published: (2025)

ChunkFlow: Communication-Aware Chunked Prefetching for Layerwise Offloading in Distributed Diffusion Transformer Inference
by: Meng, Han, et al.
Published: (2026)

A Decentralized Sequencer and Data Availability Committee for Rollups Using Set Consensus
by: Capretto, Margarita, et al.
Published: (2025)

Reconstruction-Based Adaptive Scheduling Using AI Inferences in Safety-Critical Systems
by: Alshaer, Samer, et al.
Published: (2025)

NAVIS: Concurrent Search and Update with Low Position-Seeking Overhead in On-SSD Graph-Based Vector Search
by: Song, Jaeyong, et al.
Published: (2026)

A Chunked-Object Pattern for Multi-Region Large Payload Storage in Managed NoSQL Databases
by: Chinthareddy, Manideep Reddy
Published: (2025)

Zarr-Based Chunk-Level Cumulative Sums in Reduced Dimensions
by: Zhang, Hailiang, et al.
Published: (2025)

WindVE: Collaborative CPU-NPU Vector Embedding
by: Huang, Jinqi, et al.
Published: (2025)

Towards Efficient and Scalable Distributed Vector Search with RDMA
by: Zhi, Xiangyu, et al.
Published: (2025)

PilotANN: Memory-Bounded GPU Acceleration for Vector Search
by: Gui, Yuntao, et al.
Published: (2025)

Efficient Data Labeling and Optimal Device Scheduling in HWNs Using Clustered Federated Semi-Supervised Learning
by: Hamood, Moqbel, et al.
Published: (2024)

Scalable Distributed Vector Search via Accuracy Preserving Index Construction
by: Xu, Yuming, et al.
Published: (2025)

Closer in the Gap: Towards Portable Performance on RISC-V Vector Processors
by: Shi, Ruimin, et al.
Published: (2026)

Leveraging Hardware Performance Counters for Predicting Workload Interference in Vector Supercomputers
by: Shubham, et al.
Published: (2024)

Untied Ulysses: Memory-Efficient Context Parallelism via Headwise Chunking
by: Ghadia, Ravi, et al.
Published: (2026)

Fantasy: Efficient Large-scale Vector Search on GPU Clusters with GPUDirect Async
by: Liu, Yi, et al.
Published: (2025)

An AD based library for Efficient Hessian and Hessian-Vector Product Computation on GPU
by: Ranjan, Desh, et al.
Published: (2024)

Fine-Grained Vectorized Merge Sorting on RISC-V: From Register to Cache
by: Zhang, Jin, et al.
Published: (2024)

High-performance Vector-length Agnostic Quantum Circuit Simulations on ARM Processors
by: Shi, Ruimin, et al.
Published: (2026)

An Auto-tuning Method for Run-time Data Transformation for Sparse Matrix-Vector Multiplication
by: Katagiri, Takahiro, et al.
Published: (2024)

Optimization Opportunities for Cloud-Based Data Pipeline Infrastructures
by: Jablonski, Johannes, et al.
Published: (2026)

VQ-LLM: High-performance Code Generation for Vector Quantization Augmented LLM Inference
by: Liu, Zihan, et al.
Published: (2025)

Chameleon: Taming Dynamic Operator Sequences for Memory-Intensive LLM Training
by: Wang, Zibo, et al.
Published: (2025)

Parallel DNA Sequence Alignment on High-Performance Systems with CUDA and MPI
by: Zwaka, Linus
Published: (2024)

Preparing for HPC on RISC-V: Examining Vectorization and Distributed Performance of an Astrophyiscs Application with HPX and Kokkos
by: Diehl, Patrick, et al.
Published: (2024)

Optimizing Long-context LLM Serving via Fine-grained Sequence Parallelism
by: Li, Cong, et al.
Published: (2025)

LoongTrain: Efficient Training of Long-Sequence LLMs with Head-Context Parallelism
by: Gu, Diandian, et al.
Published: (2024)

TrioSeq: A Novel Approach to Accelerate Triplet Sequence Alignment on GPUs
by: Graça, Miguel, et al.
Published: (2026)