Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2512.18318 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914212398235648 |
|---|---|
| author | Caglar, Eren Oskooei, Amirkia Rafiei Kutanoglu, Mehmet Keles, Mustafa Aktas, Mehmet S. |
| author_facet | Caglar, Eren Oskooei, Amirkia Rafiei Kutanoglu, Mehmet Keles, Mustafa Aktas, Mehmet S. |
| contents | This paper introduces a parallel and asynchronous Transformer framework designed for efficient and accurate multilingual lip synchronization in real-time video conferencing systems. The proposed architecture integrates translation, speech processing, and lip-synchronization modules within a pipeline-parallel design that enables concurrent module execution through message-queue-based decoupling, reducing end-to-end latency by up to 3.1 times compared to sequential approaches. To enhance computational efficiency and throughput, the inference workflow of each module is optimized through low-level graph compilation, mixed-precision quantization, and hardware-accelerated kernel fusion. These optimizations provide substantial gains in efficiency while preserving model accuracy and visual quality. In addition, a context-adaptive silence-detection component segments the input speech stream at semantically coherent boundaries, improving translation consistency and temporal alignment across languages. Experimental results demonstrate that the proposed parallel architecture outperforms conventional sequential pipelines in processing speed, synchronization stability, and resource utilization. The modular, message-oriented design makes this work applicable to resource-constrained IoT communication scenarios including telemedicine, multilingual kiosks, and remote assistance systems. Overall, this work advances the development of low-latency, resource-efficient multimodal communication frameworks for next-generation AIoT systems. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_18318 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Asynchronous Pipeline Parallelism for Real-Time Multilingual Lip Synchronization in Video Communication Systems Caglar, Eren Oskooei, Amirkia Rafiei Kutanoglu, Mehmet Keles, Mustafa Aktas, Mehmet S. Multimedia Artificial Intelligence Computer Vision and Pattern Recognition Distributed, Parallel, and Cluster Computing Networking and Internet Architecture I.2.7; H.4.3; C.2.4 This paper introduces a parallel and asynchronous Transformer framework designed for efficient and accurate multilingual lip synchronization in real-time video conferencing systems. The proposed architecture integrates translation, speech processing, and lip-synchronization modules within a pipeline-parallel design that enables concurrent module execution through message-queue-based decoupling, reducing end-to-end latency by up to 3.1 times compared to sequential approaches. To enhance computational efficiency and throughput, the inference workflow of each module is optimized through low-level graph compilation, mixed-precision quantization, and hardware-accelerated kernel fusion. These optimizations provide substantial gains in efficiency while preserving model accuracy and visual quality. In addition, a context-adaptive silence-detection component segments the input speech stream at semantically coherent boundaries, improving translation consistency and temporal alignment across languages. Experimental results demonstrate that the proposed parallel architecture outperforms conventional sequential pipelines in processing speed, synchronization stability, and resource utilization. The modular, message-oriented design makes this work applicable to resource-constrained IoT communication scenarios including telemedicine, multilingual kiosks, and remote assistance systems. Overall, this work advances the development of low-latency, resource-efficient multimodal communication frameworks for next-generation AIoT systems. |
| title | Asynchronous Pipeline Parallelism for Real-Time Multilingual Lip Synchronization in Video Communication Systems |
| topic | Multimedia Artificial Intelligence Computer Vision and Pattern Recognition Distributed, Parallel, and Cluster Computing Networking and Internet Architecture I.2.7; H.4.3; C.2.4 |
| url | https://arxiv.org/abs/2512.18318 |