Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2510.14631 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914149459558400 |
|---|---|
| author | Santos, Uélison Jean Lopes dos Ferri, Alessandro Nistor, Szilard Tommasini, Riccardo Binnig, Carsten Luthra, Manisha |
| author_facet | Santos, Uélison Jean Lopes dos Ferri, Alessandro Nistor, Szilard Tommasini, Riccardo Binnig, Carsten Luthra, Manisha |
| contents | In this paper, we present a vision for a new generation of multimodal streaming systems that embed MLLMs as first-class operators, enabling real-time query processing across multiple modalities. Achieving this is non-trivial: while recent work has integrated MLLMs into databases for multimodal queries, streaming systems require fundamentally different approaches due to their strict latency and throughput requirements. Our approach proposes novel optimizations at all levels, including logical, physical, and semantic query transformations that reduce model load to improve throughput while preserving accuracy. We demonstrate this with Samsara, a prototype leveraging such optimizations to improve performance by more than an order of magnitude. Moreover, we discuss a research roadmap that outlines open research challenges for building a scalable and efficient multimodal stream processing systems. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_14631 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Towards a Multimodal Stream Processing System Santos, Uélison Jean Lopes dos Ferri, Alessandro Nistor, Szilard Tommasini, Riccardo Binnig, Carsten Luthra, Manisha Databases In this paper, we present a vision for a new generation of multimodal streaming systems that embed MLLMs as first-class operators, enabling real-time query processing across multiple modalities. Achieving this is non-trivial: while recent work has integrated MLLMs into databases for multimodal queries, streaming systems require fundamentally different approaches due to their strict latency and throughput requirements. Our approach proposes novel optimizations at all levels, including logical, physical, and semantic query transformations that reduce model load to improve throughput while preserving accuracy. We demonstrate this with Samsara, a prototype leveraging such optimizations to improve performance by more than an order of magnitude. Moreover, we discuss a research roadmap that outlines open research challenges for building a scalable and efficient multimodal stream processing systems. |
| title | Towards a Multimodal Stream Processing System |
| topic | Databases |
| url | https://arxiv.org/abs/2510.14631 |