Cover Image

Everything is a Video: Unifying Modalities through Next-Frame Prediction

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Hudson, G. Thomas, Slack, Dean, Winterbottom, Thomas, Sterling, Jamie, Xiao, Chenghao, Shentu, Junjie, Moubayed, Noura Al
Format:	Preprint
Published:	2024
Subjects:	Computer Vision and Pattern Recognition Computation and Language Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

The Power of Next-Frame Prediction for Learning Physical Laws
by: Winterbottom, Thomas, et al.
Published: (2024)

Video Prediction of Dynamic Physical Simulations With Pixel-Space Spatiotemporal Transformers
by: Slack, Dean L, et al.
Published: (2025)

RAR-b: Reasoning as Retrieval Benchmark
by: Xiao, Chenghao, et al.
Published: (2024)

Early Detection and Reduction of Memorisation for Domain Adaptation and Instruction Tuning
by: Slack, Dean L., et al.
Published: (2025)

MuLD: The Multitask Long Document Benchmark
by: Hudson, G Thomas, et al.
Published: (2022)

AttenCraft: Attention-guided Disentanglement of Multiple Concepts for Text-to-Image Customization
by: Shentu, Junjie, et al.
Published: (2024)

Textual Localization: Decomposing Multi-concept Images for Subject-Driven Text-to-Image Generation
by: Shentu, Junjie, et al.
Published: (2024)

Adversarial Defence without Adversarial Defence: Enhancing Language Model Robustness via Instance-level Principal Component Removal
by: Wang, Yang, et al.
Published: (2025)

Pixel Sentence Representation Learning
by: Xiao, Chenghao, et al.
Published: (2024)

Analyzing LLMs' Knowledge Boundary Cognition Across Languages Through the Lens of Internal Representations
by: Xiao, Chenghao, et al.
Published: (2025)

Audio Contrastive-based Fine-tuning: Decoupling Representation Learning and Classification
by: Wang, Yang, et al.
Published: (2023)

MIEB: Massive Image Embedding Benchmark
by: Xiao, Chenghao, et al.
Published: (2025)

X-ray Made Simple: Lay Radiology Report Generation and Robust Evaluation
by: Zhao, Kun, et al.
Published: (2024)

Controllable Image Generation with Composed Parallel Token Prediction
by: Stirling, Jamie, et al.
Published: (2024)

Investigating Permutation-Invariant Discrete Representation Learning for Spatially Aligned Images
by: Stirling, Jamie S. J., et al.
Published: (2026)

SciMMIR: Benchmarking Scientific Multi-modal Information Retrieval
by: Wu, Siwei, et al.
Published: (2024)

Achieving Operational Universality through a Turing Complete Chemputer
by: Gahler, Daniel, et al.
Published: (2025)

Multi-Modal Framing Analysis of News
by: Arora, Arnav, et al.
Published: (2025)

On Support Samples of Next Word Prediction
by: Li, Yuqian, et al.
Published: (2025)

ENTP: Encoder-only Next Token Prediction
by: Ewer, Ethan, et al.
Published: (2024)

Detecting misinformation through Framing Theory: the Frame Element-based Model
by: Wang, Guan, et al.
Published: (2024)

Fostering Video Reasoning via Next-Event Prediction
by: Wang, Haonan, et al.
Published: (2025)

BOW: Reinforcement Learning for Bottlenecked Next Word Prediction
by: Shen, Ming, et al.
Published: (2025)

LongCat-Next: Lexicalizing Modalities as Discrete Tokens
by: Meituan LongCat Team, et al.
Published: (2026)

Frame-Voyager: Learning to Query Frames for Video Large Language Models
by: Yu, Sicheng, et al.
Published: (2024)

Enhancing Bangla Language Next Word Prediction and Sentence Completion through Extended RNN with Bi-LSTM Model On N-gram Language
by: Islam, Md Robiul, et al.
Published: (2024)

Enhancing Multimodal Sentiment Analysis for Missing Modality through Self-Distillation and Unified Modality Cross-Attention
by: Weng, Yuzhe, et al.
Published: (2024)

Framing Matters: Addressing Framing Sensitivity in Decision-Making through Behaviorally-Grounded Value Alignment
by: Hwang, Seojin, et al.
Published: (2026)

Preference learning made easy: Everything should be understood through win rate
by: Zhang, Lily H., et al.
Published: (2025)

Training LLMs Beyond Next Token Prediction -- Filling the Mutual Information Gap
by: Yang, Chun-Hao, et al.
Published: (2025)

VISTA: Visualized Text Embedding For Universal Multi-Modal Retrieval
by: Zhou, Junjie, et al.
Published: (2024)

Disentangling Racial Phenotypes: Fine-Grained Control of Race-related Facial Phenotype Characteristics
by: Yucer, Seyma, et al.
Published: (2024)

Timing is Everything: Temporal Scaffolding of Semantic Surprise in Humor
by: Ma, Yuxi, et al.
Published: (2026)

Reinforced Fast Weights with Next-Sequence Prediction
by: Hwang, Hee Seung, et al.
Published: (2026)

On the Rigour of Scientific Writing: Criteria, Analysis, and Insights
by: James, Joseph, et al.
Published: (2024)

Why Attend to Everything? Focus is the Key
by: Yao, Hengshuai, et al.
Published: (2026)

Track Anything Behind Everything: Zero-Shot Amodal Video Object Segmentation
by: Hudson, Finlay G. C., et al.
Published: (2024)

Multi-SpatialMLLM: Multi-Frame Spatial Understanding with Multi-Modal Large Language Models
by: Xu, Runsen, et al.
Published: (2025)

Semantic Frame Aggregation-based Transformer for Live Video Comment Generation
by: Fatima, Anam, et al.
Published: (2025)

LogitSpec: Accelerating Retrieval-based Speculative Decoding via Next Next Token Speculation
by: Liu, Tianyu, et al.
Published: (2025)