Cover Image

From Bytes to Ideas: Language Modeling with Autoregressive U-Nets

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Videau, Mathurin, Idrissi, Badr Youbi, Leite, Alessandro, Schoenauer, Marc, Teytaud, Olivier, Lopez-Paz, David
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Evolutionary Pre-Prompt Optimization for Mathematical Reasoning
by: Videau, Mathurin, et al.
Published: (2024)

Tuning without Peeking: Provable Generalization Bounds and Robust LLM Post-Training
by: Labiad, Ismail, et al.
Published: (2025)

Mixture of Experts in Image Classification: What's the Sweet Spot?
by: Videau, Mathurin, et al.
Published: (2024)

Evolutionary Retrofitting
by: Videau, Mathurin, et al.
Published: (2024)

Temperature Matters: Enhancing Watermark Robustness Against Paraphrasing Attacks
by: Idrissi, Badr Youbi, et al.
Published: (2025)

Better & Faster Large Language Models via Multi-token Prediction
by: Gloeckle, Fabian, et al.
Published: (2024)

Beyond Multi-Token Prediction: Pretraining LLMs with Future Summaries
by: Mahajan, Divyat, et al.
Published: (2025)

Hierarchical Autoregressive Transformers: Combining Byte- and Word-Level Processing for Robust, Adaptable Language Models
by: Neitemeier, Pit, et al.
Published: (2025)

Rosetta Stone at KSAA-RD Shared Task: A Hop From Language Modeling To Word--Definition Alignment
by: ElBakry, Ahmed, et al.
Published: (2023)

GPT-SW3: An Autoregressive Language Model for the Nordic Languages
by: Ekgren, Ariel, et al.
Published: (2023)

Evaluating Contrast Localizer for Identifying Causal Units in Social & Mathematical Tasks in Language Models
by: Jamaa, Yassine, et al.
Published: (2025)

Efficient-DLM: From Autoregressive to Diffusion Language Models, and Beyond in Speed
by: Fu, Yonggan, et al.
Published: (2025)

Continuous Autoregressive Language Models
by: Shao, Chenze, et al.
Published: (2025)

Differences in Text Generated by Diffusion and Autoregressive Language Models
by: Zhang, Zeyang, et al.
Published: (2026)

RecycleGPT: An Autoregressive Language Model with Recyclable Module
by: Jiang, Yufan, et al.
Published: (2023)

From Bytes to Borsch: Fine-Tuning Gemma and Mistral for the Ukrainian Language Representation
by: Kiulian, Artur, et al.
Published: (2024)

Algorithm of Thoughts: Enhancing Exploration of Ideas in Large Language Models
by: Sel, Bilgehan, et al.
Published: (2023)

Dominance Based Crossover Operator for Evolutionary Multi-objective Algorithms
by: Roudenko, Olga, et al.
Published: (2005)

CausalProfiler: Generating Synthetic Benchmarks for Rigorous and Transparent Evaluation of Causal Machine Learning
by: Panayiotou, Panayiotis, et al.
Published: (2025)

SpaceByte: Towards Deleting Tokenization from Large Language Modeling
by: Slagle, Kevin
Published: (2024)

Detecting Referring Expressions in Visually Grounded Dialogue with Autoregressive Language Models
by: Willemsen, Bram, et al.
Published: (2025)

Simulated Annealing Enhances Theory-of-Mind Reasoning in Autoregressive Language Models
by: Hu, Xucong, et al.
Published: (2026)

Context Dependence and Reliability in Autoregressive Language Models
by: Sengupta, Poushali, et al.
Published: (2026)

MrT5: Dynamic Token Merging for Efficient Byte-level Language Models
by: Kallini, Julie, et al.
Published: (2024)

MYTE: Morphology-Driven Byte Encoding for Better and Fairer Multilingual Language Modeling
by: Limisiewicz, Tomasz, et al.
Published: (2024)

Position: Causal Machine Learning Requires Rigorous Synthetic Experiments for Broader Adoption
by: Poinsot, Audrey, et al.
Published: (2025)

Why Diffusion Language Models Struggle with Truly Parallel (Non-Autoregressive) Decoding?
by: Li, Pengxiang, et al.
Published: (2026)

IdeaBench: Benchmarking Large Language Models for Research Idea Generation
by: Guo, Sikun, et al.
Published: (2024)

Multiscale Byte Language Models -- A Hierarchical Architecture for Causal Million-Length Sequence Modeling
by: Egli, Eric, et al.
Published: (2025)

Projected Autoregression: Autoregressive Language Generation in Continuous State Space
by: Naparstek, Oshri
Published: (2026)

Expert Threshold Routing for Autoregressive Language Modeling with Dynamic Computation Allocation and Load Balancing
by: Sun, Hanchi, et al.
Published: (2026)

ByteScience: Bridging Unstructured Scientific Literature and Structured Data with Auto Fine-tuned Large Language Model in Token Granularity
by: Xie, Tong, et al.
Published: (2024)

From Bytes to Biases: Investigating the Cultural Self-Perception of Large Language Models
by: Messner, Wolfgang, et al.
Published: (2023)

Enhancing Robustness of Autoregressive Language Models against Orthographic Attacks via Pixel-based Approach
by: Yang, Han, et al.
Published: (2025)

MixCE: Training Autoregressive Language Models by Mixing Forward and Reverse Cross-Entropies
by: Zhang, Shiyue, et al.
Published: (2023)

ALPINE: Unveiling the Planning Capability of Autoregressive Learning in Language Models
by: Wang, Siwei, et al.
Published: (2024)

Rational Metareasoning for Large Language Models
by: De Sabbata, C. Nicolò, et al.
Published: (2024)

AI Idea Bench 2025: AI Research Idea Generation Benchmark
by: Qiu, Yansheng, et al.
Published: (2025)

Autoregressive + Chain of Thought = Recurrent: Recurrence's Role in Language Models' Computability and a Revisit of Recurrent Transformer
by: Zhang, Xiang, et al.
Published: (2024)

Early Decisions Matter: Proximity Bias and Initial Trajectory Shaping in Non-Autoregressive Diffusion Language Models
by: Kim, Jiyeon, et al.
Published: (2026)