Imagen de Portada

Look Ahead or Look Around? A Theoretical Comparison Between Autoregressive and Masked Pretraining

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Zhang, Qi, Du, Tianqi, Huang, Haotian, Wang, Yifei, Wang, Yisen
Formato:	Preprint
Publicado:	2024
Materias:	Machine Learning Computation and Language
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Long-Short Alignment for Effective Long-Context Modeling in LLMs
por: Du, Tianqi, et al.
Publicado: (2025)

A Theoretical Understanding of Self-Correction through In-context Alignment
por: Wang, Yifei, et al.
Publicado: (2024)

When More is Less: Understanding Chain-of-Thought Length in LLMs
por: Wu, Yuyang, et al.
Publicado: (2025)

Advancing LLM Safe Alignment with Safety Representation Ranking
por: Du, Tianqi, et al.
Publicado: (2025)

On the Role of Discrete Tokenization in Visual Representation Learning
por: Du, Tianqi, et al.
Publicado: (2024)

Look-Ahead-Bench: a Standardized Benchmark of Look-ahead Bias in Point-in-Time LLMs for Finance
por: Benhenda, Mostapha
Publicado: (2026)

LookAhead Tuning: Safer Language Models via Partial Answer Previews
por: Liu, Kangwei, et al.
Publicado: (2025)

Local Look-Ahead Guidance via Verifier-in-the-Loop for Automated Theorem Proving
por: Rajaee, Sara, et al.
Publicado: (2025)

A Closer Look into Mixture-of-Experts in Large Language Models
por: Lo, Ka Man, et al.
Publicado: (2024)

Look Within or Look Beyond? A Theoretical Comparison Between Parameter-Efficient and Full Fine-Tuning
por: Liu, Yongkang, et al.
Publicado: (2025)

A Closer Look at Machine Unlearning for Large Language Models
por: Yuan, Xiaojian, et al.
Publicado: (2024)

Look-Ahead Reasoning on Learning Platforms
por: Zhu, Haiqing, et al.
Publicado: (2025)

Autoregressive Models Rival Diffusion Models at ANY-ORDER Generation
por: Du, Tianqi, et al.
Publicado: (2026)

Learning to Read Where to Look: Disease-Aware Vision-Language Pretraining for 3D CT
por: Ging, Simon, et al.
Publicado: (2026)

ProxSparse: Regularized Learning of Semi-Structured Sparsity Masks for Pretrained LLMs
por: Liu, Hongyi, et al.
Publicado: (2025)

Psychological Profiling in Cybersecurity: A Look at LLMs and Psycholinguistic Features
por: Tshimula, Jean Marie, et al.
Publicado: (2024)

On the Limits of Sparse Autoencoders: A Theoretical Framework and Reweighted Remedy
por: Cui, Jingyi, et al.
Publicado: (2025)

Memorization: A Close Look at Books
por: Ma, Iris, et al.
Publicado: (2025)

A Critical Look At Tokenwise Reward-Guided Text Generation
por: Rashid, Ahmad, et al.
Publicado: (2024)

Looking beyond the next token
por: Thankaraj, Abitha, et al.
Publicado: (2025)

A Fast and Effective Solution to the Problem of Look-ahead Bias in LLMs
por: Merchant, Humzah, et al.
Publicado: (2025)

Mask-Enhanced Autoregressive Prediction: Pay Less Attention to Learn More
por: Zhuang, Xialie, et al.
Publicado: (2025)

What is Wrong with Perplexity for Long-context Language Modeling?
por: Fang, Lizhe, et al.
Publicado: (2024)

Looking for the Inner Music: Probing LLMs' Understanding of Literary Style
por: Hicke, Rebecca M. M., et al.
Publicado: (2025)

Look-Ahead Screening Rules for the Lasso
por: Larsson, Johan
Publicado: (2021)

A Sober Look at Progress in Language Model Reasoning: Pitfalls and Paths to Reproducibility
por: Hochlehnert, Andreas, et al.
Publicado: (2025)

Looking Beyond The Top-1: Transformers Determine Top Tokens In Order
por: Lioubashevski, Daria, et al.
Publicado: (2024)

ReLook: Vision-Grounded RL with a Multimodal LLM Critic for Agentic Web Coding
por: Li, Yuhang, et al.
Publicado: (2025)

Base Models Look Human To AI Detectors
por: Xu, Yixuan Even, et al.
Publicado: (2026)

Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrations
por: Wei, Zeming, et al.
Publicado: (2023)

Enabling Autoregressive Models to Fill In Masked Tokens
por: Israel, Daniel, et al.
Publicado: (2025)

MAGE: All-[MASK] Block Already Knows Where to Look in Diffusion LLM
por: Kwon, Omin, et al.
Publicado: (2026)

Look Ahead Text Understanding and LLM Stitching
por: Jiang, Junlin Julian, et al.
Publicado: (2024)

A Closer Look at Classification Evaluation Metrics and a Critical Reflection of Common Evaluation Practice
por: Opitz, Juri
Publicado: (2024)

A Second Look on BASS -- Boosting Abstractive Summarization with Unified Semantic Graphs -- A Replication Study
por: Koraş, Osman Alperen, et al.
Publicado: (2024)

Look Before Leap: Look-Ahead Planning with Uncertainty in Reinforcement Learning
por: Liu, Yongshuai, et al.
Publicado: (2025)

Closer Look at Efficient Inference Methods: A Survey of Speculative Decoding
por: Ryu, Hyun, et al.
Publicado: (2024)

MaskTab: Scalable Masked Tabular Pretraining with Scaling Laws and Distillation for Industrial Classification
por: Zheng, Bo, et al.
Publicado: (2026)

On the Hardness of Reinforcement Learning with Transition Look-Ahead
por: Pla, Corentin, et al.
Publicado: (2025)

Advancing Sequential Numerical Prediction in Autoregressive Models
por: Fei, Xiang, et al.
Publicado: (2025)