Cover Image

MixCE: Training Autoregressive Language Models by Mixing Forward and Reverse Cross-Entropies

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zhang, Shiyue, Wu, Shijie, Irsoy, Ozan, Lu, Steven, Bansal, Mohit, Dredze, Mark, Rosenberg, David
Format:	Preprint
Published:	2023
Subjects:	Computation and Language Artificial Intelligence Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

RAG LLMs are Not Safer: A Safety Analysis of Retrieval-Augmented Generation for Large Language Models
by: An, Bang, et al.
Published: (2025)

M3DocRAG: Multi-modal Retrieval is What You Need for Multi-page Multi-document Understanding
by: Cho, Jaemin, et al.
Published: (2024)

Improving Instruct Models for Free: A Study on Partial Adaptation
by: İrsoy, Ozan, et al.
Published: (2025)

Amuro and Char: Analyzing the Relationship between Pre-Training and Fine-Tuning of Large Language Models
by: Sun, Kaiser, et al.
Published: (2024)

Evaluating the Retrieval Robustness of Large Language Models
by: Cao, Shuyang, et al.
Published: (2025)

Entropy-Guided Token Dropout: Training Autoregressive Language Models with Limited Domain Data
by: Wang, Jiapeng, et al.
Published: (2025)

Reversal Invariance in Autoregressive Language Models
by: Sahasrabudhe, Mihir
Published: (2025)

QAPyramid: Fine-grained Evaluation of Content Selection for Text Summarization
by: Zhang, Shiyue, et al.
Published: (2024)

Benchmarking Large Language Models on Answering and Explaining Challenging Medical Questions
by: Chen, Hanjie, et al.
Published: (2024)

Contrastive Region Guidance: Improving Grounding in Vision-Language Models without Training
by: Wan, David, et al.
Published: (2024)

On the Failure of Latent State Persistence in Large Language Models
by: Huang, Jen-tse, et al.
Published: (2025)

Towards Closing the Autoregressive Gap in Language Modeling via Entropy-Gated Continuous Bitstream Diffusion
by: Batzolis, Georgios, et al.
Published: (2026)

Language Models Identify Ambiguities and Exploit Loopholes
by: Choi, Jio, et al.
Published: (2025)

AutoMix: Automatically Mixing Language Models
by: Aggarwal, Pranjal, et al.
Published: (2023)

Can one size fit all?: Measuring Failure in Multi-Document Summarization Domain Transfer
by: DeLucia, Alexandra, et al.
Published: (2025)

Localizing Factual Inconsistencies in Attributable Text Generation
by: Cattan, Arie, et al.
Published: (2024)

LACIE: Listener-Aware Finetuning for Confidence Calibration in Large Language Models
by: Stengel-Eskin, Elias, et al.
Published: (2024)

Exposing and Addressing Cross-Task Inconsistency in Unified Vision-Language Models
by: Maharana, Adyasha, et al.
Published: (2023)

Are Clinical T5 Models Better for Clinical Text?
by: Li, Yahan, et al.
Published: (2024)

Evaluating the Evaluators: Are readability metrics good measures of readability?
by: Cachola, Isabel, et al.
Published: (2025)

MedMax: Mixed-Modal Instruction Tuning for Training Biomedical Assistants
by: Bansal, Hritik, et al.
Published: (2024)

Task Matters: Knowledge Requirements Shape LLM Responses to Context-Memory Conflict
by: Sun, Kaiser, et al.
Published: (2025)

DnDScore: Decontextualization and Decomposition for Factuality Verification in Long-Form Text Generation
by: Wanner, Miriam, et al.
Published: (2024)

The Role of Mixed-Language Documents for Multilingual Large Language Model Pretraining
by: Shao, Jiandong, et al.
Published: (2026)

Code-Mixed Probes Show How Pre-Trained Models Generalise On Code-Switched Text
by: De Leon, Frances A. Laureano, et al.
Published: (2024)

Gender Bias in Decision-Making with Large Language Models: A Study of Relationship Conflicts
by: Levy, Sharon, et al.
Published: (2024)

Merging by Matching Models in Task Parameter Subspaces
by: Tam, Derek, et al.
Published: (2023)

The Unreasonable Effectiveness of Easy Training Data for Hard Tasks
by: Hase, Peter, et al.
Published: (2024)

Evaluating Implicit Biases in LLM Reasoning through Logic Grid Puzzles
by: Jahara, Fatima, et al.
Published: (2025)

Param$Δ$ for Direct Weight Mixing: Post-Train Large Language Model at Zero Cost
by: Cao, Sheng, et al.
Published: (2025)

Soft Self-Consistency Improves Language Model Agents
by: Wang, Han, et al.
Published: (2024)

Rephrase, Augment, Reason: Visual Grounding of Questions for Vision-Language Models
by: Prasad, Archiki, et al.
Published: (2023)

Mix- and MoE-DPO: A Variational Inference Approach to Direct Preference Optimization
by: Bohne, Jason, et al.
Published: (2025)

Delta-CoMe: Training-Free Delta-Compression with Mixed-Precision for Large Language Models
by: Ping, Bowen, et al.
Published: (2024)

REAL Sampling: Boosting Factuality and Diversity of Open-Ended Generation via Asymptotic Entropy
by: Chang, Haw-Shiuan, et al.
Published: (2024)

Probing Multimodal Large Language Models on Cognitive Biases in Chinese Short-Video Misinformation
by: Huang, Jen-tse, et al.
Published: (2026)

Multi-Attribute Steering of Language Models via Targeted Intervention
by: Nguyen, Duy, et al.
Published: (2025)

LLMs are Better Than You Think: Label-Guided In-Context Learning for Named Entity Recognition
by: Bai, Fan, et al.
Published: (2025)

Understanding Fact Recall in Language Models: Why Two-Stage Training Encourages Memorization but Mixed Training Teaches Knowledge
by: Zhang, Ying, et al.
Published: (2025)

CREMA: Generalizable and Efficient Video-Language Reasoning via Multimodal Modular Fusion
by: Yu, Shoubin, et al.
Published: (2024)