Imagen de Portada

Instability in Downstream Task Performance During LLM Pretraining

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Nishida, Yuto, Isonuma, Masaru, Oda, Yusuke
Formato:	Preprint
Publicado:	2025
Materias:	Computation and Language
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Unlearning Traces the Influential Training Data of Language Models
por: Isonuma, Masaru, et al.
Publicado: (2024)

What's New in My Data? Novelty Exploration via Contrastive Generation
por: Isonuma, Masaru, et al.
Publicado: (2024)

Investigating Training and Generalization in Faithful Self-Explanations of Large Language Models
por: Doi, Tomoki, et al.
Publicado: (2025)

Comprehensive Evaluation of Large Language Models for Topic Modeling
por: Doi, Tomoki, et al.
Publicado: (2024)

Exclusive Unlearning
por: Sasaki, Mutsumi, et al.
Publicado: (2026)

Massive Supervised Fine-tuning Experiments Reveal How Data, Layer, and Training Factors Shape LLM Alignment Quality
por: Harada, Yuto, et al.
Publicado: (2025)

Do LLMs Need to Think in One Language? Correlation between Latent Language and Task Performance
por: Ozaki, Shintaro, et al.
Publicado: (2025)

UniDetox: Universal Detoxification of Large Language Models via Dataset Distillation
por: Lu, Huimin, et al.
Publicado: (2025)

Towards Transfer Unlearning: Empirical Evidence of Cross-Domain Bias Mitigation
por: Lu, Huimin, et al.
Publicado: (2024)

How a Bilingual LM Becomes Bilingual: Tracing Internal Representations with Sparse Autoencoders
por: Inaba, Tatsuro, et al.
Publicado: (2025)

Scaling Laws for Downstream Task Performance of Large Language Models
por: Isik, Berivan, et al.
Publicado: (2024)

How to Make the Most of LLMs' Grammatical Knowledge for Acceptability Judgments
por: Ide, Yusuke, et al.
Publicado: (2024)

Exploring the Impact of a Transformer's Latent Space Geometry on Downstream Task Performance
por: Marbut, Anna C., et al.
Publicado: (2024)

Contrastive Learning for Task-Independent SpeechLLM-Pretraining
por: Züfle, Maike, et al.
Publicado: (2024)

Task-Informed Anti-Curriculum by Masking Improves Downstream Performance on Text
por: Jarca, Andrei, et al.
Publicado: (2025)

llm-jp-modernbert: A ModernBERT Model Trained on a Large-Scale Japanese Corpus with Long Context Length
por: Sugiura, Issa, et al.
Publicado: (2025)

Generating Diverse Translation with Perturbed kNN-MT
por: Nishida, Yuto, et al.
Publicado: (2024)

Understanding and Mitigating Bias Inheritance in LLM-based Data Augmentation on Downstream Tasks
por: Li, Miaomiao, et al.
Publicado: (2025)

Measuring the Effect of Transcription Noise on Downstream Language Understanding Tasks
por: Shapira, Ori, et al.
Publicado: (2025)

An Evaluation of Sindhi Word Embedding in Semantic Analogies and Downstream Tasks
por: Ali, Wazir, et al.
Publicado: (2024)

Edit Distances and Their Applications to Downstream Tasks in Research and Commercial Contexts
por: Carmo, Félix do, et al.
Publicado: (2024)

Which Programming Language and What Features at Pre-training Stage Affect Downstream Logical Inference Performance?
por: Uchiyama, Fumiya, et al.
Publicado: (2024)

Applying Intrinsic Debiasing on Downstream Tasks: Challenges and Considerations for Machine Translation
por: Iluz, Bar, et al.
Publicado: (2024)

Forecasting Downstream Performance of LLMs With Proxy Metrics
por: Patel, Arkil, et al.
Publicado: (2026)

FedEval-LLM: Federated Evaluation of Large Language Models on Downstream Tasks with Collective Wisdom
por: He, Yuanqin, et al.
Publicado: (2024)

Scaling Laws Are Unreliable for Downstream Tasks: A Reality Check
por: Lourie, Nicholas, et al.
Publicado: (2025)

Llama-Mimi: Exploring the Limits of Flattened Speech Language Modeling
por: Sugiura, Issa, et al.
Publicado: (2025)

Vaporetto: Efficient Japanese Tokenization Based on Improved Pointwise Linear Classification
por: Akabe, Koichi, et al.
Publicado: (2024)

Quantifying the Importance of Data Alignment in Downstream Model Performance
por: Chawla, Krrish, et al.
Publicado: (2025)

Long-Tail Crisis in Nearest Neighbor Language Models
por: Nishida, Yuto, et al.
Publicado: (2025)

Can Character-based Language Models Improve Downstream Task Performance in Low-Resource and Noisy Language Scenarios?
por: Riabi, Arij, et al.
Publicado: (2021)

How Does Code Pretraining Affect Language Model Task Performance?
por: Petty, Jackson, et al.
Publicado: (2024)

Adapting Decoder-Based Language Models for Diverse Encoder Downstream Tasks
por: Suganthan, Paul, et al.
Publicado: (2025)

Optimising Language Models for Downstream Tasks: A Post-Training Perspective
por: Shi, Zhengyan
Publicado: (2025)

Personality as a Probe for LLM Evaluation: Method Trade-offs and Downstream Effects
por: Handa, Gunmay, et al.
Publicado: (2025)

Capability Salience Vector: Fine-grained Alignment of Loss and Capabilities for Downstream Task Scaling Law
por: Ge, Qiming, et al.
Publicado: (2025)

Improving the Downstream Performance of Mixture-of-Experts Transformers via Weak Vanilla Transformers
por: Lu, Xin, et al.
Publicado: (2024)

CoLA: Cross-Modal Low-rank Adaptation for Multimodal Downstream Tasks
por: Suharitdamrong, Wish, et al.
Publicado: (2026)

Scaling Laws for Predicting Downstream Performance in LLMs
por: Chen, Yangyi, et al.
Publicado: (2024)

Protoknowledge Shapes Behaviour of LLMs in Downstream Tasks: Memorization and Generalization with Knowledge Graphs
por: Ranaldi, Federico, et al.
Publicado: (2025)