Imagen de Portada

Chameleon: A Flexible Data-mixing Framework for Language Model Pretraining and Finetuning

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Xie, Wanyun, Tonin, Francesco, Cevher, Volkan
Formato:	Preprint
Publicado:	2025
Materias:	Machine Learning Computation and Language
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

MaD-Mix: Multi-Modal Data Mixtures via Latent Space Coupling for Vision-Language Model Training
por: Xie, Wanyun, et al.
Publicado: (2026)

ESLM: Risk-Averse Selective Language Modeling for Efficient Pretraining
por: Bal, Melis Ilayda, et al.
Publicado: (2025)

Efficient Large Language Model Inference with Neural Block Linearization
por: Erdogan, Mete, et al.
Publicado: (2025)

Membership Inference Attacks against Large Vision-Language Models
por: Li, Zhan, et al.
Publicado: (2024)

SAMPa: Sharpness-aware Minimization Parallelized
por: Xie, Wanyun, et al.
Publicado: (2024)

Stable Nonconvex-Nonconcave Training via Linear Interpolation
por: Pethick, Thomas, et al.
Publicado: (2023)

Generalizable and Stable Finetuning of Pretrained Language Models on Low-Resource Texts
por: Somayajula, Sai Ashish, et al.
Publicado: (2024)

Optimistic Dual Averaging Unifies Modern Optimizers
por: Pethick, Thomas, et al.
Publicado: (2026)

Revisiting SMoE Language Models by Evaluating Inefficiencies with Task Specific Expert Pruning
por: Sarkar, Soumajyoti, et al.
Publicado: (2024)

Scaling Laws for Forgetting during Finetuning with Pretraining Data Injection
por: Bethune, Louis, et al.
Publicado: (2025)

Revisiting Character-level Adversarial Attacks for Language Models
por: Rocamora, Elias Abad, et al.
Publicado: (2024)

Single-pass Detection of Jailbreaking Input in Large Language Models
por: Candogan, Leyla Naz, et al.
Publicado: (2025)

Certified Robustness Under Bounded Levenshtein Distance
por: Rocamora, Elias Abad, et al.
Publicado: (2025)

Improving SAM Requires Rethinking its Optimization Formulation
por: Xie, Wanyun, et al.
Publicado: (2024)

Towards Active Synthetic Data Generation for Finetuning Language Models
por: Kessler, Samuel, et al.
Publicado: (2025)

Extractive Structures Learned in Pretraining Enable Generalization on Finetuned Facts
por: Feng, Jiahai, et al.
Publicado: (2024)

Accelerating Spectral Clustering under Fairness Constraints
por: Tonin, Francesco, et al.
Publicado: (2025)

Leveraging the Context through Multi-Round Interactions for Jailbreaking Attacks
por: Cheng, Yixin, et al.
Publicado: (2024)

When Scaling Meets LLM Finetuning: The Effect of Data, Model and Finetuning Method
por: Zhang, Biao, et al.
Publicado: (2024)

SemEval-2025 Task 4: Unlearning sensitive content from Large Language Models
por: Ramakrishna, Anil, et al.
Publicado: (2025)

Selective Rotary Position Embedding
por: Movahedi, Sajad, et al.
Publicado: (2025)

Hierarchical Retrieval: The Geometry and a Pretrain-Finetune Recipe
por: You, Chong, et al.
Publicado: (2025)

Ensembling Finetuned Language Models for Text Classification
por: Arango, Sebastian Pineda, et al.
Publicado: (2024)

Transfer Learning for Finetuning Large Language Models
por: Strangmann, Tobias, et al.
Publicado: (2024)

GRASP: Deterministic argument ranking in interaction graphs
por: Misra, Diganta, et al.
Publicado: (2026)

ATLAS: Adaptive Transfer Scaling Laws for Multilingual Pretraining, Finetuning, and Decoding the Curse of Multilinguality
por: Longpre, Shayne, et al.
Publicado: (2025)

Unlearning as multi-task optimization: A normalized gradient difference approach with an adaptive learning rate
por: Bu, Zhiqi, et al.
Publicado: (2024)

Are Large Language Models Chameleons? An Attempt to Simulate Social Surveys
por: Geng, Mingmeng, et al.
Publicado: (2024)

Data Mixing for Large Language Models Pretraining: A Survey and Outlook
por: Chen, Zhuo, et al.
Publicado: (2026)

Transfer Learning of Tabular Data by Finetuning Large Language Models
por: Rabbani, Shourav B., et al.
Publicado: (2025)

Finetuning Language Models to Emit Linguistic Expressions of Uncertainty
por: Chaudhry, Arslan, et al.
Publicado: (2024)

Analyzing Similarity Metrics for Data Selection for Language Model Pretraining
por: Sam, Dylan, et al.
Publicado: (2025)

Procedural Pretraining: Warming Up Language Models with Abstract Data
por: Jiang, Liangze, et al.
Publicado: (2026)

Understanding Contextual Recall in Transformers: How Finetuning Enables In-Context Reasoning over Pretraining Knowledge
por: Vasudeva, Bhavya, et al.
Publicado: (2026)

Efficient and Flexible Topic Modeling using Pretrained Embeddings and Bag of Sentences
por: Schneider, Johannes
Publicado: (2023)

Language Models Improve When Pretraining Data Matches Target Tasks
por: Mizrahi, David, et al.
Publicado: (2025)

Temporal Entailment Pretraining for Clinical Language Models over EHR Data
por: Tanaka, Tatsunori, et al.
Publicado: (2025)

Large Language Models to Diffusion Finetuning
por: Cetin, Edoardo, et al.
Publicado: (2025)

An Experimental Design Framework for Label-Efficient Supervised Finetuning of Large Language Models
por: Bhatt, Gantavya, et al.
Publicado: (2024)

Training Deep Learning Models with Norm-Constrained LMOs
por: Pethick, Thomas, et al.
Publicado: (2025)