Imagen de Portada

A Practice of Post-Training on Llama-3 70B with Optimal Selection of Additional Language Mixture Ratio

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Xi, Ningyuan, Wu, Yetao, Fan, Kun, Chen, Teng, Gu, Qingqing, Ji, Luo
Formato:	Preprint
Publicado:	2024
Materias:	Computation and Language Artificial Intelligence Machine Learning
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

LaMsS: When Large Language Models Meet Self-Skepticism
por: Wu, Yetao, et al.
Publicado: (2024)

MeTHanol: Modularized Thinking Language Models with Intermediate Layer Thinking, Decoding and Bootstrapping Reasoning
por: Xi, Ningyuan, et al.
Publicado: (2024)

Learn-to-learn on Arbitrary Textual Conditioning: A Hypernetwork-Driven Meta-Gated LLM
por: Ji, Luo, et al.
Publicado: (2026)

Multi-Party Supervised Fine-tuning of Language Models for Multi-Party Dialogue Generation
por: Wang, Xiaoyu, et al.
Publicado: (2024)

MGH Radiology Llama: A Llama 3 70B Model for Radiology
por: Shi, Yucheng, et al.
Publicado: (2024)

Dream to Chat: Model-based Reinforcement Learning on Dialogues with User Belief Modeling
por: Zhao, Yue, et al.
Publicado: (2025)

Making Language Model a Hierarchical Classifier
por: Wang, Yihong, et al.
Publicado: (2025)

Large Language Model Can Be a Foundation for Hidden Rationale-Based Retrieval
por: Ji, Luo, et al.
Publicado: (2024)

From Descriptive to Prescriptive: Uncover the Social Value Alignment of LLM-based Agents
por: Qu, Jinxian, et al.
Publicado: (2026)

Applying Refusal-Vector Ablation to Llama 3.1 70B Agents
por: Lermen, Simon, et al.
Publicado: (2024)

Domain Adaptation of Llama3-70B-Instruct through Continual Pre-Training and Model Merging: A Comprehensive Evaluation
por: Siriwardhana, Shamane, et al.
Publicado: (2024)

Steering Llama 2 via Contrastive Activation Addition
por: Panickssery, Nina, et al.
Publicado: (2023)

Llama Scope: Extracting Millions of Features from Llama-3.1-8B with Sparse Autoencoders
por: He, Zhengfu, et al.
Publicado: (2024)

CMR Scaling Law: Predicting Critical Mixture Ratios for Continual Pre-training of Language Models
por: Gu, Jiawei, et al.
Publicado: (2024)

Llama-Mob: Instruction-Tuning Llama-3-8B Excels in City-Scale Mobility Prediction
por: Tang, Peizhi, et al.
Publicado: (2024)

Llama-3-Nanda-10B-Chat: An Open Generative Large Language Model for Hindi
por: Choudhury, Monojit, et al.
Publicado: (2025)

FiSMiness: A Finite State Machine Based Paradigm for Emotional Support Conversations
por: Zhao, Yue, et al.
Publicado: (2025)

Can Post-Training Quantization Benefit from an Additional QLoRA Integration?
por: Zhu, Xiliang, et al.
Publicado: (2025)

BadLlama: cheaply removing safety fine-tuning from Llama 2-Chat 13B
por: Gade, Pranav, et al.
Publicado: (2023)

Efficient Rationale-based Retrieval: On-policy Distillation from Generative Rerankers based on JEPA
por: Chen, Teng, et al.
Publicado: (2026)

ChocoLlama: Lessons Learned From Teaching Llamas Dutch
por: Meeus, Matthieu, et al.
Publicado: (2024)

LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Models
por: Zheng, Yaowei, et al.
Publicado: (2024)

SinLlama -- A Large Language Model for Sinhala
por: Aravinda, H. W. K., et al.
Publicado: (2025)

Convert Language Model into a Value-based Strategic Planner
por: Wang, Xiaoyu, et al.
Publicado: (2025)

Tulu 3: Pushing Frontiers in Open Language Model Post-Training
por: Lambert, Nathan, et al.
Publicado: (2024)

QwT-v2: Practical, Effective and Efficient Post-Training Quantization
por: Tang, Ningyuan, et al.
Publicado: (2025)

Data Selection via Optimal Control for Language Models
por: Gu, Yuxian, et al.
Publicado: (2024)

Arithmetic in the Wild: Llama uses Base-10 Addition to Reason About Cyclic Concepts
por: Feucht, Sheridan, et al.
Publicado: (2026)

PASER: Post-Training Data Selection for Efficient Pruned Large Language Model Recovery
por: He, Bowei, et al.
Publicado: (2025)

Chain-of-Conceptual-Thought Elicits Daily Conversation in Large Language Models
por: Gu, Qingqing, et al.
Publicado: (2025)

Chinese-Vicuna: A Chinese Instruction-following Llama-based Model
por: Fan, Chenghao, et al.
Publicado: (2025)

Addition is All You Need for Energy-efficient Language Models
por: Luo, Hongyin, et al.
Publicado: (2024)

The Llama 3 Herd of Models
por: Grattafiori, Aaron, et al.
Publicado: (2024)

Llama-Mimi: Exploring the Limits of Flattened Speech Language Modeling
por: Sugiura, Issa, et al.
Publicado: (2025)

Llama SLayer 8B: Shallow Layers Hold the Key to Knowledge Injection
por: Chen, Tianxiang, et al.
Publicado: (2024)

SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models
por: Xiao, Guangxuan, et al.
Publicado: (2022)

TinyLlama: An Open-Source Small Language Model
por: Zhang, Peiyuan, et al.
Publicado: (2024)

Sparsity Induction for Accurate Post-Training Pruning of Large Language Models
por: Jiang, Minhao, et al.
Publicado: (2026)

Extending Llama-3's Context Ten-Fold Overnight
por: Zhang, Peitian, et al.
Publicado: (2024)

Llama See, Llama Do: A Mechanistic Perspective on Contextual Entrainment and Distraction in LLMs
por: Niu, Jingcheng, et al.
Publicado: (2025)