Copertina

Aligning Instruction Tuning with Pre-training

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Liang, Yiming, Zheng, Tianyu, Du, Xinrun, Zhang, Ge, Liu, Jiaheng, Qu, Xingwei, Zu, Wenqiang, Xing, Xingrun, Zheng, Chujie, Ma, Lei, Wang, Guoyin, Zhang, Zhaoxiang, Huang, Wenhao, Yue, Xiang, Zhang, Jiajun
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Artificial Intelligence
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

I-SHEEP: Self-Alignment of LLM from Scratch through an Iterative Self-Enhancement Paradigm
di: Liang, Yiming, et al.
Pubblicazione: (2024)

BiPFT: Binary Pre-trained Foundation Transformer with Low-rank Estimation of Binarization Residual Polynomials
di: Xing, Xingrun, et al.
Pubblicazione: (2023)

Kun: Answer Polishment for Chinese Self-Alignment with Instruction Back-Translation
di: Zheng, Tianyu, et al.
Pubblicazione: (2024)

Reconstructive Visual Instruction Tuning
di: Wang, Haochen, et al.
Pubblicazione: (2024)

KOR-Bench: Benchmarking Language Models on Knowledge-Orthogonal Reasoning Tasks
di: Ma, Kaijing, et al.
Pubblicazione: (2024)

Mitigating Training Imbalance in LLM Fine-Tuning via Selective Parameter Merging
di: Ju, Yiming, et al.
Pubblicazione: (2024)

GIEBench: Towards Holistic Evaluation of Group Identity-based Empathy for Large Language Models
di: Wang, Leyan, et al.
Pubblicazione: (2024)

Chinese Tiny LLM: Pretraining a Chinese-Centric Large Language Model
di: Du, Xinrun, et al.
Pubblicazione: (2024)

SpikeLM: Towards General Spike-Driven Language Modeling via Elastic Bi-Spiking Mechanisms
di: Xing, Xingrun, et al.
Pubblicazione: (2024)

EfficientLLM: Scalable Pruning-Aware Pretraining for Architecture-Agnostic Edge Language Models
di: Xing, Xingrun, et al.
Pubblicazione: (2025)

On Representation Redundancy in Large-Scale Instruction Tuning Data Selection
di: Shu, Youwei, et al.
Pubblicazione: (2026)

MORE-3S:Multimodal-based Offline Reinforcement Learning with Shared Semantic Spaces
di: Zheng, Tianyu, et al.
Pubblicazione: (2024)

SpikeLLM: Scaling up Spiking Neural Network to Large Language Models via Saliency-based Spiking
di: Xing, Xingrun, et al.
Pubblicazione: (2024)

KLoB: a Benchmark for Assessing Knowledge Locating Methods in Language Models
di: Ju, Yiming, et al.
Pubblicazione: (2023)

Pre-trained Models Succeed in Medical Imaging with Representation Similarity Degradation
di: Zu, Wenqiang, et al.
Pubblicazione: (2025)

LogFormer: A Pre-train and Tuning Pipeline for Log Anomaly Detection
di: Guo, Hongcheng, et al.
Pubblicazione: (2024)

KARPA: A Training-free Method of Adapting Knowledge Graph as References for Large Language Model's Reasoning Path Aggregation
di: Fang, Siyuan, et al.
Pubblicazione: (2024)

Aligning the Spectrum: Hybrid Graph Pre-training and Prompt Tuning across Homophily and Heterophily
di: Luo, Haitong, et al.
Pubblicazione: (2025)

LongIns: A Challenging Long-context Instruction-based Exam for LLMs
di: Gavin, Shawn, et al.
Pubblicazione: (2024)

PretrainZero: Reinforcement Active Pretraining
di: Xing, Xingrun, et al.
Pubblicazione: (2025)

COIG-CQIA: Quality is All You Need for Chinese Instruction Fine-tuning
di: Bai, Yuelin, et al.
Pubblicazione: (2024)

Pre-trained Model Guided Fine-Tuning for Zero-Shot Adversarial Robustness
di: Wang, Sibo, et al.
Pubblicazione: (2024)

A Survey on Data Selection for LLM Instruction Tuning
di: Zhang, Bolin, et al.
Pubblicazione: (2024)

Steel-LLM:From Scratch to Open Source -- A Personal Journey in Building a Chinese-Centric LLM
di: Gu, Qingshui, et al.
Pubblicazione: (2025)

D-CPT Law: Domain-specific Continual Pre-Training Scaling Law for Large Language Models
di: Que, Haoran, et al.
Pubblicazione: (2024)

TEGEE: Task dEfinition Guided Expert Ensembling for Generalizable and Few-shot Learning
di: Qu, Xingwei, et al.
Pubblicazione: (2024)

True Knowledge Comes from Practice: Aligning LLMs with Embodied Environments via Reinforcement Learning
di: Tan, Weihao, et al.
Pubblicazione: (2024)

MAmmoTH2: Scaling Instructions from the Web
di: Yue, Xiang, et al.
Pubblicazione: (2024)

Towards Building the Federated GPT: Federated Instruction Tuning
di: Zhang, Jianyi, et al.
Pubblicazione: (2023)

Encyclo-K: Evaluating LLMs with Dynamically Composed Knowledge Statements
di: Liang, Yiming, et al.
Pubblicazione: (2025)

Improving Deep Assertion Generation via Fine-Tuning Retrieval-Augmented Pre-trained Language Models
di: Zhang, Quanjun, et al.
Pubblicazione: (2025)

GPD-1: Generative Pre-training for Driving
di: Xie, Zixun, et al.
Pubblicazione: (2024)

Ross3D: Reconstructive Visual Instruction Tuning with 3D-Awareness
di: Wang, Haochen, et al.
Pubblicazione: (2025)

The Fine Line: Navigating Large Language Model Pretraining with Down-streaming Capability Analysis
di: Yang, Chen, et al.
Pubblicazione: (2024)

Practical Continual Forgetting for Pre-trained Vision Models
di: Zhao, Hongbo, et al.
Pubblicazione: (2025)

Instruction Tuning for Large Language Models: A Survey
di: Zhang, Shengyu, et al.
Pubblicazione: (2023)

Fine-Tuning on Noisy Instructions: Effects on Generalization and Performance
di: Alajrami, Ahmed, et al.
Pubblicazione: (2025)

First Return, Entropy-Eliciting Explore
di: Zheng, Tianyu, et al.
Pubblicazione: (2025)

RECOST: External Knowledge Guided Data-efficient Instruction Tuning
di: Zhang, Qi, et al.
Pubblicazione: (2024)

CMMMU: A Chinese Massive Multi-discipline Multimodal Understanding Benchmark
di: Zhang, Ge, et al.
Pubblicazione: (2024)