Imagen de Portada

Knowledge-Instruct: Effective Continual Pre-training from Limited Data using Instructions

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Ovadia, Oded, Brief, Meni, Lemberg, Rachel, Sheetrit, Eitam
Formato:	Preprint
Publicado:	2025
Materias:	Computation and Language Artificial Intelligence
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Mixing It Up: The Cocktail Effect of Multi-Task Fine-Tuning on LLM Performance -- A Case Study in Finance
por: Brief, Meni, et al.
Publicado: (2024)

SECQUE: A Benchmark for Evaluating Real-World Financial Analysis Capabilities
por: Yoash, Noga Ben, et al.
Publicado: (2025)

Fine-Tuning or Retrieval? Comparing Knowledge Injection in LLMs
por: Ovadia, Oded, et al.
Publicado: (2023)

ReMatch: Retrieval Enhanced Schema Matching with LLMs
por: Sheetrit, Eitam, et al.
Publicado: (2024)

Synthesize-on-Graph: Knowledgeable Synthetic Data Generation for Continue Pre-training of Large Language Models
por: Ma, Shengjie, et al.
Publicado: (2025)

CamemBERT-bio: Leveraging Continual Pre-training for Cost-Effective Models on French Biomedical Data
por: Touchent, Rian, et al.
Publicado: (2023)

IKnow: Instruction-Knowledge-Aware Continual Pretraining for Effective Domain Adaptation
por: Zhang, Tianyi, et al.
Publicado: (2025)

Pre-training Limited Memory Language Models with Internal and External Knowledge
por: Zhao, Linxi, et al.
Publicado: (2025)

Ada-Instruct: Adapting Instruction Generators for Complex Reasoning
por: Cui, Wanyun, et al.
Publicado: (2023)

Topic Over Source: The Key to Effective Data Mixing for Language Models Pre-training
por: Peng, Jiahui, et al.
Publicado: (2025)

LuxInstruct: A Cross-Lingual Instruction Tuning Dataset For Luxembourgish
por: Philippy, Fred, et al.
Publicado: (2025)

Infinity Instruct: Scaling Instruction Selection and Synthesis to Enhance Language Models
por: Li, Jijie, et al.
Publicado: (2025)

LexInstructEval: Lexical Instruction Following Evaluation for Large Language Models
por: Ren, Huimin, et al.
Publicado: (2025)

Scaling Towards the Information Boundary of Instruction Sets: The Infinity Instruct Subject Technical Report
por: Du, Li, et al.
Publicado: (2025)

BioInstruct: Instruction Tuning of Large Language Models for Biomedical Natural Language Processing
por: Tran, Hieu, et al.
Publicado: (2023)

Robo-Instruct: Simulator-Augmented Instruction Alignment For Finetuning Code LLMs
por: Hu, Zichao, et al.
Publicado: (2024)

Probing Language Models for Pre-training Data Detection
por: Liu, Zhenhua, et al.
Publicado: (2024)

Data Mixing Agent: Learning to Re-weight Domains for Continual Pre-training
por: Yang, Kailai, et al.
Publicado: (2025)

Non-instructional Fine-tuning: Enabling Instruction-Following Capabilities in Pre-trained Language Models without Instruction-Following Data
por: Xie, Juncheng, et al.
Publicado: (2024)

VisualWebInstruct: Scaling up Multimodal Instruction Data through Web Search
por: Jia, Yiming, et al.
Publicado: (2025)

OpenMathInstruct-2: Accelerating AI for Math with Massive Open-Source Instruction Data
por: Toshniwal, Shubham, et al.
Publicado: (2024)

DataMan: Data Manager for Pre-training Large Language Models
por: Peng, Ru, et al.
Publicado: (2025)

Timber: Training-free Instruct Model Refining with Base via Effective Rank
por: Wu, Taiqiang, et al.
Publicado: (2025)

Instruct-Imagen: Image Generation with Multi-modal Instruction
por: Hu, Hexiang, et al.
Publicado: (2024)

Learning to Instruct for Visual Instruction Tuning
por: Zhou, Zhihan, et al.
Publicado: (2025)

RAG-Instruct: Boosting LLMs with Diverse Retrieval-Augmented Instructions
por: Liu, Wanlong, et al.
Publicado: (2024)

InstructEdit: Instruction-based Knowledge Editing for Large Language Models
por: Zhang, Ningyu, et al.
Publicado: (2024)

Data Darwinism Part I: Unlocking the Value of Scientific Data for Pre-training
por: Qin, Yiwei, et al.
Publicado: (2026)

Safer-Instruct: Aligning Language Models with Automated Preference Data
por: Shi, Taiwei, et al.
Publicado: (2023)

MELT: Materials-aware Continued Pre-training for Language Model Adaptation to Materials Science
por: Kim, Junho, et al.
Publicado: (2024)

Making Pre-trained Language Models Better Continual Few-Shot Relation Extractors
por: Ma, Shengkun, et al.
Publicado: (2024)

Understanding Data Temporality Impact on Large Language Models Pre-training
por: Pilchen, Hippolyte, et al.
Publicado: (2026)

RegMix: Data Mixture as Regression for Language Model Pre-training
por: Liu, Qian, et al.
Publicado: (2024)

Multi-domain Knowledge Graph Collaborative Pre-training and Prompt Tuning for Diverse Downstream Tasks
por: Zhang, Yichi, et al.
Publicado: (2024)

Pre-training LLMs using human-like development data corpus
por: Bhardwaj, Khushi, et al.
Publicado: (2023)

Domain Adaptation of Llama3-70B-Instruct through Continual Pre-Training and Model Merging: A Comprehensive Evaluation
por: Siriwardhana, Shamane, et al.
Publicado: (2024)

InstructAV: Instruction Fine-tuning Large Language Models for Authorship Verification
por: Hu, Yujia, et al.
Publicado: (2024)

InverseCoder: Self-improving Instruction-Tuned Code LLMs with Inverse-Instruct
por: Wu, Yutong, et al.
Publicado: (2024)

Efficient Pre-training for Localized Instruction Generation of Videos
por: Batra, Anil, et al.
Publicado: (2023)

Efficient Data Learning for Open Information Extraction with Pre-trained Language Models
por: Fan, Zhiyuan, et al.
Publicado: (2023)