Copertina

FRAME: Boosting LLMs with A Four-Quadrant Multi-Stage Pretraining Strategy

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Zhang, Xuemiao, Duan, Feiyu, Xu, Liangyu, Zhou, Yongwei, Wang, Sirui, Weng, Rongxiang, Wang, Jingang, Cai, Xunliang
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Computation and Language
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Preference Curriculum: LLMs Should Always Be Pretrained on Their Preferred Data
di: Zhang, Xuemiao, et al.
Pubblicazione: (2025)

FIRE: Flexible Integration of Data Quality Ratings for Effective Pre-Training
di: Xu, Liangyu, et al.
Pubblicazione: (2025)

Enhancing LLMs via High-Knowledge Data Selection
di: Duan, Feiyu, et al.
Pubblicazione: (2025)

LinkQA: Synthesizing Diverse QA from Multiple Seeds Strongly Linked by Knowledge Points
di: Zhang, Xuemiao, et al.
Pubblicazione: (2025)

Large-Scale Diverse Synthesis for Mid-Training
di: Zhang, Xuemiao, et al.
Pubblicazione: (2025)

Expanding Reasoning Potential in Foundation Model by Learning Diverse Chains of Thought Patterns
di: Zhang, Xuemiao, et al.
Pubblicazione: (2025)

LLMs Know What They Need: Leveraging a Missing Information Guided Framework to Empower Retrieval-Augmented Generation
di: Wang, Keheng, et al.
Pubblicazione: (2024)

A Survey on LLM Mid-Training
di: Tu, Chengying, et al.
Pubblicazione: (2025)

Unlocking Implicit Experience: Synthesizing Tool-Use Trajectories from Text
di: Xu, Zhihao, et al.
Pubblicazione: (2026)

Multi-Objective and Mixed-Reward Reinforcement Learning via Reward-Decorrelated Policy Optimization
di: Bai, Yang, et al.
Pubblicazione: (2026)

Libra: Assessing and Improving Reward Model by Learning to Think
di: Zhou, Meng, et al.
Pubblicazione: (2025)

Length Desensitization in Direct Preference Optimization
di: Liu, Wei, et al.
Pubblicazione: (2024)

Speculative Decoding via Early-exiting for Faster LLM Inference with Thompson Sampling Control Mechanism
di: Liu, Jiahao, et al.
Pubblicazione: (2024)

Ltri-LLM: Streaming Long Context Inference for LLMs with Training-Free Dynamic Triangular Attention Pattern
di: Tang, Hongyin, et al.
Pubblicazione: (2024)

LANG: Reinforcement Learning for Multilingual Reasoning with Language-Adaptive Hint Guidance
di: Fan, Yuchun, et al.
Pubblicazione: (2026)

Beyond the Known: Investigating LLMs Performance on Out-of-Domain Intent Detection
di: Wang, Pei, et al.
Pubblicazione: (2024)

Four Quadrants of Difficulty: A Simple Categorisation and its Limits
di: Toborek, Vanessa, et al.
Pubblicazione: (2026)

Dynamic Fisher-weighted Model Merging via Bayesian Optimization
di: Lee, Sanwoo, et al.
Pubblicazione: (2025)

Multi-Programming Language Sandbox for LLMs
di: Dou, Shihan, et al.
Pubblicazione: (2024)

Not All Contexts Are Equal: Teaching LLMs Credibility-aware Generation
di: Pan, Ruotong, et al.
Pubblicazione: (2024)

ReMamba: Equip Mamba with Effective Long-Sequence Modeling
di: Yuan, Danlong, et al.
Pubblicazione: (2024)

Investigating and Scaling up Code-Switching for Multilingual Language Model Pre-Training
di: Wang, Zhijun, et al.
Pubblicazione: (2025)

SEAS: Self-Evolving Adversarial Safety Optimization for Large Language Models
di: Diao, Muxi, et al.
Pubblicazione: (2024)

Too Consistent to Detect: A Study of Self-Consistent Errors in LLMs
di: Tan, Hexiang, et al.
Pubblicazione: (2025)

DolphCoder: Echo-Locating Code Large Language Models with Diverse and Multi-Objective Instruction Tuning
di: Wang, Yejie, et al.
Pubblicazione: (2024)

Earlier Tokens Contribute More: Learning Direct Preference Optimization From Temporal Decay Perspective
di: Shao, Ruichen, et al.
Pubblicazione: (2025)

NeedleInATable: Exploring Long-Context Capability of Large Language Models towards Long-Structured Tables
di: Wang, Lanrui, et al.
Pubblicazione: (2025)

APP: Adaptive Prototypical Pseudo-Labeling for Few-shot OOD Detection
di: Wang, Pei, et al.
Pubblicazione: (2023)

Parallel Decoding via Hidden Transfer for Lossless Large Language Model Acceleration
di: Wu, Pengfei, et al.
Pubblicazione: (2024)

FIRP: Faster LLM inference via future intermediate representation prediction
di: Wu, Pengfei, et al.
Pubblicazione: (2024)

Graph-Structured Speculative Decoding
di: Gong, Zhuocheng, et al.
Pubblicazione: (2024)

PositionID: LLMs can Control Lengths, Copy and Paste with Explicit Positional Awareness
di: Wang, Zekun, et al.
Pubblicazione: (2024)

A Preliminary Study on the Promises and Challenges of Native Top-$k$ Sparse Attention
di: Xiu, Di, et al.
Pubblicazione: (2025)

How Do Your Code LLMs Perform? Empowering Code Instruction Tuning with High-Quality Data
di: Wang, Yejie, et al.
Pubblicazione: (2024)

SampleMix: A Sample-wise Pre-training Data Mixing Strategey by Coordinating Data Quality and Diversity
di: Xi, Xiangyu, et al.
Pubblicazione: (2025)

AgentRefine: Enhancing Agent Generalization through Refinement Tuning
di: Fu, Dayuan, et al.
Pubblicazione: (2025)

Predictor-Corrector Enhanced Transformers with Exponential Moving Average Coefficient Learning
di: Li, Bei, et al.
Pubblicazione: (2024)

Sub-Scaling Laws: On the Role of Data Density and Training Strategies in LLMs
di: Chen, Zhengyu, et al.
Pubblicazione: (2025)

Prefix Teach, Suffix Fade: Local Teachability Collapse in Strong-to-Weak On-Policy Distillation
di: Liu, Kaiyuan, et al.
Pubblicazione: (2026)

Causal Autoregressive Diffusion Language Model
di: Ruan, Junhao, et al.
Pubblicazione: (2026)