Imagen de Portada

Landscape-Aware Growing: The Power of a Little LAG

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Karp, Stefani, Saunshi, Nikunj, Miryoosefi, Sobhan, Reddi, Sashank J., Kumar, Sanjiv
Formato:	Preprint
Publicado:	2024
Materias:	Machine Learning Computation and Language
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

On the Inductive Bias of Stacking Towards Improving Reasoning
por: Saunshi, Nikunj, et al.
Publicado: (2024)

Efficient Stagewise Pretraining via Progressive Subnetworks
por: Panigrahi, Abhishek, et al.
Publicado: (2024)

Reasoning with Latent Thoughts: On the Power of Looped Transformers
por: Saunshi, Nikunj, et al.
Publicado: (2025)

On the Role of Depth and Looping for In-Context Learning with Task Diversity
por: Gatmiry, Khashayar, et al.
Publicado: (2024)

Can Looped Transformers Learn to Implement Multi-step Gradient Descent for In-context Learning?
por: Gatmiry, Khashayar, et al.
Publicado: (2024)

A Little Help Goes a Long Way: Efficient LLM Training by Leveraging Small LMs
por: Rawat, Ankit Singh, et al.
Publicado: (2024)

Structured Preconditioners in Adaptive Optimization: A Unified Analysis
por: Xie, Shuo, et al.
Publicado: (2025)

Learning to Keep a Promise: Scaling Language Model Decoding Parallelism with Learned Asynchronous Decoding
por: Jin, Tian, et al.
Publicado: (2025)

Mimetic Initialization Helps State Space Models Learn to Recall
por: Trockman, Asher, et al.
Publicado: (2024)

HierRouter: Coordinated Routing of Specialized Large Language Models via Reinforcement Learning
por: Gupta, Nikunj, et al.
Publicado: (2025)

Deep sequence models tend to memorize geometrically; it is unclear why
por: Noroozizadeh, Shahriar, et al.
Publicado: (2025)

LoRA-Augmented Generation (LAG) for Knowledge-Intensive Language Tasks
por: Fleshman, William, et al.
Publicado: (2025)

Analyzing Similarity Metrics for Data Selection for Language Model Pretraining
por: Sam, Dylan, et al.
Publicado: (2025)

Are More Tokens Rational? Inference-Time Scaling in Language Models as Adaptive Resource Rationality
por: Hu, Zhimin, et al.
Publicado: (2026)

Perplexity-Aware Data Scaling Law: Perplexity Landscapes Predict Performance for Continual Pre-training
por: Liu, Lei, et al.
Publicado: (2025)

Promises and Pitfalls of Generative Masked Language Modeling: Theoretical Framework and Practical Guidelines
por: Li, Yuchen, et al.
Publicado: (2024)

Hierarchical Retrieval: The Geometry and a Pretrain-Finetune Recipe
por: You, Chong, et al.
Publicado: (2025)

Growing Transformers: Modular Composition and Layer-wise Expansion on a Frozen Substrate
por: Bochkov, A.
Publicado: (2025)

Two-stage LLM Fine-tuning with Less Specialization and More Generalization
por: Wang, Yihan, et al.
Publicado: (2022)

Multi-LLM QA with Embodied Exploration
por: Patel, Bhrij, et al.
Publicado: (2024)

Harnessing the Power of Large Language Model for Uncertainty Aware Graph Processing
por: Qian, Zhenyu, et al.
Publicado: (2024)

Exploring and Improving Drafts in Blockwise Parallel Decoding
por: Kim, Taehyeon, et al.
Publicado: (2024)

SpacTor-T5: Pre-training T5 Models with Span Corruption and Replaced Token Detection
por: Ye, Ke, et al.
Publicado: (2024)

Language Model Cascades: Token-level uncertainty and beyond
por: Gupta, Neha, et al.
Publicado: (2024)

Think before you speak: Training Language Models With Pause Tokens
por: Goyal, Sachin, et al.
Publicado: (2023)

StagFormer: Time Staggering Transformer Decoding for RunningLayers In Parallel
por: Cutler, Dylan, et al.
Publicado: (2025)

Faster Cascades via Speculative Decoding
por: Narasimhan, Harikrishna, et al.
Publicado: (2024)

Simplicity Bias via Global Convergence of Sharpness Minimization
por: Gatmiry, Khashayar, et al.
Publicado: (2024)

Idea2Plan: Exploring AI-Powered Research Planning
por: Huang, Jin, et al.
Publicado: (2025)

A Little Human Data Goes A Long Way
por: Ashok, Dhananjay, et al.
Publicado: (2024)

Grow Up and Merge: Scaling Strategies for Efficient Language Adaptation
por: Glocker, Kevin, et al.
Publicado: (2025)

LittleBit: Ultra Low-Bit Quantization via Latent Factorization
por: Lee, Banseok, et al.
Publicado: (2025)

Universal Model Routing for Efficient LLM Inference
por: Jitkrittum, Wittawat, et al.
Publicado: (2025)

RE-GrievanceAssist: Enhancing Customer Experience through ML-Powered Complaint Management
por: C, Venkatesh, et al.
Publicado: (2024)

From Growing to Looping: A Unified View of Iterative Computation in LLMs
por: Kapl, Ferdinand, et al.
Publicado: (2026)

Multi-turn Training with Basic Human Feedback Helps Little on LLM Reasoning
por: Liu, Qiang, et al.
Publicado: (2025)

LoRA Done RITE: Robust Invariant Transformation Equilibration for LoRA Optimization
por: Yen, Jui-Nan, et al.
Publicado: (2024)

DistillSpec: Improving Speculative Decoding via Knowledge Distillation
por: Zhou, Yongchao, et al.
Publicado: (2023)

Understanding Warmup-Stable-Decay Learning Rates: A River Valley Loss Landscape Perspective
por: Wen, Kaiyue, et al.
Publicado: (2024)

The Growing Pains of Frontier Models: When Leaderboards Stop Separating and What to Measure Next
por: Amin, Adil
Publicado: (2026)