Copertina

S0 Tuning: Zero-Overhead Adaptation of Hybrid Recurrent-Attention Models

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autore principale:	Young, Jack
Natura:	Preprint
Pubblicazione:	2026
Soggetti:	Computation and Language Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Learning to Generate Instruction Tuning Datasets for Zero-Shot Task Adaptation
di: Nayak, Nihal V., et al.
Pubblicazione: (2024)

Zero-Overhead Introspection for Adaptive Test-Time Compute
di: Manvi, Rohin, et al.
Pubblicazione: (2025)

Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Models
di: De, Soham, et al.
Pubblicazione: (2024)

LanguaShrink: Reducing Token Overhead with Psycholinguistics
di: Liang, Xuechen, et al.
Pubblicazione: (2024)

Neural Attention Search Linear: Towards Adaptive Token-Level Hybrid Attention Models
di: Deng, Difan, et al.
Pubblicazione: (2026)

QJL: 1-Bit Quantized JL Transform for KV Cache Quantization with Zero Overhead
di: Zandieh, Amir, et al.
Pubblicazione: (2024)

Depth-Recurrent Attention Mixtures: Giving Latent Reasoning the Attention it Deserves
di: Knupp, Jonas, et al.
Pubblicazione: (2026)

Flux Attention: Context-Aware Hybrid Attention for Efficient LLMs Inference
di: Qiu, Quantong, et al.
Pubblicazione: (2026)

QDyLoRA: Quantized Dynamic Low-Rank Adaptation for Efficient Large Language Model Tuning
di: Rajabzadeh, Hossein, et al.
Pubblicazione: (2024)

Dynamic Adaptation of LoRA Fine-Tuning for Efficient and Task-Specific Optimization of Large Language Models
di: Liao, Xiaoxuan, et al.
Pubblicazione: (2025)

Fine-Tuning Without Forgetting In-Context Learning: A Theoretical Analysis of Linear Attention Models
di: Lee, Chungpa, et al.
Pubblicazione: (2026)

GLiREL -- Generalist Model for Zero-Shot Relation Extraction
di: Boylan, Jack, et al.
Pubblicazione: (2025)

Native Hybrid Attention for Efficient Sequence Modeling
di: Du, Jusen, et al.
Pubblicazione: (2025)

LoRA-SP: Streamlined Partial Parameter Adaptation for Resource-Efficient Fine-Tuning of Large Language Models
di: Wu, Yichao, et al.
Pubblicazione: (2024)

Alleviating Forgetfulness of Linear Attention by Hybrid Sparse Attention and Contextualized Learnable Token Eviction
di: He, Mutian, et al.
Pubblicazione: (2025)

Optimizing Fine-Tuning through Advanced Initialization Strategies for Low-Rank Adaptation
di: Xue, Yongfu
Pubblicazione: (2025)

DropLoRA: Sparse Low-Rank Adaptation for Parameter-Efficient Fine-Tuning
di: Zhang, Haojie
Pubblicazione: (2025)

TRIM: Token-wise Attention-Derived Saliency for Data-Efficient Instruction Tuning
di: Nagaraj, Manish, et al.
Pubblicazione: (2025)

On the Representational Capacity of Recurrent Neural Language Models
di: Nowak, Franz, et al.
Pubblicazione: (2023)

LSR-Adapt: Ultra-Efficient Parameter Tuning with Matrix Low Separation Rank Kernel Adaptation
di: Li, Xin, et al.
Pubblicazione: (2025)

Distil-xLSTM: Learning Attention Mechanisms through Recurrent Structures
di: Thiombiano, Abdoul Majid O., et al.
Pubblicazione: (2025)

LocMoE: A Low-Overhead MoE for Large Language Model Training
di: Li, Jing, et al.
Pubblicazione: (2024)

Context Tuning for In-Context Optimization
di: Lu, Jack, et al.
Pubblicazione: (2025)

ROSA: Random Subspace Adaptation for Efficient Fine-Tuning
di: Hameed, Marawan Gamal Abdel, et al.
Pubblicazione: (2024)

Recurrent Drafter for Fast Speculative Decoding in Large Language Models
di: Cheng, Yunfei, et al.
Pubblicazione: (2024)

LaMDA: Large Model Fine-Tuning via Spectrally Decomposed Low-Dimensional Adaptation
di: Azizi, Seyedarmin, et al.
Pubblicazione: (2024)

Aligning the Spectrum: Hybrid Graph Pre-training and Prompt Tuning across Homophily and Heterophily
di: Luo, Haitong, et al.
Pubblicazione: (2025)

Efficient Parallel Samplers for Recurrent-Depth Models and Their Connection to Diffusion Language Models
di: Geiping, Jonas, et al.
Pubblicazione: (2025)

MoR: Mixture of Ranks for Low-Rank Adaptation Tuning
di: Tang, Chuanyu, et al.
Pubblicazione: (2024)

Supervised Fine-Tuning Achieve Rapid Task Adaption Via Alternating Attention Head Activation Patterns
di: Zhao, Yang, et al.
Pubblicazione: (2024)

Rethinking State Tracking in Recurrent Models Through Error Control Dynamics
di: Chung, Jiwan, et al.
Pubblicazione: (2026)

AFLoRA: Adaptive Freezing of Low Rank Adaptation in Parameter Efficient Fine-Tuning of Large Models
di: Liu, Zeyu, et al.
Pubblicazione: (2024)

$V_0$: A Generalist Value Model for Any Policy at State Zero
di: Zhang, Yi-Kai, et al.
Pubblicazione: (2026)

Hybrid Quantum-Classical Recurrent Neural Networks
di: Xu, Wenduan
Pubblicazione: (2025)

Recurrent Confidence Chain: Temporal-Aware Uncertainty Quantification in Large Language Models
di: Mao, Zhenjiang, et al.
Pubblicazione: (2026)

LightTransfer: Your Long-Context LLM is Secretly a Hybrid Model with Effortless Adaptation
di: Zhang, Xuan, et al.
Pubblicazione: (2024)

MiniCPM-SALA: Hybridizing Sparse and Linear Attention for Efficient Long-Context Modeling
di: MiniCPM Team, et al.
Pubblicazione: (2026)

An Algebraic View of the Expressivity of Recurrent Language Models
di: Nowak, Franz, et al.
Pubblicazione: (2026)

Not All Adapters Matter: Selective Adapter Freezing for Memory-Efficient Fine-Tuning of Language Models
di: Son, Hyegang, et al.
Pubblicazione: (2024)

RoSA: Accurate Parameter-Efficient Fine-Tuning via Robust Adaptation
di: Nikdan, Mahdi, et al.
Pubblicazione: (2024)