Copertina

Where Should LoRA Go? Component-Type Placement in Hybrid Language Models

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Borobia, Hector, Seguí-Mas, Elies, Tormo-Carbó, Guillermina
Natura:	Preprint
Pubblicazione:	2026
Soggetti:	Computation and Language Machine Learning I.2.7
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Component-Aware Self-Speculative Decoding in Hybrid Language Models
di: Borobia, Hector, et al.
Pubblicazione: (2026)

How Pruning Reshapes Features: Sparse Autoencoder Analysis of Weight-Pruned Language Models
di: Borobia, Hector, et al.
Pubblicazione: (2026)

Functional Component Ablation Reveals Specialization Patterns in Hybrid Language Model Architectures
di: Borobia, Hector, et al.
Pubblicazione: (2026)

Linear-Readout Floors and Threshold Recovery in Computation in Superposition
di: Borobia, Hector, et al.
Pubblicazione: (2026)

FIM-LoRA: Task-Informative Rank Allocation for LoRA via Calibration-Time Gradient-Variance Estimation
di: Sathyavageeswaran, Ramakrishnan
Pubblicazione: (2026)

SpectralLoRA: Is Low-Frequency Structure Sufficient for LoRA Adaptation? A Spectral Analysis of Weight Updates
di: Singh, Rajveer
Pubblicazione: (2026)

MeteoRA: Multiple-tasks Embedded LoRA for Large Language Models
di: Xu, Jingwei, et al.
Pubblicazione: (2024)

Annotation Entropy Predicts Per-Example Learning Dynamics in LoRA Fine-Tuning
di: Steele, Brady
Pubblicazione: (2026)

Ouroboros: Dynamic Weight Generation for Recursive Transformers via Input-Conditioned LoRA Modulation
di: Jaber, Jaber, et al.
Pubblicazione: (2026)

Comparative Analysis of LoRA-Adapted Embedding Models for Clinical Cardiology Text Representation
di: Young, Richard J., et al.
Pubblicazione: (2025)

Disaster Question Answering with LoRA Efficiency and Accurate End Position
di: Yasuno, Takato
Pubblicazione: (2026)

LayerRoute: Input-Conditioned Adaptive Layer Skipping via LoRA Fine-Tuning for Agentic Language Models
di: Sikdar, Prateek Kumar
Pubblicazione: (2026)

FedRot-LoRA: Mitigating Rotational Misalignment in Federated LoRA
di: Zhang, Haoran, et al.
Pubblicazione: (2026)

Towards Multi-Level Transcript Segmentation: LoRA Fine-Tuning for Table-of-Contents Generation
di: Freisinger, Steffen, et al.
Pubblicazione: (2026)

Why LoRA Resists Label Noise: A Theoretical Framework for Noise-Robust Parameter-Efficient Fine-Tuning
di: Steele, Brady
Pubblicazione: (2026)

Evaluating the Efficacy of Hybrid Deep Learning Models in Distinguishing AI-Generated Text
di: Oketunji, Abiodun Finbarrs
Pubblicazione: (2023)

Token-Level Generalization in LoRA Adapter Backdoors: Attack Characterization and Behavioral Detection
di: Lelle, Travis
Pubblicazione: (2026)

Targeted Lexical Injection: Unlocking Latent Cross-Lingual Alignment in Lugha-Llama via Early-Layer LoRA Fine-Tuning
di: Ngugi, Stanley
Pubblicazione: (2025)

Have Faith in Faithfulness: Going Beyond Circuit Overlap When Finding Model Mechanisms
di: Hanna, Michael, et al.
Pubblicazione: (2024)

jina-embeddings-v3: Multilingual Embeddings With Task LoRA
di: Sturua, Saba, et al.
Pubblicazione: (2024)

Planning vs Reasoning: Ablations to Test Capabilities of LoRA layers
di: Redkar, Neel
Pubblicazione: (2024)

MALoRA: Mixture of Asymmetric Low-Rank Adaptation for Enhanced Multi-Task Learning
di: Wang, Xujia, et al.
Pubblicazione: (2024)

LoRS: Efficient Low-Rank Adaptation for Sparse Large Language Model
di: Hu, Yuxuan, et al.
Pubblicazione: (2025)

Optimizing Retrieval-Augmented Generation (RAG) for Colloquial Cantonese: A LoRA-Based Systematic Review
di: Calonge, David Santandreu, et al.
Pubblicazione: (2025)

Large Language Model (LLM) Bias Index -- LLMBI
di: Oketunji, Abiodun Finbarrs, et al.
Pubblicazione: (2023)

Engineering A Large Language Model From Scratch
di: Oketunji, Abiodun Finbarrs
Pubblicazione: (2024)

The Appeal and Reality of Recycling LoRAs with Adaptive Merging
di: Liu, Haokun, et al.
Pubblicazione: (2026)

Towards Human Understanding of Paraphrase Types in Large Language Models
di: Meier, Dominik, et al.
Pubblicazione: (2024)

Towards Fundamental Language Models: Does Linguistic Competence Scale with Model Size?
di: Collado-Montañez, Jaime, et al.
Pubblicazione: (2025)

Ensemble Language Models for Multilingual Sentiment Analysis
di: Hasan, Md Arid
Pubblicazione: (2024)

RBCorr: Response Bias Correction in Language Models
di: Bhatt, Om, et al.
Pubblicazione: (2026)

Compact Example-Based Explanations for Language Models
di: Schoenegger, Loris, et al.
Pubblicazione: (2026)

On the Effect of (Near) Duplicate Subwords in Language Modelling
di: Schäfer, Anton, et al.
Pubblicazione: (2024)

Entropy-Based Measurement of Value Drift and Alignment Work in Large Language Models
di: Fadli, Samih
Pubblicazione: (2025)

The Open Source Advantage in Large Language Models (LLMs)
di: Manchanda, Jiya, et al.
Pubblicazione: (2024)

MMSciBench: Benchmarking Language Models on Chinese Multimodal Scientific Problems
di: Ye, Xinwu, et al.
Pubblicazione: (2025)

Scaling Laws for Forgetting When Fine-Tuning Large Language Models
di: Kalajdzievski, Damjan
Pubblicazione: (2024)

Distilling Robustness into Natural Language Inference Models with Domain-Targeted Augmentation
di: Stacey, Joe, et al.
Pubblicazione: (2023)

Sparsing Law: Towards Large Language Models with Greater Activation Sparsity
di: Luo, Yuqi, et al.
Pubblicazione: (2024)

A Language Model based Framework for New Concept Placement in Ontologies
di: Dong, Hang, et al.
Pubblicazione: (2024)