Imagen de Portada

Exploring Efficient Learning of Small BERT Networks with LoRA and DoRA

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Frees, Daniel, Bhagirath, Aditri, Bolling, Moritz
Formato:	Preprint
Publicado:	2025
Materias:	Machine Learning 68T50
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Towards Optimal Convolutional Transfer Learning Architectures for Breast Lesion Classification and ACL Tear Detection
por: Frees, Daniel, et al.
Publicado: (2025)

QuAILoRA: Quantization-Aware Initialization for LoRA
por: Lawton, Neal, et al.
Publicado: (2024)

CausalSent: Interpretable Sentiment Classification with RieszNet
por: Frees, Daniel, et al.
Publicado: (2025)

Improving Recursive Transformers with Mixture of LoRAs
por: Nouriborji, Mohammadmahdi, et al.
Publicado: (2025)

Evaluating Embedding Generalization: How LLMs, LoRA, and SLERP Shape Representational Geometry
por: Kabane, Siyaxolisa
Publicado: (2025)

FedRot-LoRA: Mitigating Rotational Misalignment in Federated LoRA
por: Zhang, Haoran, et al.
Publicado: (2026)

LoRA meets Riemannion: Muon Optimizer for Parametrization-independent Low-Rank Adapters
por: Bogachev, Vladimir, et al.
Publicado: (2025)

Balanced LoRA: Removing Parameter Invariance to Accelerate Convergence
por: Castin, Valérie, et al.
Publicado: (2026)

Robust Hybrid Classical-Quantum Transfer Learning Model for Text Classification Using GPT-Neo 125M with LoRA & SMOTE Enhancement
por: Wishal, Santanam
Publicado: (2025)

Two Is Better Than One: Rotations Scale LoRAs
por: Guo, Hongcan, et al.
Publicado: (2025)

Sub-Token Routing in LoRA for Adaptation and Query-Aware KV Compression
por: Jiang, Wei, et al.
Publicado: (2026)

jina-embeddings-v3: Multilingual Embeddings With Task LoRA
por: Sturua, Saba, et al.
Publicado: (2024)

Preserving Empirical Probabilities in BERT for Small-sample Clinical Entity Recognition
por: Rehman, Abdul, et al.
Publicado: (2024)

CE-LoRA: Computation-Efficient LoRA Fine-Tuning for Language Models
por: Chen, Guanduo, et al.
Publicado: (2025)

R-LoRA: Randomized Multi-Head LoRA for Efficient Multi-Task Learning
por: Liu, Jinda, et al.
Publicado: (2025)

Efficient Calibration for RRAM-based In-Memory Computing using DoRA
por: Dong, Weirong, et al.
Publicado: (2025)

Deep Learning and Transfer Learning Architectures for English Premier League Player Performance Forecasting
por: Frees, Daniel, et al.
Publicado: (2024)

Run LoRA Run: Faster and Lighter LoRA Implementations
por: Cherniuk, Daria, et al.
Publicado: (2023)

LoRA-drop: Efficient LoRA Parameter Pruning based on Output Evaluation
por: Zhou, Hongyun, et al.
Publicado: (2024)

Dynamic Parameter Memory: Temporary LoRA-Enhanced LLM for Long-Sequence Emotion Recognition in Conversation
por: Mai, Jialong, et al.
Publicado: (2025)

Optimizing Retrieval-Augmented Generation (RAG) for Colloquial Cantonese: A LoRA-Based Systematic Review
por: Calonge, David Santandreu, et al.
Publicado: (2025)

Hallucinations and Truth: A Comprehensive Accuracy Evaluation of RAG, LoRA and DoRA
por: Baqar, Mohammad, et al.
Publicado: (2025)

Targeted Lexical Injection: Unlocking Latent Cross-Lingual Alignment in Lugha-Llama via Early-Layer LoRA Fine-Tuning
por: Ngugi, Stanley
Publicado: (2025)

Beyond the Surface: Uncovering Implicit Locations with LLMs for Personalized Local News
por: Katz, Gali, et al.
Publicado: (2025)

PLoRA: Efficient LoRA Hyperparameter Tuning for Large Models
por: Yan, Minghao, et al.
Publicado: (2025)

PRoLoRA: Partial Rotation Empowers More Parameter-Efficient LoRA
por: Wang, Sheng, et al.
Publicado: (2024)

LoRA Diffusion: Zero-Shot LoRA Synthesis for Diffusion Model Personalization
por: Smith, Ethan, et al.
Publicado: (2024)

AC-LoRA: Auto Component LoRA for Personalized Artistic Style Image Generation
por: Cui, Zhipu, et al.
Publicado: (2025)

S-LoRA: Serving Thousands of Concurrent LoRA Adapters
por: Sheng, Ying, et al.
Publicado: (2023)

Surfing the modeling of PoS taggers in low-resource scenarios
por: Ferro, Manuel Vilares, et al.
Publicado: (2024)

Block-Diagonal LoRA for Eliminating Communication Overhead in Tensor Parallel LoRA Serving
por: Wang, Xinyu, et al.
Publicado: (2025)

tLoRA: Efficient Multi-LoRA Training with Elastic Shared Super-Models
por: Li, Kevin, et al.
Publicado: (2026)

SC-LoRA: Balancing Efficient Fine-tuning and Knowledge Preservation via Subspace-Constrained LoRA
por: Luo, Minrui, et al.
Publicado: (2025)

GeLoRA: Geometric Adaptive Ranks For Efficient LoRA Fine-tuning
por: Ed-dib, Abdessalam, et al.
Publicado: (2024)

LoRA on the Go: Instance-level Dynamic LoRA Selection and Merging
por: Lee, Seungeon, et al.
Publicado: (2025)

Multi-Model Synthetic Training for Mission-Critical Small Language Models
por: Platt, Nolan, et al.
Publicado: (2025)

KD-LoRA: A Hybrid Approach to Efficient Fine-Tuning with LoRA and Knowledge Distillation
por: Azimi, Rambod, et al.
Publicado: (2024)

Evaluating AI Grading on Real-World Handwritten College Mathematics: A Large-Scale Study Toward a Benchmark
por: Yu, Zhiqi, et al.
Publicado: (2026)

CopRA: A Progressive LoRA Training Strategy
por: Zhuang, Zhan, et al.
Publicado: (2024)

Cross-LoRA: A Data-Free LoRA Transfer Framework across Heterogeneous LLMs
por: Xia, Feifan, et al.
Publicado: (2025)