Cover Image

DoTA: Weight-Decomposed Tensor Adaptation for Large Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Hu, Xiaolin, Cheng, Xiang, Liu, Peiyu, Liu, Wei, Luan, Jian, Wang, Bin, Liu, Yong
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

PMSS: Pretrained Matrices Skeleton Selection for LLM Fine-tuning
by: Wang, Qibin, et al.
Published: (2024)

BiDoRA: Bi-level Optimization-Based Weight-Decomposed Low-Rank Adaptation
by: Qin, Peijia, et al.
Published: (2024)

DoTA-RAG: Dynamic of Thought Aggregation RAG
by: Ruangtanusak, Saksorn, et al.
Published: (2025)

Accelerating the Low-Rank Decomposed Models
by: Hajimolahoseini, Habib, et al.
Published: (2024)

HoPE: A Novel Positional Encoding Without Long-Term Decay for Enhanced Context Awareness and Extrapolation
by: Chen, Yuhan, et al.
Published: (2024)

Towards Auto-Regressive Next-Token Prediction: In-Context Learning Emerges from Generalization
by: Gong, Zixuan, et al.
Published: (2025)

On the Thinking-Language Modeling Gap in Large Language Models
by: Liu, Chenxi, et al.
Published: (2025)

Exploring Context Window of Large Language Models via Decomposed Positional Vectors
by: Dong, Zican, et al.
Published: (2024)

LoTR: Low Tensor Rank Weight Adaptation
by: Bershatsky, Daniel, et al.
Published: (2024)

GUNDAM: Aligning Large Language Models with Graph Understanding
by: Ouyang, Sheng, et al.
Published: (2024)

Dataless Knowledge Fusion by Merging Weights of Language Models
by: Jin, Xisen, et al.
Published: (2022)

Theoretical Insights into Fine-Tuning Attention Mechanism: Generalization and Optimization
by: Yao, Xinhao, et al.
Published: (2024)

DATA: Decomposed Attention-based Task Adaptation for Rehearsal-Free Continual Learning
by: Liao, Huanxuan, et al.
Published: (2025)

EDoRA: Efficient Weight-Decomposed Low-Rank Adaptation via Singular Value Decomposition
by: Nasiri, Hamid, et al.
Published: (2025)

CoRA: Optimizing Low-Rank Adaptation with Common Subspace of Large Language Models
by: Xiao, Xiaojun, et al.
Published: (2024)

DoRA: Weight-Decomposed Low-Rank Adaptation
by: Liu, Shih-Yang, et al.
Published: (2024)

LIDAO: Towards Limited Interventions for Debiasing (Large) Language Models
by: Liu, Tianci, et al.
Published: (2024)

Domain-Specific Pruning of Large Mixture-of-Experts Models with Few-shot Demonstrations
by: Dong, Zican, et al.
Published: (2025)

Do LLMs Overcome Shortcut Learning? An Evaluation of Shortcut Challenges in Large Language Models
by: Yuan, Yu, et al.
Published: (2024)

Stability-Weighted Decoding for Diffusion Language Models
by: Wu, Yue, et al.
Published: (2026)

LaMDA: Large Model Fine-Tuning via Spectrally Decomposed Low-Dimensional Adaptation
by: Azizi, Seyedarmin, et al.
Published: (2024)

AutoTimes: Autoregressive Time Series Forecasters via Large Language Models
by: Liu, Yong, et al.
Published: (2024)

PEANuT: Parameter-Efficient Adaptation with Weight-aware Neural Tweakers
by: Zhong, Yibo, et al.
Published: (2024)

PrefixQuant: Eliminating Outliers by Prefixed Tokens for Large Language Models Quantization
by: Chen, Mengzhao, et al.
Published: (2024)

Vaccine: Perturbation-aware Alignment for Large Language Models against Harmful Fine-tuning Attack
by: Huang, Tiansheng, et al.
Published: (2024)

CEQuest: Benchmarking Large Language Models for Construction Estimation
by: Wu, Yanzhao, et al.
Published: (2025)

HiDe-LLaVA: Hierarchical Decoupling for Continual Instruction Tuning of Multimodal Large Language Model
by: Guo, Haiyang, et al.
Published: (2025)

MAP: Revisiting Weight Decomposition for Low-Rank Adaptation
by: Si, Chongjie, et al.
Published: (2025)

DFA-RAG: Conversational Semantic Router for Large Language Model with Definite Finite Automaton
by: Sun, Yiyou, et al.
Published: (2024)

Exploring Vulnerabilities and Protections in Large Language Models: A Survey
by: Liu, Frank Weizhen, et al.
Published: (2024)

Model Merging in Pre-training of Large Language Models
by: Li, Yunshui, et al.
Published: (2025)

GWQ: Gradient-Aware Weight Quantization for Large Language Models
by: Shao, Yihua, et al.
Published: (2024)

A Survey on Medical Large Language Models: Technology, Application, Trustworthiness, and Future Directions
by: Liu, Lei, et al.
Published: (2024)

Revisiting Entropy in Reinforcement Learning for Large Reasoning Models
by: Jin, Renren, et al.
Published: (2025)

Dynamic Adaptation of LoRA Fine-Tuning for Efficient and Task-Specific Optimization of Large Language Models
by: Liao, Xiaoxuan, et al.
Published: (2025)

CrossTrafficLLM: A Human-Centric Framework for Interpretable Traffic Intelligence via Large Language Model
by: Du, Zeming, et al.
Published: (2025)

OMoE: Diversifying Mixture of Low-Rank Adaptation by Orthogonal Finetuning
by: Feng, Jinyuan, et al.
Published: (2025)

CBQ: Cross-Block Quantization for Large Language Models
by: Ding, Xin, et al.
Published: (2023)

Linear Dynamics in the RLVR Training of Large Language Models
by: Wang, Tianle, et al.
Published: (2026)

C$^2$-Cite: Contextual-Aware Citation Generation for Attributed Large Language Models
by: Yu, Yue, et al.
Published: (2025)