Image de couverture de livre

Wonderful Matrices: More Efficient and Effective Architecture for Language Modeling Tasks

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Shi, Jingze, Wu, Bingheng, He, Lu, Jiang, Luchang
Format:	Preprint
Publié:	2024
Sujets:	Machine Learning Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Wonderful Matrices: Combining for a More Efficient and Effective Foundation Model Architecture
par: Shi, Jingze, et autres
Publié: (2024)

Trainable Dynamic Mask Sparse Attention
par: Shi, Jingze, et autres
Publié: (2025)

Learning Multi-Indicator Weights for Data Selection: A Joint Task-Model Adaptation Framework with Efficient Proxies
par: Song, Jingze, et autres
Publié: (2026)

When Linear Attention Meets Autoregressive Decoding: Towards More Effective and Efficient Linearized Large Language Models
par: You, Haoran, et autres
Publié: (2024)

$\mathrm{E^{2}CFD}$: Towards Effective and Efficient Cost Function Design for Safe Reinforcement Learning via Large Language Model
par: Wang, Zepeng, et autres
Publié: (2024)

Can RLHF be More Efficient with Imperfect Reward Models? A Policy Coverage Perspective
par: Huang, Jiawei, et autres
Publié: (2025)

Learning to Play 7 Wonders Duel Without Human Supervision
par: Paolini, Giovanni, et autres
Publié: (2024)

TransXSSM: A Hybrid Transformer State Space Model with Unified Rotary Position Embedding
par: Wu, Bingheng, et autres
Publié: (2025)

Generalizing Teacher Networks for Effective Knowledge Distillation Across Student Architectures
par: Binici, Kuluhan, et autres
Publié: (2024)

CorDA: Context-Oriented Decomposition Adaptation of Large Language Models for Task-Aware Parameter-Efficient Fine-tuning
par: Yang, Yibo, et autres
Publié: (2024)

The Effectiveness of Approximate Regularized Replay for Efficient Supervised Fine-Tuning of Large Language Models
par: Riemer, Matthew, et autres
Publié: (2025)

SMMF: Square-Matricized Momentum Factorization for Memory-Efficient Optimization
par: Park, Kwangryeol, et autres
Publié: (2024)

Efficient Multi-Task Reinforcement Learning with Cross-Task Policy Guidance
par: He, Jinmin, et autres
Publié: (2025)

KGLens: Towards Efficient and Effective Knowledge Probing of Large Language Models with Knowledge Graphs
par: Zheng, Shangshang, et autres
Publié: (2023)

ProteinOPD: Towards Effective and Efficient Preference Alignment for Protein Design
par: Zhang, Yulin, et autres
Publié: (2026)

ShiftAddNAS: Hardware-Inspired Search for More Accurate and Efficient Neural Networks
par: You, Haoran, et autres
Publié: (2022)

Asynchronous RLHF: Faster and More Efficient Off-Policy RL for Language Models
par: Noukhovitch, Michael, et autres
Publié: (2024)

More Efficient Randomized Exploration for Reinforcement Learning via Approximate Sampling
par: Ishfaq, Haque, et autres
Publié: (2024)

Evaluating the Paperclip Maximizer: Are RL-Based Language Models More Likely to Pursue Instrumental Goals?
par: He, Yufei, et autres
Publié: (2025)

Be Bayesian by Attachments to Catch More Uncertainty
par: Shen, Shiyu, et autres
Publié: (2023)

Revisiting Sharpness-Aware Minimization: A More Faithful and Effective Implementation
par: Chen, Jianlong, et autres
Publié: (2026)

Scaling Laws Across Model Architectures: A Comparative Analysis of Dense and MoE Models in Large Language Models
par: Wang, Siqi, et autres
Publié: (2024)

Language Control Diffusion: Efficiently Scaling through Space, Time, and Tasks
par: Zhang, Edwin, et autres
Publié: (2022)

Rapid and Continuous Trust Evaluation for Effective Task Collaboration Through Siamese Model
par: Zhu, Botao, et autres
Publié: (2025)

Unveiling Mode Connectivity in Graph Neural Networks
par: Li, Bingheng, et autres
Publié: (2025)

MGAS: Multi-Granularity Architecture Search for Trade-Off Between Model Effectiveness and Efficiency
par: Liu, Xiaoyun, et autres
Publié: (2023)

Aggregation-aware MLP: An Unsupervised Approach for Graph Message-passing
par: Xie, Xuanting, et autres
Publié: (2025)

Federated Data-Efficient Instruction Tuning for Large Language Models
par: Qin, Zhen, et autres
Publié: (2024)

Efficient UAV Swarm-Based Multi-Task Federated Learning with Dynamic Task Knowledge Sharing
par: Yang, Yubo, et autres
Publié: (2025)

Jet-Nemotron: Efficient Language Model with Post Neural Architecture Search
par: Gu, Yuxian, et autres
Publié: (2025)

Riemannian Networks over Full-Rank Correlation Matrices
par: Chen, Ziheng, et autres
Publié: (2026)

Shape of Thought: When Distribution Matters More than Correctness in Reasoning Tasks
par: Chandra, Abhranil, et autres
Publié: (2025)

Supernova: Achieving More with Less in Transformer Architectures
par: Tanase, Andrei-Valentin, et autres
Publié: (2025)

MO-EMT-NAS: Multi-Objective Continuous Transfer of Architectural Knowledge Between Tasks from Different Datasets
par: Liao, Peng, et autres
Publié: (2024)

Unified Multi-Task Learning & Model Fusion for Efficient Language Model Guardrailing
par: Neill, James O', et autres
Publié: (2025)

CoDAR: Continuous Diffusion Language Models are More Powerful Than You Think
par: Shen, Junzhe, et autres
Publié: (2026)

Efficient Multi-Task Modeling through Automated Fusion of Trained Models
par: Zhou, Jingxuan, et autres
Publié: (2025)

DiRL: An Efficient Post-Training Framework for Diffusion Language Models
par: Zhu, Ying, et autres
Publié: (2025)

LRQ: Optimizing Post-Training Quantization for Large Language Models by Learning Low-Rank Weight-Scaling Matrices
par: Lee, Jung Hyun, et autres
Publié: (2024)

BLAST: Block-Level Adaptive Structured Matrices for Efficient Deep Neural Network Inference
par: Lee, Changwoo, et autres
Publié: (2024)