Cover Image

Cross-layer Attention Sharing for Pre-trained Large Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Mu, Yongyu, Wu, Yuzhang, Fan, Yuchun, Wang, Chenglong, Li, Hengyu, Zeng, Jiali, He, Qiaozhi, Yang, Murun, Meng, Fandong, Zhou, Jie, Xiao, Tong, Zhu, Jingbo
Format:	Preprint
Published:	2024
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

LRHP: Learning Representations for Human Preferences via Preference Pairs
by: Wang, Chenglong, et al.
Published: (2024)

Offline Exploration-Aware Fine-Tuning for Long-Chain Mathematical Reasoning
by: Mu, Yongyu, et al.
Published: (2026)

DaPT: A Dual-Path Framework for Multilingual Multi-hop Question Answering
by: Wang, Yilin, et al.
Published: (2026)

Dissecting Long-Chain-of-Thought Reasoning Models: An Empirical Study
by: Mu, Yongyu, et al.
Published: (2025)

Boosting Text-To-Image Generation via Multilingual Prompting in Large Multimodal Models
by: Mu, Yongyu, et al.
Published: (2025)

GRAM-R$^2$: Self-Training Generative Foundation Reward Models for Reward Reasoning
by: Wang, Chenglong, et al.
Published: (2025)

GRAM: A Generative Foundation Reward Model for Reward Generalization
by: Wang, Chenglong, et al.
Published: (2025)

RoVRM: A Robust Visual Reward Model Optimized via Auxiliary Textual Preference Data
by: Wang, Chenglong, et al.
Published: (2024)

Probing Preference Representations: A Multi-Dimensional Evaluation and Analysis Method for Reward Models
by: Wang, Chenglong, et al.
Published: (2025)

Translate-and-Revise: Boosting Large Language Models for Constrained Translation
by: Huang, Pengcheng, et al.
Published: (2024)

Language-Specific Layer Matters: Efficient Multilingual Enhancement for Large Vision-Language Models
by: Fan, Yuchun, et al.
Published: (2025)

TIM: Teaching Large Language Models to Translate with Comparison
by: Zeng, Jiali, et al.
Published: (2023)

Improving Machine Translation with Large Language Models: A Preliminary Study with Cooperative Decoding
by: Zeng, Jiali, et al.
Published: (2023)

Revealing the Parallel Multilingual Learning within Large Language Models
by: Mu, Yongyu, et al.
Published: (2024)

Hybrid Alignment Training for Large Language Models
by: Wang, Chenglong, et al.
Published: (2024)

Understanding and Addressing the Under-Translation Problem from the Perspective of Decoding Objective
by: Shao, Chenze, et al.
Published: (2024)

MRO: Enhancing Reasoning in Diffusion Language Models via Multi-Reward Optimization
by: Wang, Chenglong, et al.
Published: (2025)

SLAM: Towards Efficient Multilingual Reasoning via Selective Language Alignment
by: Fan, Yuchun, et al.
Published: (2025)

TasTe: Teaching Large Language Models to Translate through Self-Reflection
by: Wang, Yutong, et al.
Published: (2024)

APR: Penalizing Structural Redundancy in Large Reasoning Models via Anchor-based Process Rewards
by: Chang, Kaiyan, et al.
Published: (2026)

Generative Multi-Modal Knowledge Retrieval with Large Language Models
by: Long, Xinwei, et al.
Published: (2024)

LANG: Reinforcement Learning for Multilingual Reasoning with Language-Adaptive Hint Guidance
by: Fan, Yuchun, et al.
Published: (2026)

LongDPO: Unlock Better Long-form Generation Abilities for LLMs via Critique-augmented Stepwise Information
by: Ping, Bowen, et al.
Published: (2025)

MSRL: Scaling Generative Multimodal Reward Modeling via Multi-Stage Reinforcement Learning
by: Wang, Chenglong, et al.
Published: (2026)

SPS: Steering Probability Squeezing for Better Exploration in Reinforcement Learning for Large Language Models
by: Huo, Yifu, et al.
Published: (2026)

LexMatcher: Dictionary-centric Data Collection for LLM-based Machine Translation
by: Yin, Yongjing, et al.
Published: (2024)

Accelerating Inference in Large Language Models with a Unified Layer Skipping Strategy
by: Liu, Yijin, et al.
Published: (2024)

Beyond Next Token Prediction: Patch-Level Training for Large Language Models
by: Shao, Chenze, et al.
Published: (2024)

Foundations of Large Language Models
by: Xiao, Tong, et al.
Published: (2025)

Beyond Decoder-only: Large Language Models Can be Good Encoders for Machine Translation
by: Luo, Yingfeng, et al.
Published: (2025)

DeepRAG: Thinking to Retrieve Step by Step for Large Language Models
by: Guan, Xinyan, et al.
Published: (2025)

NDP: Next Distribution Prediction as a More Broad Target
by: Ruan, Junhao, et al.
Published: (2024)

MiniPLM: Knowledge Distillation for Pre-Training Language Models
by: Gu, Yuxian, et al.
Published: (2024)

Enhanced Atrial Fibrillation Prediction in ESUS Patients with Hypergraph-based Pre-training
by: Xie, Yuzhang, et al.
Published: (2026)

Prior Constraints-based Reward Model Training for Aligning Large Language Models
by: Zhou, Hang, et al.
Published: (2024)

LORA-CRAFT: Cross-layer Rank Adaptation via Frozen Tucker Decomposition of Pre-trained Attention Weights
by: Dewage, Kasun, et al.
Published: (2026)

DelTA: An Online Document-Level Translation Agent Based on Multi-Level Memory
by: Wang, Yutong, et al.
Published: (2024)

Vehicle-centric Perception via Multimodal Structured Pre-training
by: Wu, Wentao, et al.
Published: (2025)

TEAL: Tokenize and Embed ALL for Multi-modal Large Language Models
by: Yang, Zhen, et al.
Published: (2023)

Exploring Scaling Laws for Local SGD in Large Language Model Training
by: He, Qiaozhi, et al.
Published: (2024)