Imagen de Portada

Cross-layer Attention Sharing for Pre-trained Large Language Models

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Mu, Yongyu, Wu, Yuzhang, Fan, Yuchun, Wang, Chenglong, Li, Hengyu, Zeng, Jiali, He, Qiaozhi, Yang, Murun, Meng, Fandong, Zhou, Jie, Xiao, Tong, Zhu, Jingbo
Formato:	Preprint
Publicado:	2024
Materias:	Computation and Language
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

LRHP: Learning Representations for Human Preferences via Preference Pairs
por: Wang, Chenglong, et al.
Publicado: (2024)

Offline Exploration-Aware Fine-Tuning for Long-Chain Mathematical Reasoning
por: Mu, Yongyu, et al.
Publicado: (2026)

DaPT: A Dual-Path Framework for Multilingual Multi-hop Question Answering
por: Wang, Yilin, et al.
Publicado: (2026)

Dissecting Long-Chain-of-Thought Reasoning Models: An Empirical Study
por: Mu, Yongyu, et al.
Publicado: (2025)

Boosting Text-To-Image Generation via Multilingual Prompting in Large Multimodal Models
por: Mu, Yongyu, et al.
Publicado: (2025)

GRAM-R$^2$: Self-Training Generative Foundation Reward Models for Reward Reasoning
por: Wang, Chenglong, et al.
Publicado: (2025)

GRAM: A Generative Foundation Reward Model for Reward Generalization
por: Wang, Chenglong, et al.
Publicado: (2025)

RoVRM: A Robust Visual Reward Model Optimized via Auxiliary Textual Preference Data
por: Wang, Chenglong, et al.
Publicado: (2024)

Probing Preference Representations: A Multi-Dimensional Evaluation and Analysis Method for Reward Models
por: Wang, Chenglong, et al.
Publicado: (2025)

Translate-and-Revise: Boosting Large Language Models for Constrained Translation
por: Huang, Pengcheng, et al.
Publicado: (2024)

Language-Specific Layer Matters: Efficient Multilingual Enhancement for Large Vision-Language Models
por: Fan, Yuchun, et al.
Publicado: (2025)

TIM: Teaching Large Language Models to Translate with Comparison
por: Zeng, Jiali, et al.
Publicado: (2023)

Improving Machine Translation with Large Language Models: A Preliminary Study with Cooperative Decoding
por: Zeng, Jiali, et al.
Publicado: (2023)

Revealing the Parallel Multilingual Learning within Large Language Models
por: Mu, Yongyu, et al.
Publicado: (2024)

Hybrid Alignment Training for Large Language Models
por: Wang, Chenglong, et al.
Publicado: (2024)

Understanding and Addressing the Under-Translation Problem from the Perspective of Decoding Objective
por: Shao, Chenze, et al.
Publicado: (2024)

MRO: Enhancing Reasoning in Diffusion Language Models via Multi-Reward Optimization
por: Wang, Chenglong, et al.
Publicado: (2025)

SLAM: Towards Efficient Multilingual Reasoning via Selective Language Alignment
por: Fan, Yuchun, et al.
Publicado: (2025)

TasTe: Teaching Large Language Models to Translate through Self-Reflection
por: Wang, Yutong, et al.
Publicado: (2024)

APR: Penalizing Structural Redundancy in Large Reasoning Models via Anchor-based Process Rewards
por: Chang, Kaiyan, et al.
Publicado: (2026)

Generative Multi-Modal Knowledge Retrieval with Large Language Models
por: Long, Xinwei, et al.
Publicado: (2024)

LANG: Reinforcement Learning for Multilingual Reasoning with Language-Adaptive Hint Guidance
por: Fan, Yuchun, et al.
Publicado: (2026)

LongDPO: Unlock Better Long-form Generation Abilities for LLMs via Critique-augmented Stepwise Information
por: Ping, Bowen, et al.
Publicado: (2025)

MSRL: Scaling Generative Multimodal Reward Modeling via Multi-Stage Reinforcement Learning
por: Wang, Chenglong, et al.
Publicado: (2026)

SPS: Steering Probability Squeezing for Better Exploration in Reinforcement Learning for Large Language Models
por: Huo, Yifu, et al.
Publicado: (2026)

LexMatcher: Dictionary-centric Data Collection for LLM-based Machine Translation
por: Yin, Yongjing, et al.
Publicado: (2024)

Accelerating Inference in Large Language Models with a Unified Layer Skipping Strategy
por: Liu, Yijin, et al.
Publicado: (2024)

Beyond Next Token Prediction: Patch-Level Training for Large Language Models
por: Shao, Chenze, et al.
Publicado: (2024)

Foundations of Large Language Models
por: Xiao, Tong, et al.
Publicado: (2025)

Beyond Decoder-only: Large Language Models Can be Good Encoders for Machine Translation
por: Luo, Yingfeng, et al.
Publicado: (2025)

DeepRAG: Thinking to Retrieve Step by Step for Large Language Models
por: Guan, Xinyan, et al.
Publicado: (2025)

NDP: Next Distribution Prediction as a More Broad Target
por: Ruan, Junhao, et al.
Publicado: (2024)

MiniPLM: Knowledge Distillation for Pre-Training Language Models
por: Gu, Yuxian, et al.
Publicado: (2024)

Enhanced Atrial Fibrillation Prediction in ESUS Patients with Hypergraph-based Pre-training
por: Xie, Yuzhang, et al.
Publicado: (2026)

Prior Constraints-based Reward Model Training for Aligning Large Language Models
por: Zhou, Hang, et al.
Publicado: (2024)

LORA-CRAFT: Cross-layer Rank Adaptation via Frozen Tucker Decomposition of Pre-trained Attention Weights
por: Dewage, Kasun, et al.
Publicado: (2026)

DelTA: An Online Document-Level Translation Agent Based on Multi-Level Memory
por: Wang, Yutong, et al.
Publicado: (2024)

Vehicle-centric Perception via Multimodal Structured Pre-training
por: Wu, Wentao, et al.
Publicado: (2025)

TEAL: Tokenize and Embed ALL for Multi-modal Large Language Models
por: Yang, Zhen, et al.
Publicado: (2023)

Exploring Scaling Laws for Local SGD in Large Language Model Training
por: He, Qiaozhi, et al.
Publicado: (2024)