Cover Image

PanGu-$π$: Enhancing Language Model Architectures via Nonlinearity Compensation

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Wang, Yunhe, Chen, Hanting, Tang, Yehui, Guo, Tianyu, Han, Kai, Nie, Ying, Wang, Xutao, Hu, Hailin, Bai, Zheyuan, Wang, Yun, Liu, Fangcheng, Liu, Zhicheng, Guo, Jianyuan, Zeng, Sinan, Zhang, Yinchen, Xu, Qinghua, Liu, Qun, Yao, Jun, Xu, Chao, Tao, Dacheng
Format:	Preprint
Published:	2023
Subjects:	Computation and Language Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

PanGu-$π$ Pro:Rethinking Optimization and Architecture for Tiny Language Models
by: Tang, Yehui, et al.
Published: (2024)

DiJiang: Efficient Large Language Models through Compact Kernelization
by: Chen, Hanting, et al.
Published: (2024)

A Survey on Transformer Compression
by: Tang, Yehui, et al.
Published: (2024)

PanGu-Draw: Advancing Resource-Efficient Text-to-Image Synthesis with Time-Decoupled Training and Reusable Coop-Diffusion
by: Lu, Guansong, et al.
Published: (2023)

Multiscale Positive-Unlabeled Detection of AI-Generated Texts
by: Tian, Yuchuan, et al.
Published: (2023)

Token Compensator: Altering Inference Cost of Vision Transformer without Re-Tuning
by: Jie, Shibo, et al.
Published: (2024)

Vision Superalignment: Weak-to-Strong Generalization for Vision Foundation Models
by: Guo, Jianyuan, et al.
Published: (2024)

Kangaroo: Lossless Self-Speculative Decoding via Double Early Exiting
by: Liu, Fangcheng, et al.
Published: (2024)

Free Video-LLM: Prompt-guided Visual Perception for Efficient Training-free Video LLMs
by: Han, Kai, et al.
Published: (2024)

LiveClawBench: Benchmarking LLM Agents on Complex, Real-World Assistant Tasks
by: Long, Xiang, et al.
Published: (2026)

SAM-DiffSR: Structure-Modulated Diffusion Model for Image Super-Resolution
by: Wang, Chengcheng, et al.
Published: (2024)

Gold-YOLO: Efficient Object Detector via Gather-and-Distribute Mechanism
by: Wang, Chengcheng, et al.
Published: (2023)

ScaleNet: Scaling up Pretrained Neural Networks with Incremental Parameters
by: Hao, Zhiwei, et al.
Published: (2025)

Unshackling Context Length: An Efficient Selective Attention Approach through Query-Key Compression
by: Wang, Haoyu, et al.
Published: (2025)

SlimLLM: Accurate Structured Pruning for Large Language Models
by: Guo, Jialong, et al.
Published: (2025)

SLAB: Efficient Transformers with Simplified Linear Attention and Progressive Re-parameterized Batch Normalization
by: Guo, Jialong, et al.
Published: (2024)

Saliency-driven Dynamic Token Pruning for Large Language Models
by: Tao, Yao, et al.
Published: (2025)

ParameterNet: Parameters Are All You Need
by: Han, Kai, et al.
Published: (2023)

Nexus: Higher-Order Attention Mechanisms in Transformers
by: Chen, Hanting, et al.
Published: (2025)

MoRAgent: Parameter Efficient Agent Tuning with Mixture-of-Roles
by: Han, Jing, et al.
Published: (2025)

DenseMamba: State Space Models with Dense Hidden Connection for Efficient Large Language Models
by: He, Wei, et al.
Published: (2024)

ROOT: Robust Orthogonalized Optimizer for Neural Network Training
by: He, Wei, et al.
Published: (2025)

Pangu Pro MoE: Mixture of Grouped Experts for Efficient Sparsity
by: Tang, Yehui, et al.
Published: (2025)

GhostNetV3: Exploring the Training Strategies for Compact Models
by: Liu, Zhenhua, et al.
Published: (2024)

Forest-of-Thought: Scaling Test-Time Compute for Enhancing LLM Reasoning
by: Bi, Zhenni, et al.
Published: (2024)

EMS-SD: Efficient Multi-sample Speculative Decoding for Accelerating Large Language Models
by: Ni, Yunsheng, et al.
Published: (2024)

Deferred Commitment Decoding for Diffusion Language Models
by: Shu, Yingte, et al.
Published: (2026)

C-MOP: Integrating Momentum and Boundary-Aware Clustering for Enhanced Prompt Evolution
by: Yan, Binwei, et al.
Published: (2026)

On the Positive and Negative $p$-Energies of Graphs under Edge Addition
by: Tang, Quanyu, et al.
Published: (2024)

On a conjecture of Nikiforov concerning the minimal $p$-energy of connected graphs
by: Tang, Quanyu, et al.
Published: (2024)

CFinBench: A Comprehensive Chinese Financial Benchmark for Large Language Models
by: Nie, Ying, et al.
Published: (2024)

LLM Data Selection and Utilization via Dynamic Bi-level Optimization
by: Yu, Yang, et al.
Published: (2025)

Multi-Granularity Semantic Revision for Large Language Model Distillation
by: Liu, Xiaoyu, et al.
Published: (2024)

Revealing the Power of Post-Training for Small Language Models via Knowledge Distillation
by: Rang, Miao, et al.
Published: (2025)

Data-efficient Large Vision Models through Sequential Autoregression
by: Guo, Jianyuan, et al.
Published: (2024)

Path-Minimality of $p$-Energy for Connected Graphs
by: Liu, Yinchen, et al.
Published: (2026)

ReGuLaR: Variational Latent Reasoning Guided by Rendered Chain-of-Thought
by: Wang, Fanmeng, et al.
Published: (2026)

Successful Treatment of Escherichia coli ‐Related Spondylitis With an Extended Meropenem Infusion Time for a 72‐Year‐Old Woman: A Case Report
by: Wang Hongbin, et al.
Published: (2025)

Transferable text data distillation by trajectory matching
by: Yao, Rong, et al.
Published: (2025)

Eve: Efficient Multimodal Vision Language Models with Elastic Visual Experts
by: Rang, Miao, et al.
Published: (2025)