Imagen de Portada

Punctuation-aware Hybrid Trainable Sparse Attention for Large Language Models

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Qiu, Junxiang, Wang, Shuo, Chen, Zhengsu, Zhang, Hengheng, Lu, Jinda, Li, Changcheng, Tian, Qi
Formato:	Preprint
Publicado:	2026
Materias:	Computation and Language
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Confidence Before Answering: A Paradigm Shift for Efficient LLM Uncertainty Estimation
por: Li, Changcheng, et al.
Publicado: (2026)

Punctuation-aware treebank tree binarization
por: Klinger, Eitan, et al.
Publicado: (2025)

Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention
por: Yuan, Jingyang, et al.
Publicado: (2025)

Trainable Dynamic Mask Sparse Attention
por: Shi, Jingze, et al.
Publicado: (2025)

Punctuation and Predicates in Language Models
por: Chauhan, Sonakshi, et al.
Publicado: (2025)

Beyond Empathy: Integrating Diagnostic and Therapeutic Reasoning with Large Language Models for Mental Health Counseling
por: Hu, He, et al.
Publicado: (2025)

FASA: Frequency-aware Sparse Attention
por: Wang, Yifei, et al.
Publicado: (2026)

SparseD: Sparse Attention for Diffusion Language Models
por: Wang, Zeqing, et al.
Publicado: (2025)

ART: Attention Run-time Termination for Efficient Large Language Model Decoding
por: Qiu, Chen, et al.
Publicado: (2026)

Accelerating Controllable Generation via Hybrid-grained Cache
por: Liu, Lin, et al.
Publicado: (2025)

MeTA-LoRA: Data-Efficient Multi-Task Fine-Tuning for Large Language Models
por: Cheng, Bo, et al.
Publicado: (2025)

Punctuation Restoration for Singaporean Spoken Languages: English, Malay, and Mandarin
por: Rao, Abhinav, et al.
Publicado: (2022)

Attention-Aligned Reasoning for Large Language Models
por: Zhang, Hongxiang, et al.
Publicado: (2025)

Accelerating Diffusion Transformer via Gradient-Optimized Cache
por: Qiu, Junxiang, et al.
Publicado: (2025)

SparseEval: Efficient Evaluation of Large Language Models by Sparse Optimization
por: Zhang, Taolin, et al.
Publicado: (2026)

Is Random Attention Sufficient for Sequence Modeling? Disentangling Trainable Components in the Transformer
por: Dong, Yihe, et al.
Publicado: (2025)

Predicting Punctuation in Ancient Chinese Texts: A Multi-Layered LSTM and Attention-Based Approach
por: Cai, Tracy, et al.
Publicado: (2024)

MiniCPM-SALA: Hybridizing Sparse and Linear Attention for Efficient Long-Context Modeling
por: MiniCPM Team, et al.
Publicado: (2026)

Probabilistic Calibration Is a Trainable Capability in Language Models
por: Baldelli, Davide, et al.
Publicado: (2026)

A Hybrid Attention Framework for Fake News Detection with Large Language Models
por: Xu, Xiaochuan, et al.
Publicado: (2025)

SSA: Sparse Sparse Attention by Aligning Full and Sparse Attention Outputs in Feature Space
por: Shen, Zhenyi, et al.
Publicado: (2025)

Punctuation Prediction for Polish Texts using Transformers
por: Pokrywka, Jakub
Publicado: (2024)

A Small and Fast BERT for Chinese Medical Punctuation Restoration
por: Ling, Tongtao, et al.
Publicado: (2023)

Garbage Attention in Large Language Models: BOS Sink Heads and Sink-aware Pruning
por: Sok, Jaewon, et al.
Publicado: (2026)

Rhea: Role-aware Heuristic Episodic Attention for Conversational LLMs
por: Hong, Wanyang, et al.
Publicado: (2025)

Resolving Transcription Ambiguity in Spanish: A Hybrid Acoustic-Lexical System for Punctuation Restoration
por: Zhu, Xiliang, et al.
Publicado: (2024)

How Attention Sinks Emerge in Large Language Models: An Interpretability Perspective
por: Peng, Runyu, et al.
Publicado: (2026)

Advancing Text Classification with Large Language Models and Neural Attention Mechanisms
por: Lyu, Ning, et al.
Publicado: (2025)

Structure-aware Domain Knowledge Injection for Large Language Models
por: Liu, Kai, et al.
Publicado: (2024)

Punctuation Restoration Improves Structure Understanding Without Supervision
por: Min, Junghyun, et al.
Publicado: (2024)

Reminding Multimodal Large Language Models of Object-aware Knowledge with Retrieved Tags
por: Qi, Daiqing, et al.
Publicado: (2024)

LLaMA based Punctuation Restoration With Forward Pass Only Decoding
por: Pang, Yutong, et al.
Publicado: (2024)

End-to-end Joint Punctuated and Normalized ASR with a Limited Amount of Punctuated Training Data
por: Cui, Can, et al.
Publicado: (2023)

Rectified Sparse Attention
por: Sun, Yutao, et al.
Publicado: (2025)

ALPS: Attention Localization and Pruning Strategy for Efficient Alignment of Large Language Models
por: Chen, Hao, et al.
Publicado: (2025)

Mitigating Object Hallucinations in Large Vision-Language Models with Assembly of Global and Local Attention
por: An, Wenbin, et al.
Publicado: (2024)

UNIQUE: Universal Top-k Sparse Attention for Training-free Inference and Sparsity-aware Training
por: Deng, Keqi, et al.
Publicado: (2026)

Hybrid and Unitary PEFT for Resource-Efficient Large Language Models
por: Qi, Haomin, et al.
Publicado: (2025)

Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Free
por: Qiu, Zihan, et al.
Publicado: (2025)

Optimizing Multi-Task Learning for Enhanced Performance in Large Language Models
por: Qi, Zhen, et al.
Publicado: (2024)