Copertina

Generative Pretrained Structured Transformers: Unsupervised Syntactic Language Models at Scale

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Hu, Xiang, Ji, Pengyu, Zhu, Qingyang, Wu, Wei, Tu, Kewei
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Computation and Language Artificial Intelligence
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

A Systematic Study of Compositional Syntactic Transformer Language Models
di: Zhao, Yida, et al.
Pubblicazione: (2025)

Augmenting Transformers with Recursively Composed Multi-grained Representations
di: Hu, Xiang, et al.
Pubblicazione: (2023)

Unsupervised Morphological Tree Tokenizer
di: Zhu, Qingyang, et al.
Pubblicazione: (2024)

Dependency Transformer Grammars: Integrating Dependency Structures into Transformer Language Models
di: Zhao, Yida, et al.
Pubblicazione: (2024)

Efficient Length-Generalizable Attention via Causal Retrieval for Long-Context Language Modeling
di: Hu, Xiang, et al.
Pubblicazione: (2024)

Hardware-aligned Hierarchical Sparse Attention for Efficient Long-term Memory Access
di: Hu, Xiang, et al.
Pubblicazione: (2025)

The Grammar of Transformers: A Systematic Review of Interpretability Research on Syntactic Knowledge in Language Models
di: Graichen, Nora, et al.
Pubblicazione: (2026)

Syntactic Evolution in Language Usage
di: Kumar, Surbhit
Pubblicazione: (2025)

Flash Multi-Head Feed-Forward Network
di: Zhang, Minshen, et al.
Pubblicazione: (2025)

Syntactic Language Change in English and German: Metrics, Parsers, and Convergences
di: Chen, Yanran, et al.
Pubblicazione: (2024)

CodePMP: Scalable Preference Model Pretraining for Large Language Model Reasoning
di: Yu, Huimu, et al.
Pubblicazione: (2024)

Evaluating Semantic and Syntactic Understanding in Large Language Models for Payroll Systems
di: Maclean, Hendrika, et al.
Pubblicazione: (2026)

Syntactic Control of Language Models by Posterior Inference
di: Xefteri, Vicky, et al.
Pubblicazione: (2025)

Graph Neural Network Framework for Sentiment Analysis Using Syntactic Feature
di: Wu, Linxiao, et al.
Pubblicazione: (2024)

Vision-and-Language Navigation Generative Pretrained Transformer
di: Hanlin, Wen
Pubblicazione: (2024)

Every Token Counts: Generalizing 16M Ultra-Long Context in Large Language Models
di: Hu, Xiang, et al.
Pubblicazione: (2025)

Attributing Culture-Conditioned Generations to Pretraining Corpora
di: Li, Huihan, et al.
Pubblicazione: (2024)

TransGPT: Multi-modal Generative Pre-trained Transformer for Transportation
di: Wang, Peng, et al.
Pubblicazione: (2024)

Exploring the Benefits of Domain-Pretraining of Generative Large Language Models for Chemistry
di: Acharya, Anurag, et al.
Pubblicazione: (2024)

FASTopic: Pretrained Transformer is a Fast, Adaptive, Stable, and Transferable Topic Model
di: Wu, Xiaobao, et al.
Pubblicazione: (2024)

Multilingual Pretraining for Pixel Language Models
di: Kesen, Ilker, et al.
Pubblicazione: (2025)

S$^4$C: Speculative Sampling with Syntactic and Semantic Coherence for Efficient Inference of Large Language Models
di: He, Tao, et al.
Pubblicazione: (2025)

BhashaKritika: Building Synthetic Pretraining Data at Scale for Indic Languages
di: Manoj, Guduru, et al.
Pubblicazione: (2025)

Unsupervised Elicitation of Language Models
di: Wen, Jiaxin, et al.
Pubblicazione: (2025)

Text Understanding and Generation Using Transformer Models for Intelligent E-commerce Recommendations
di: Xiang, Yafei, et al.
Pubblicazione: (2024)

Learning Robust Named Entity Recognizers From Noisy Data With Retrieval Augmentation
di: Ai, Chaoyi, et al.
Pubblicazione: (2024)

Memory Decoder: A Pretrained, Plug-and-Play Memory for Large Language Models
di: Cao, Jiaqi, et al.
Pubblicazione: (2025)

KELPS: A Framework for Verified Multi-Language Autoformalization via Semantic-Syntactic Alignment
di: Zhang, Jiyao, et al.
Pubblicazione: (2025)

Multi-View Attention Syntactic Enhanced Graph Convolutional Network for Aspect-based Sentiment Analysis
di: Huang, Xiang, et al.
Pubblicazione: (2025)

On Linear Representations and Pretraining Data Frequency in Language Models
di: Merullo, Jack, et al.
Pubblicazione: (2025)

Drop Dropout on Single-Epoch Language Model Pretraining
di: Liu, Houjun, et al.
Pubblicazione: (2025)

Gender Encoding Patterns in Pretrained Language Model Representations
di: Zakizadeh, Mahdi, et al.
Pubblicazione: (2025)

Rethinking the Role of Text Complexity in Language Model Pretraining
di: Velasco, Dan John, et al.
Pubblicazione: (2025)

Syntactic and Semantic Control of Large Language Models via Sequential Monte Carlo
di: Loula, João, et al.
Pubblicazione: (2025)

Parrot Mind: Towards Explaining the Complex Task Reasoning of Pretrained Large Language Models with Template-Content Structure
di: Yang, Haotong, et al.
Pubblicazione: (2023)

Pruning as a Cooperative Game: Surrogate-Assisted Layer Contribution Estimation for Large Language Models
di: Ding, Xuan, et al.
Pubblicazione: (2026)

DAM: Dynamic Attention Mask for Long-Context Large Language Model Inference Acceleration
di: Zhang, Hanzhi, et al.
Pubblicazione: (2025)

Atoxia: Red-teaming Large Language Models with Target Toxic Answers
di: Du, Yuhao, et al.
Pubblicazione: (2024)

Code Pretraining Improves Entity Tracking Abilities of Language Models
di: Kim, Najoung, et al.
Pubblicazione: (2024)

Pretrained Generative Language Models as General Learning Frameworks for Sequence-Based Tasks
di: Fauber, Ben
Pubblicazione: (2024)