Imagen de Portada

DA-Cramming: Enhancing Cost-Effective Language Model Pretraining with Dependency Agreement Integration

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Kuo, Martin, Zhang, Jianyi, Li, Dongting, Chen, Yiran
Formato:	Preprint
Publicado:	2023
Materias:	Computation and Language Artificial Intelligence
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

An Enhanced Prompt-Based LLM Reasoning Scheme via Knowledge Graph-Integrated Collaboration
por: Li, Yihao, et al.
Publicado: (2024)

SLED: Self Logits Evolution Decoding for Improving Factuality in Large Language Models
por: Zhang, Jianyi, et al.
Publicado: (2024)

MLLM-LLaVA-FL: Multimodal Large Language Model Assisted Federated Learning
por: Zhang, Jianyi, et al.
Publicado: (2024)

Cequel: Cost-Effective Querying of Large Language Models for Text Clustering
por: Wang, Hongtao, et al.
Publicado: (2025)

On Leveraging Large Language Models for Enhancing Entity Resolution: A Cost-efficient Approach
por: Li, Huahang, et al.
Publicado: (2024)

Enhancing Coreference Resolution with Pretrained Language Models: Bridging the Gap Between Syntax and Semantics
por: Liu, Xingzu, et al.
Publicado: (2025)

Dependency Transformer Grammars: Integrating Dependency Structures into Transformer Language Models
por: Zhao, Yida, et al.
Publicado: (2024)

CCI4.0: A Bilingual Pretraining Dataset for Enhancing Reasoning in Large Language Models
por: Liu, Guang, et al.
Publicado: (2025)

T2S-Bench & Structure-of-Thought: Benchmarking and Prompting Comprehensive Text-to-Structure Reasoning
por: Wang, Qinsi, et al.
Publicado: (2026)

Infant Agent: A Tool-Integrated, Logic-Driven Agent with Cost-Effective API Usage
por: Lei, Bin, et al.
Publicado: (2024)

Multilingual Pretraining for Pixel Language Models
por: Kesen, Ilker, et al.
Publicado: (2025)

PhayaThaiBERT: Enhancing a Pretrained Thai Language Model with Unassimilated Loanwords
por: Sriwirote, Panyut, et al.
Publicado: (2023)

How do Large Language Models Handle Multilingualism?
por: Zhao, Yiran, et al.
Publicado: (2024)

IDEA Prune: An Integrated Enlarge-and-Prune Pipeline in Generative Language Model Pretraining
por: Li, Yixiao, et al.
Publicado: (2025)

Refining Salience-Aware Sparse Fine-Tuning Strategies for Language Models
por: Liu, Xinxin, et al.
Publicado: (2024)

PonderLM: Pretraining Language Models to Ponder in Continuous Space
por: Zeng, Boyi, et al.
Publicado: (2025)

MASS: Mathematical Data Selection via Skill Graphs for Pretraining Large Language Models
por: Li, Jiazheng, et al.
Publicado: (2025)

IKnow: Instruction-Knowledge-Aware Continual Pretraining for Effective Domain Adaptation
por: Zhang, Tianyi, et al.
Publicado: (2025)

Train Once, Answer All: Many Pretraining Experiments for the Cost of One
por: Bordt, Sebastian, et al.
Publicado: (2025)

Just Ask One More Time! Self-Agreement Improves Reasoning of Language Models in (Almost) All Scenarios
por: Lin, Lei, et al.
Publicado: (2023)

CodePMP: Scalable Preference Model Pretraining for Large Language Model Reasoning
por: Yu, Huimu, et al.
Publicado: (2024)

On Linear Representations and Pretraining Data Frequency in Language Models
por: Merullo, Jack, et al.
Publicado: (2025)

Drop Dropout on Single-Epoch Language Model Pretraining
por: Liu, Houjun, et al.
Publicado: (2025)

Gender Encoding Patterns in Pretrained Language Model Representations
por: Zakizadeh, Mahdi, et al.
Publicado: (2025)

Rethinking the Role of Text Complexity in Language Model Pretraining
por: Velasco, Dan John, et al.
Publicado: (2025)

The Emergence of Abstract Thought in Large Language Models Beyond Any Language
por: Chen, Yuxin, et al.
Publicado: (2025)

Chinese Tiny LLM: Pretraining a Chinese-Centric Large Language Model
por: Du, Xinrun, et al.
Publicado: (2024)

Exploring Polyglot Harmony: On Multilingual Data Allocation for Large Language Models Pretraining
por: Guo, Ping, et al.
Publicado: (2025)

Small Language Model Makes an Effective Long Text Extractor
por: Chen, Yelin, et al.
Publicado: (2025)

ORBIT: Cost-Effective Dataset Curation for Large Language Model Domain Adaptation with an Astronomy Case Study
por: Modesitt, Eric, et al.
Publicado: (2024)

Beyond Random Sampling: Efficient Language Model Pretraining via Curriculum Learning
por: Zhang, Yang, et al.
Publicado: (2025)

Psychological Assessments with Large Language Models: A Privacy-Focused and Cost-Effective Approach
por: Blanco-Cuaresma, Sergi
Publicado: (2024)

Memory Decoder: A Pretrained, Plug-and-Play Memory for Large Language Models
por: Cao, Jiaqi, et al.
Publicado: (2025)

Code Pretraining Improves Entity Tracking Abilities of Language Models
por: Kim, Najoung, et al.
Publicado: (2024)

Emergent Abilities of Large Language Models under Continued Pretraining for Language Adaptation
por: Elhady, Ahmed, et al.
Publicado: (2025)

Deciphering the Impact of Pretraining Data on Large Language Models through Machine Unlearning
por: Zhao, Yang, et al.
Publicado: (2024)

Cost-efficient Knowledge-based Question Answering with Large Language Models
por: Dong, Junnan, et al.
Publicado: (2024)

Segment Policy Optimization: Effective Segment-Level Credit Assignment in RL for Large Language Models
por: Guo, Yiran, et al.
Publicado: (2025)

Polyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation
por: Dekoninck, Jasper, et al.
Publicado: (2024)

GLoRE: Evaluating Logical Reasoning of Large Language Models
por: liu, Hanmeng, et al.
Publicado: (2023)