Imagen de Portada

Emergent inabilities? Inverse scaling over the course of pretraining

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Michaelov, James A., Bergen, Benjamin K.
Formato:	Preprint
Publicado:	2023
Materias:	Computation and Language
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Revenge of the Fallen? Recurrent Models Match Transformers at Predicting Human Language Comprehension Metrics
por: Michaelov, James A., et al.
Publicado: (2024)

Language Model Behavioral Phases are Consistent Across Architecture, Training Data, and Scale
por: Michaelov, James A., et al.
Publicado: (2025)

On the Acquisition of Shared Grammatical Representations in Bilingual Language Models
por: Arnett, Catherine, et al.
Publicado: (2025)

Not quite Sherlock Holmes: Language model predictions do not reliably differentiate impossible from improbable events
por: Michaelov, James A., et al.
Publicado: (2025)

Disaggregation Reveals Hidden Training Dynamics: The Case of Agreement Attraction
por: Michaelov, James A., et al.
Publicado: (2025)

N-gram-like Language Models Predict Reading Time Best
por: Michaelov, James A., et al.
Publicado: (2026)

Bigram Subnetworks: Mapping to Next Tokens in Transformer Language Models
por: Chang, Tyler A., et al.
Publicado: (2025)

Why do language models perform worse for morphologically complex languages?
por: Arnett, Catherine, et al.
Publicado: (2024)

Do Large Language Models Exhibit Spontaneous Rational Deception?
por: Taylor, Samuel M., et al.
Publicado: (2025)

How Open Must Language Models be to Enable Reliable Scientific Inference?
por: Michaelov, James A., et al.
Publicado: (2026)

Large Language Models Pass the Turing Test
por: Jones, Cameron R., et al.
Publicado: (2025)

Characterizing Learning Curves During Language Model Pre-Training: Learning, Forgetting, and Stability
por: Chang, Tyler A., et al.
Publicado: (2023)

A Bit of a Problem: Measurement Disparities in Dataset Sizes Across Languages
por: Arnett, Catherine, et al.
Publicado: (2024)

EVOKE: Emotion Vocabulary Of Korean and English
por: Jung, Yoonwon, et al.
Publicado: (2026)

Does GPT-4 pass the Turing test?
por: Jones, Cameron R., et al.
Publicado: (2023)

Synthetic bootstrapped pretraining
por: Yang, Zitong, et al.
Publicado: (2025)

Language Statistics and False Belief Reasoning: Evidence from 41 Open-Weight LMs
por: Trott, Sean, et al.
Publicado: (2026)

Goldfish: Monolingual Language Models for 350 Languages
por: Chang, Tyler A., et al.
Publicado: (2024)

Explaining and Mitigating Crosslingual Tokenizer Inequities
por: Arnett, Catherine, et al.
Publicado: (2025)

Synthetic continued pretraining
por: Yang, Zitong, et al.
Publicado: (2024)

Dissecting the Ullman Variations with a SCALPEL: Why do LLMs fail at Trivial Alterations to the False Belief Task?
por: Pi, Zhiqiang, et al.
Publicado: (2024)

GPT-4 is judged more human than humans in displaced and inverted Turing tests
por: Rathi, Ishika, et al.
Publicado: (2024)

The Book of Life approach: Enabling richness and scale for life course research
por: Verhagen, Mark D., et al.
Publicado: (2025)

Prompting from the bench: Large-scale pretraining is not sufficient to prepare LLMs for ordinary meaning analysis
por: Purushothama, Abhishek, et al.
Publicado: (2025)

Lies, Damned Lies, and Distributional Language Statistics: Persuasion and Deception with Large Language Models
por: Jones, Cameron R., et al.
Publicado: (2024)

TextGram: Towards a better domain-adaptive pretraining
por: Hiwarkhedkar, Sharayu, et al.
Publicado: (2024)

How far can bias go? Tracing bias from pretraining data to alignment
por: Thaler, Marion, et al.
Publicado: (2024)

BPDec: Unveiling the Potential of Masked Language Modeling Decoder in BERT pretraining
por: Liang, Wen, et al.
Publicado: (2024)

Do pretrained Transformers Learn In-Context by Gradient Descent?
por: Shen, Lingfeng, et al.
Publicado: (2023)

Analyzing the relationships between pretraining language, phonetic, tonal, and speaker information in self-supervised speech models
por: Gubian, Michele, et al.
Publicado: (2025)

Tgea: An error-annotated dataset and benchmark tasks for text generation from pretrained language models
por: He, Jie, et al.
Publicado: (2025)

MedicalBERT: enhancing biomedical natural language processing using pretrained BERT-based model
por: Reddy, K. Sahit, et al.
Publicado: (2025)

Empirical study of pretrained multilingual language models for zero-shot cross-lingual knowledge transfer in generation
por: Chirkova, Nadezhda, et al.
Publicado: (2023)

Proving membership in LLM pretraining data via data watermarks
por: Wei, Johnny Tian-Zheng, et al.
Publicado: (2024)

ks-pret-5m: a 5 million word, 12 million token kashmiri pretraining dataset
por: Malik, Haq Nawaz, et al.
Publicado: (2026)

[Call for Papers] The 2nd BabyLM Challenge: Sample-efficient pretraining on a developmentally plausible corpus
por: Choshen, Leshem, et al.
Publicado: (2024)

LLMs and people both learn to form conventions -- just not with each other
por: Jones, Cameron R., et al.
Publicado: (2026)

Emergent effects of scaling on the functional hierarchies within large language models
por: Bogdan, Paul C.
Publicado: (2025)

GLAP: General contrastive audio-text pretraining across domains and languages
por: Dinkel, Heinrich, et al.
Publicado: (2025)

The effectiveness of MAE pre-pretraining for billion-scale pretraining
por: Singh, Mannat, et al.
Publicado: (2023)