Copertina

A distributional simplicity bias in the learning dynamics of transformers

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Rende, Riccardo, Gerace, Federica, Laio, Alessandro, Goldt, Sebastian
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Computation and Language
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Mapping of attention mechanisms to a generalized Potts model
di: Rende, Riccardo, et al.
Pubblicazione: (2023)

A quantitative analysis of semantic information in deep representations of text and images
di: Acevedo, Santiago, et al.
Pubblicazione: (2025)

Are queries and keys always relevant? A case study on Transformer wave functions
di: Rende, Riccardo, et al.
Pubblicazione: (2024)

Learning from higher-order statistics, efficiently: hypothesis tests, random features, and neural networks
di: Székely, Eszter, et al.
Pubblicazione: (2023)

Fine-tuning Neural Network Quantum States
di: Rende, Riccardo, et al.
Pubblicazione: (2024)

A simple linear algebra identity to optimize Large-Scale Neural Network Quantum States
di: Rende, Riccardo, et al.
Pubblicazione: (2023)

Transformer Wave Function for two dimensional frustrated magnets: emergence of a Spin-Liquid Phase in the Shastry-Sutherland Model
di: Viteritti, Luciano Loris, et al.
Pubblicazione: (2023)

Emergence of a High-Dimensional Abstraction Phase in Language Transformers
di: Cheng, Emily, et al.
Pubblicazione: (2024)

Foundation Neural-Networks Quantum States as a Unified Ansatz for Multiple Hamiltonians
di: Rende, Riccardo, et al.
Pubblicazione: (2025)

Humans and transformer LMs: Abstraction drives language learning
di: Jian, Jasper, et al.
Pubblicazione: (2026)

A Fourier perspective on the learning dynamics of neural networks: from sample complexities to mechanistic insights
di: Ricci, Fabiola, et al.
Pubblicazione: (2026)

Two failure modes of deep transformers and how to avoid them: a unified theory of signal propagation at initialisation
di: Giorlandino, Alessio, et al.
Pubblicazione: (2025)

DIETA: A Decoder-only transformer-based model for Italian-English machine TrAnslation
di: Kasela, Pranav, et al.
Pubblicazione: (2026)

Statistical mechanics of transfer learning in fully-connected networks in the proportional limit
di: Ingrosso, Alessandro, et al.
Pubblicazione: (2024)

How far can bias go? Tracing bias from pretraining data to alignment
di: Thaler, Marion, et al.
Pubblicazione: (2024)

Locality enhanced dynamic biasing and sampling strategies for contextual ASR
di: Jalal, Md Asif, et al.
Pubblicazione: (2024)

Preference learning in shades of gray: Interpretable and bias-aware reward modeling for human preferences
di: Oprea, Simona-Vasilica, et al.
Pubblicazione: (2026)

Sliding down the stairs: how correlated latent variables accelerate learning with neural networks
di: Bardone, Lorenzo, et al.
Pubblicazione: (2024)

Actuation without production bias
di: Kirby, James, et al.
Pubblicazione: (2024)

Unsupervised Domain Shift Detection with Interpretable Subspace Attribution
di: Springer, Sebastian, et al.
Pubblicazione: (2026)

To Bias or Not to Bias: Detecting bias in News with bias-detector
di: Ghosh, Himel, et al.
Pubblicazione: (2025)

De-biased Multimodal Electrocardiogram Analysis
di: Li, Haitao, et al.
Pubblicazione: (2024)

Anthropocentric bias in language model evaluation
di: Millière, Raphaël, et al.
Pubblicazione: (2024)

Large Language Model (LLM) AI text generation detection based on transformer deep learning algorithm
di: Mo, Yuhong, et al.
Pubblicazione: (2024)

Downstream bias mitigation is all you need
di: Baksi, Arkadeep, et al.
Pubblicazione: (2024)

Effect of dimensionality change on the bias of word embeddings
di: Rai, Rohit Raj, et al.
Pubblicazione: (2023)

Emergence of a phonological bias in ChatGPT
di: Toro, Juan Manuel
Pubblicazione: (2023)

Large Language Models are biased to overestimate profoundness
di: Herrera-Berg, Eugenio, et al.
Pubblicazione: (2023)

The Biased Samaritan: LLM biases in Perceived Kindness
di: Fagan, Jack H, et al.
Pubblicazione: (2025)

Meaningless is better: hashing bias-inducing words in LLM prompts improves performance in logical reasoning and statistical learning
di: Chadimová, Milena, et al.
Pubblicazione: (2024)

Exploring simplicity bias in 1D dynamical systems
di: Dingle, Kamaludin, et al.
Pubblicazione: (2024)

Word Alignment-Based Evaluation of Uniform Meaning Representations
di: Zeman, Daniel, et al.
Pubblicazione: (2026)

Signal-to-Noise Ratio and Sample Size Govern Representational Alignment in Neural Networks
di: Umar, Ali Hussaini, et al.
Pubblicazione: (2026)

A database to support the evaluation of gender biases in GPT-4o output
di: Mehner, Luise, et al.
Pubblicazione: (2025)

'Neural howlround' in large language models: a self-reinforcing bias phenomenon, and a dynamic attenuation solution
di: Drake, Seth
Pubblicazione: (2025)

Discovering alternative solutions beyond the simplicity bias in recurrent neural networks
di: Qian, William, et al.
Pubblicazione: (2025)

Addressing cognitive bias in medical language models
di: Schmidgall, Samuel, et al.
Pubblicazione: (2024)

Semantic Properties of cosine based bias scores for word embeddings
di: Schröder, Sarah, et al.
Pubblicazione: (2024)

The high dimensional psychological profile and cultural bias of ChatGPT
di: Yuan, Hang, et al.
Pubblicazione: (2024)

BIPOLAR: Polarization-based granular framework for LLM bias evaluation
di: Pavlíček, Martin, et al.
Pubblicazione: (2025)