Cover Image

The geometry of BERT

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Bonino, Matteo, Ghione, Giorgia, Cirrincione, Giansalvo
Format:	Preprint
Published:	2025
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Rank, Head-Channel Non-Identifiability, and Symmetry Breaking: A Precise Analysis of Representational Collapse in Transformers
by: Cirrincione, Giansalvo
Published: (2026)

On the Geometry of Positional Encodings in Transformers
by: Cirrincione, Giansalvo
Published: (2026)

DDCL: Deep Dual Competitive Learning: A Differentiable End-to-End Framework for Unsupervised Prototype-Based Representation Learning
by: Cirrincione, Giansalvo
Published: (2026)

DDCL-INCRT: A Self-Organising Transformer with Hierarchical Prototype Structure (Theoretical Foundations)
by: Cirrincione, Giansalvo
Published: (2026)

INCRT: An Incremental Transformer That Determines Its Own Architecture
by: Cirrincione, Giansalvo
Published: (2026)

Hierarchical Kernel Transformer: Multi-Scale Attention with an Information-Theoretic Approximation Analysis
by: Cirrincione, Giansalvo
Published: (2026)

Learned Lyapunov Shielding for Adaptive Control
by: Cirrincione, Giansalvo, et al.
Published: (2026)

Temporal Attention for Adaptive Control of Euler-Lagrange Systems with Unobservable Memory
by: Cirrincione, Giansalvo, et al.
Published: (2026)

Collapse-Free Prototype Readout Layer for Transformer Encoders
by: Cirrincione, Giansalvo, et al.
Published: (2026)

TimesBERT: A BERT-Style Foundation Model for Time Series Understanding
by: Zhang, Haoran, et al.
Published: (2025)

GoBERT: Gene Ontology Graph Informed BERT for Universal Gene Function Prediction
by: Miao, Yuwei, et al.
Published: (2025)

Task structure and nonlinearity jointly determine learned representational geometry
by: Alleman, Matteo, et al.
Published: (2024)

FlowBERT: Prompt-tuned BERT for variable flow field prediction
by: Zou, Weihao, et al.
Published: (2025)

BarcodeBERT: Transformers for Biodiversity Analysis
by: Arias, Pablo Millan, et al.
Published: (2023)

Learning Complex Physical Regimes via Coverage-oriented Uncertainty Quantification: An application to the Critical Heat Flux
by: Cazzola, Michele, et al.
Published: (2026)

On Feasible Rewards in Multi-Agent Inverse Reinforcement Learning
by: Freihaut, Till, et al.
Published: (2024)

MaxPoolBERT: Enhancing BERT Classification via Layer- and Token-Wise Aggregation
by: Behrendt, Maike, et al.
Published: (2025)

DE$^3$-BERT: Distance-Enhanced Early Exiting for BERT based on Prototypical Networks
by: He, Jianing, et al.
Published: (2024)

An investigation of structures responsible for gender bias in BERT and DistilBERT
by: Leteno, Thibaud, et al.
Published: (2024)

Spam Detection Using BERT
by: Sahmoud, Thaer, et al.
Published: (2022)

MenakBERT -- Hebrew Diacriticizer
by: Cohen, Ido, et al.
Published: (2024)

Empowering Interdisciplinary Research with BERT-Based Models: An Approach Through SciBERT-CNN with Topic Modeling
by: Likhareva, Darya, et al.
Published: (2024)

TREB: a BERT attempt for imputing tabular data imputation
by: Wang, Shuyue, et al.
Published: (2024)

Privacy-Preserving Inference for Quantized BERT Models
by: Lu, Tianpei, et al.
Published: (2025)

Energy and Carbon Considerations of Fine-Tuning BERT
by: Wang, Xiaorong, et al.
Published: (2023)

MedicalBERT: enhancing biomedical natural language processing using pretrained BERT-based model
by: Reddy, K. Sahit, et al.
Published: (2025)

LegalPro-BERT: Classification of Legal Provisions by fine-tuning BERT Large Language Model
by: Tewari, Amit
Published: (2024)

MelHuBERT: A simplified HuBERT on Mel spectrograms
by: Lin, Tzu-Quan, et al.
Published: (2022)

RoboBERT: An End-to-end Multimodal Robotic Manipulation Model
by: Wang, Sicheng, et al.
Published: (2025)

From BERT to Qwen: Hate Detection across architectures
by: Mon, Ariadna, et al.
Published: (2025)

A Contextualized BERT model for Knowledge Graph Completion
by: Gul, Haji, et al.
Published: (2024)

Weight-Inherited Distillation for Task-Agnostic BERT Compression
by: Wu, Taiqiang, et al.
Published: (2023)

GottBERT: a pure German Language Model
by: Scheible, Raphael, et al.
Published: (2020)

MCUBERT: Memory-Efficient BERT Inference on Commodity Microcontrollers
by: Yang, Zebin, et al.
Published: (2024)

Sarcasm Detection in Tweets with BERT and GloVe Embeddings
by: Khatri, Akshay, et al.
Published: (2020)

CEEBERT: Cross-Domain Inference in Early Exit BERT
by: Bajpai, Divya Jyoti, et al.
Published: (2024)

BERT Learns (and Teaches) Chemistry
by: Payne, Josh, et al.
Published: (2020)

HELM-BERT: A Transformer for Medium-sized Peptide Property Prediction
by: Lee, Seungeon, et al.
Published: (2025)

Modules 1 and 2 of The Orchard Framework - May 2026 Iteration - Mathematical Formalism - with minimal alignment geometry
by: ASHER, KIMBERLEY LAVERNE
Published: (2026)

Enhancing TinyBERT for Financial Sentiment Analysis Using GPT-Augmented FinBERT Distillation
by: Thomas, Graison Jos
Published: (2024)