Imagen de Portada

The geometry of BERT

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Bonino, Matteo, Ghione, Giorgia, Cirrincione, Giansalvo
Formato:	Preprint
Publicado:	2025
Materias:	Machine Learning
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Rank, Head-Channel Non-Identifiability, and Symmetry Breaking: A Precise Analysis of Representational Collapse in Transformers
por: Cirrincione, Giansalvo
Publicado: (2026)

On the Geometry of Positional Encodings in Transformers
por: Cirrincione, Giansalvo
Publicado: (2026)

DDCL: Deep Dual Competitive Learning: A Differentiable End-to-End Framework for Unsupervised Prototype-Based Representation Learning
por: Cirrincione, Giansalvo
Publicado: (2026)

DDCL-INCRT: A Self-Organising Transformer with Hierarchical Prototype Structure (Theoretical Foundations)
por: Cirrincione, Giansalvo
Publicado: (2026)

INCRT: An Incremental Transformer That Determines Its Own Architecture
por: Cirrincione, Giansalvo
Publicado: (2026)

Hierarchical Kernel Transformer: Multi-Scale Attention with an Information-Theoretic Approximation Analysis
por: Cirrincione, Giansalvo
Publicado: (2026)

Learned Lyapunov Shielding for Adaptive Control
por: Cirrincione, Giansalvo, et al.
Publicado: (2026)

Temporal Attention for Adaptive Control of Euler-Lagrange Systems with Unobservable Memory
por: Cirrincione, Giansalvo, et al.
Publicado: (2026)

Collapse-Free Prototype Readout Layer for Transformer Encoders
por: Cirrincione, Giansalvo, et al.
Publicado: (2026)

TimesBERT: A BERT-Style Foundation Model for Time Series Understanding
por: Zhang, Haoran, et al.
Publicado: (2025)

GoBERT: Gene Ontology Graph Informed BERT for Universal Gene Function Prediction
por: Miao, Yuwei, et al.
Publicado: (2025)

Task structure and nonlinearity jointly determine learned representational geometry
por: Alleman, Matteo, et al.
Publicado: (2024)

FlowBERT: Prompt-tuned BERT for variable flow field prediction
por: Zou, Weihao, et al.
Publicado: (2025)

BarcodeBERT: Transformers for Biodiversity Analysis
por: Arias, Pablo Millan, et al.
Publicado: (2023)

Learning Complex Physical Regimes via Coverage-oriented Uncertainty Quantification: An application to the Critical Heat Flux
por: Cazzola, Michele, et al.
Publicado: (2026)

On Feasible Rewards in Multi-Agent Inverse Reinforcement Learning
por: Freihaut, Till, et al.
Publicado: (2024)

MaxPoolBERT: Enhancing BERT Classification via Layer- and Token-Wise Aggregation
por: Behrendt, Maike, et al.
Publicado: (2025)

DE$^3$-BERT: Distance-Enhanced Early Exiting for BERT based on Prototypical Networks
por: He, Jianing, et al.
Publicado: (2024)

An investigation of structures responsible for gender bias in BERT and DistilBERT
por: Leteno, Thibaud, et al.
Publicado: (2024)

Spam Detection Using BERT
por: Sahmoud, Thaer, et al.
Publicado: (2022)

MenakBERT -- Hebrew Diacriticizer
por: Cohen, Ido, et al.
Publicado: (2024)

Empowering Interdisciplinary Research with BERT-Based Models: An Approach Through SciBERT-CNN with Topic Modeling
por: Likhareva, Darya, et al.
Publicado: (2024)

TREB: a BERT attempt for imputing tabular data imputation
por: Wang, Shuyue, et al.
Publicado: (2024)

Privacy-Preserving Inference for Quantized BERT Models
por: Lu, Tianpei, et al.
Publicado: (2025)

Energy and Carbon Considerations of Fine-Tuning BERT
por: Wang, Xiaorong, et al.
Publicado: (2023)

MedicalBERT: enhancing biomedical natural language processing using pretrained BERT-based model
por: Reddy, K. Sahit, et al.
Publicado: (2025)

LegalPro-BERT: Classification of Legal Provisions by fine-tuning BERT Large Language Model
por: Tewari, Amit
Publicado: (2024)

MelHuBERT: A simplified HuBERT on Mel spectrograms
por: Lin, Tzu-Quan, et al.
Publicado: (2022)

RoboBERT: An End-to-end Multimodal Robotic Manipulation Model
por: Wang, Sicheng, et al.
Publicado: (2025)

From BERT to Qwen: Hate Detection across architectures
por: Mon, Ariadna, et al.
Publicado: (2025)

A Contextualized BERT model for Knowledge Graph Completion
por: Gul, Haji, et al.
Publicado: (2024)

Weight-Inherited Distillation for Task-Agnostic BERT Compression
por: Wu, Taiqiang, et al.
Publicado: (2023)

GottBERT: a pure German Language Model
por: Scheible, Raphael, et al.
Publicado: (2020)

MCUBERT: Memory-Efficient BERT Inference on Commodity Microcontrollers
por: Yang, Zebin, et al.
Publicado: (2024)

Sarcasm Detection in Tweets with BERT and GloVe Embeddings
por: Khatri, Akshay, et al.
Publicado: (2020)

CEEBERT: Cross-Domain Inference in Early Exit BERT
por: Bajpai, Divya Jyoti, et al.
Publicado: (2024)

BERT Learns (and Teaches) Chemistry
por: Payne, Josh, et al.
Publicado: (2020)

HELM-BERT: A Transformer for Medium-sized Peptide Property Prediction
por: Lee, Seungeon, et al.
Publicado: (2025)

Modules 1 and 2 of The Orchard Framework - May 2026 Iteration - Mathematical Formalism - with minimal alignment geometry
por: ASHER, KIMBERLEY LAVERNE
Publicado: (2026)

Enhancing TinyBERT for Financial Sentiment Analysis Using GPT-Augmented FinBERT Distillation
por: Thomas, Graison Jos
Publicado: (2024)