Imagen de Portada

MosaicBERT: A Bidirectional Encoder Optimized for Fast Pretraining

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Portes, Jacob, Trott, Alex, Havens, Sam, King, Daniel, Venigalla, Abhinav, Nadeem, Moin, Sardana, Nikhil, Khudia, Daya, Frankle, Jonathan
Formato:	Preprint
Publicado:	2023
Materias:	Computation and Language Machine Learning
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Beyond Chinchilla-Optimal: Accounting for Inference in Language Model Scaling Laws
por: Sardana, Nikhil, et al.
Publicado: (2023)

Long Context RAG Performance of Large Language Models
por: Leng, Quinn, et al.
Publicado: (2024)

LoRA Learns Less and Forgets Less
por: Biderman, Dan, et al.
Publicado: (2024)

Is the GPU Half-Empty or Half-Full? Practical Scheduling Techniques for LLMs
por: Kossmann, Ferdi, et al.
Publicado: (2024)

RexBERT: Context Specialized Bidirectional Encoders for E-commerce
por: Bajaj, Rahul, et al.
Publicado: (2026)

Dynamic Masking Rate Schedules for MLM Pretraining
por: Ankner, Zachary, et al.
Publicado: (2023)

m3BERT: A Modern, Multi-lingual, Matryoshka Bidirectional Encoder
por: Wang, Yaoxiang, et al.
Publicado: (2026)

BioMedLM: A 2.7B Parameter Language Model Trained On Biomedical Text
por: Bolton, Elliot, et al.
Publicado: (2024)

Dataset Difficulty and the Role of Inductive Bias
por: Kwok, Devin, et al.
Publicado: (2024)

Sparse Upcycling: Inference Inefficient Finetuning
por: Doubov, Sasha, et al.
Publicado: (2024)

Retrieval Capabilities of Large Language Models Scale with Pretraining FLOPs
por: Portes, Jacob, et al.
Publicado: (2025)

FinBERT2: A Specialized Bidirectional Encoder for Bridging the Gap in Finance-Specific Deployment of Large Language Models
por: Xu, Xuan, et al.
Publicado: (2025)

When Can I Put down My Pen? If I Put down My Pen, Do I Get a Secret Decoder Ring?
por: Frankle, Raymond A.
Publicado: (1992)

Black Television Travels
por: Havens, Timothy
Publicado: (2024)

MIRVs and Money
por: Havens, Shirley
Publicado: (1969)

Libraries: Drawn to Knowledge.
por: Havens, Kevin
Publicado: (2003)

The Case for Aging: A Bibliographic Essay.
por: Havens, Carolyn
Publicado: (1988)

Bug Destiny Prediction in Large Open-Source Software Repositories through Sentiment Analysis and BERT Topic Modeling
por: Pope, Sophie C., et al.
Publicado: (2025)

Fast and Adaptive Bulk Loading of Multidimensional Points
por: Moti, Moin Hussain, et al.
Publicado: (2024)

ManufactuBERT: Efficient Continual Pretraining for Manufacturing
por: Armingaud, Robin, et al.
Publicado: (2025)

Patent Language Model Pretraining with ModernBERT
por: Yousefiramandi, Amirhossein, et al.
Publicado: (2025)

KARL: Knowledge Agents via Reinforcement Learning
por: Chang, Jonathan D., et al.
Publicado: (2026)

Real-Time Evaluation Models for RAG: Who Detects Hallucinations Best?
por: Sardana, Ashish
Publicado: (2025)

Poemas de Zuca Sardana
por: Zuca Sardana
Publicado: (2006)

EuroBERT: Scaling Multilingual Encoders for European Languages
por: Boizard, Nicolas, et al.
Publicado: (2025)

Non-Determinism and the Lawlessness of Machine Learning Code
por: Cooper, A. Feder, et al.
Publicado: (2022)

PolyBERT: Fine-Tuned Poly Encoder BERT-Based Model for Word Sense Disambiguation
por: Xia, Linhan, et al.
Publicado: (2025)

Pretraining and Benchmarking Modern Encoders for Latvian
por: Znotins, Arturs
Publicado: (2026)

The Targeted Standard Siren Cosmology with Pulsar Timing Arrays
por: Sardana, Shubhit, et al.
Publicado: (2026)

Large Print in Focus.
por: Havens, Shirley E.
Publicado: (1987)

DocFetch - Towards Generating Software Documentation from Multiple Software Artifacts
por: Venigalla, Akhila Sri Manasa, et al.
Publicado: (2025)

Is There a Correlation Between Readme Content and Project Meta‐Characteristics?
por: Akhila Sri Manasa Venigalla, et al.
Publicado: (2024)

A Semiautomated Approach for Detecting Ambiguities in Software Requirements Using SpanBERT and Named Entity Recognition
por: Fiza Talha, et al.
Publicado: (2025)

OrQstrator: An AI-Powered Framework for Advanced Quantum Circuit Optimization
por: Baird, Laura, et al.
Publicado: (2025)

TocBERT: Medical Document Structure Extraction Using Bidirectional Transformers
por: Saleh, Majd, et al.
Publicado: (2024)

Renaissance: Investigating the Pretraining of Vision-Language Encoders
por: Fields, Clayton, et al.
Publicado: (2024)

A Bayesian Optimization-Based AutoML Framework for Non-Intrusive Load Monitoring
por: Siavash, Nazanin, et al.
Publicado: (2026)

FarSSiBERT: A Novel Transformer-based Model for Semantic Similarity Measurement of Persian Social Networks Informal Texts
por: Sadjadi, Seyed Mojtaba, et al.
Publicado: (2024)

Chapter 13 A Change of Heart: Animality, Power, and Black Posthuman Enhancement in Malorie Blackman’s Pig-Heart Boy
por: Trott, Emma
Publicado: (2024)

Toward a Theory of Generalizability in LLM Mechanistic Interpretability Research
por: Trott, Sean
Publicado: (2025)