Cover Image

Better Models, Faster Training: Sigmoid Attention for single-cell Foundation Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Sadashivaiah, Vijay, Dasoulas, Georgios, Mueller, Judith, Ghosh, Soumya
Format:	Preprint
Published:	2026
Subjects:	Machine Learning Quantitative Methods
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

TEDDY: A Family Of Foundation Models For Understanding Single Cell Biology
by: Chevalier, Alexis, et al.
Published: (2025)

Geometric Self-Supervised Pretraining on 3D Protein Structures using Subgraphs
by: Chatzianastasis, Michail, et al.
Published: (2024)

Better Protein Function Prediction by Modeling Survivorship Bias
by: Chao, Zhongmou, et al.
Published: (2026)

Feature selection revisited in the single-cell era
by: Yang, Pengyi, et al.
Published: (2021)

Uncertainty-Guided Model Selection for Tabular Foundation Models in Biomolecule Efficacy Prediction
by: Li, Jie, et al.
Published: (2025)

Foundation Models for Environmental Science: A Survey of Emerging Frontiers
by: Yu, Runlong, et al.
Published: (2025)

ViroBench: Benchmarking Nucleotide Foundation Models on Viral Genomics Tasks
by: Ye, Dongxin, et al.
Published: (2026)

Pearl: A Foundation Model for Placing Every Atom in the Right Location
by: Genesis Research Team, et al.
Published: (2025)

SMILES-Mamba: Chemical Mamba Foundation Models for Drug ADMET Prediction
by: Xu, Bohao, et al.
Published: (2024)

EXAONE Path 2.5: Pathology Foundation Model with Multi-Omics Alignment
by: Yun, Juseung, et al.
Published: (2025)

Efficient Data Selection for Training Genomic Perturbation Models
by: Panagopoulos, George, et al.
Published: (2025)

Hierarchical novel class discovery for single-cell transcriptomic profiles
by: Senoussi, Malek, et al.
Published: (2024)

Multitask Multimodal Fusion with Tabular Foundation Models for Peak and Durability Prediction of Pertussis Booster Response
by: Sitani, Divya
Published: (2026)

Efficient Fine-Tuning of Single-Cell Foundation Models Enables Zero-Shot Molecular Perturbation Prediction
by: Maleki, Sepideh, et al.
Published: (2024)

Can Molecular Foundation Models Know What They Don't Know? A Simple Remedy with Preference Optimization
by: He, Langzhou, et al.
Published: (2025)

Integrating Dynamical Systems Learning with Foundational Models: A Meta-Evolutionary AI Framework for Clinical Trials
by: Geraci, Joseph, et al.
Published: (2025)

Parameter-free representations outperform single-cell foundation models on downstream benchmarks
by: Souza, Huan, et al.
Published: (2026)

LAMP-PRo: Label-aware Attention for Multi-label Prediction of DNA- and RNA-binding Proteins using Protein Language Models
by: Ghosh, Nimisha, et al.
Published: (2025)

scDiffusion: conditional generation of high-quality single-cell data using diffusion model
by: Luo, Erpai, et al.
Published: (2024)

Dual Triangle Attention: Effective Bidirectional Attention Without Positional Embeddings
by: Hallee, Logan, et al.
Published: (2026)

The Next Layer: Augmenting Foundation Models with Structure-Preserving and Attention-Guided Learning for Local Patches to Global Context Awareness in Computational Pathology
by: Waqas, Muhammad, et al.
Published: (2025)

scMRDR: A scalable and flexible framework for unpaired single-cell multi-omics data integration
by: Sun, Jianle, et al.
Published: (2025)

Integrating Genomics into Multimodal EHR Foundation Models
by: Amar, Jonathan, et al.
Published: (2025)

Towards a Foundation Model for Brain Age Prediction using coVariance Neural Networks
by: Sihag, Saurabh, et al.
Published: (2024)

The cell as a token: high-dimensional geometry in language models and cell embeddings
by: Gilpin, William
Published: (2025)

What Topological and Geometric Structure Do Biological Foundation Models Learn? Evidence from 141 Hypotheses
by: Kendiukhov, Ihor
Published: (2026)

Predicting Side Effect of Drug Molecules using Recurrent Neural Networks
by: Beaudoin, Collin, et al.
Published: (2023)

GramSeq-DTA: A grammar-based drug-target affinity prediction approach fusing gene expression information
by: Debnath, Kusal, et al.
Published: (2024)

Informing Acquisition Functions via Foundation Models for Molecular Discovery
by: Chen, Qi, et al.
Published: (2025)

Pretraining a Foundation Model for Small-Molecule Natural Products
by: Ding, Yuheng, et al.
Published: (2025)

Improving Prediction of Need for Mechanical Ventilation using Cross-Attention
by: Mohanty, Anwesh, et al.
Published: (2024)

Random Matrix Theory-guided sparse PCA for single-cell RNA-seq data
by: Chardès, Victor
Published: (2025)

An efficient heuristic for geometric analysis of cell deformations
by: Soto, Yaima Paz, et al.
Published: (2026)

Attention-Based Multimodal Survival Prediction with Cross-Modal Bilinear Fusion
by: Keshvarikhojasteh, Hassan, et al.
Published: (2026)

Molecular Odor Prediction Based on Multi-Feature Graph Attention Networks
by: Xie, HongXin, et al.
Published: (2025)

Training Compute-Optimal Protein Language Models
by: Cheng, Xingyi, et al.
Published: (2024)

BioArc: Discovering Optimal Neural Architectures for Biological Foundation Models
by: Fang, Yi, et al.
Published: (2025)

ATTNSOM: Learning Cross-Isoform Attention for Cytochrome P450 Site-of-Metabolism
by: Kim, Hajung, et al.
Published: (2026)

CINNAMON: A hybrid approach to change point detection and parameter estimation in single-particle tracking data
by: Malinowski, Jakub, et al.
Published: (2025)

The Geometric Alignment Tax: Tokenization vs. Continuous Geometry in Scientific Foundation Models
by: Raju, Prashant C.
Published: (2026)