Masked Autoencoders for Microscopy are Scalable Learners of Cellular Biology
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913868246155264 |
|---|---|
| author | Kraus, Oren Kenyon-Dean, Kian Saberian, Saber Fallah, Maryam McLean, Peter Leung, Jess Sharma, Vasudev Khan, Ayla Balakrishnan, Jia Celik, Safiye Beaini, Dominique Sypetkowski, Maciej Cheng, Chi Vicky Morse, Kristen Makes, Maureen Mabey, Ben Earnshaw, Berton |
| author_facet | Kraus, Oren Kenyon-Dean, Kian Saberian, Saber Fallah, Maryam McLean, Peter Leung, Jess Sharma, Vasudev Khan, Ayla Balakrishnan, Jia Celik, Safiye Beaini, Dominique Sypetkowski, Maciej Cheng, Chi Vicky Morse, Kristen Makes, Maureen Mabey, Ben Earnshaw, Berton |
| contents | Featurizing microscopy images for use in biological research remains a significant challenge, especially for large-scale experiments spanning millions of images. This work explores the scaling properties of weakly supervised classifiers and self-supervised masked autoencoders (MAEs) when training with increasingly larger model backbones and microscopy datasets. Our results show that ViT-based MAEs outperform weakly supervised classifiers on a variety of tasks, achieving as much as a 11.5% relative improvement when recalling known biological relationships curated from public databases. Additionally, we develop a new channel-agnostic MAE architecture (CA-MAE) that allows for inputting images of different numbers and orders of channels at inference time. We demonstrate that CA-MAEs effectively generalize by inferring and evaluating on a microscopy image dataset (JUMP-CP) generated under different experimental conditions with a different channel structure than our pretraining data (RPI-93M). Our findings motivate continued research into scaling self-supervised learning on microscopy data in order to create powerful foundation models of cellular biology that have the potential to catalyze advancements in drug discovery and beyond. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_10242 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Masked Autoencoders for Microscopy are Scalable Learners of Cellular Biology Kraus, Oren Kenyon-Dean, Kian Saberian, Saber Fallah, Maryam McLean, Peter Leung, Jess Sharma, Vasudev Khan, Ayla Balakrishnan, Jia Celik, Safiye Beaini, Dominique Sypetkowski, Maciej Cheng, Chi Vicky Morse, Kristen Makes, Maureen Mabey, Ben Earnshaw, Berton Computer Vision and Pattern Recognition Artificial Intelligence Machine Learning Featurizing microscopy images for use in biological research remains a significant challenge, especially for large-scale experiments spanning millions of images. This work explores the scaling properties of weakly supervised classifiers and self-supervised masked autoencoders (MAEs) when training with increasingly larger model backbones and microscopy datasets. Our results show that ViT-based MAEs outperform weakly supervised classifiers on a variety of tasks, achieving as much as a 11.5% relative improvement when recalling known biological relationships curated from public databases. Additionally, we develop a new channel-agnostic MAE architecture (CA-MAE) that allows for inputting images of different numbers and orders of channels at inference time. We demonstrate that CA-MAEs effectively generalize by inferring and evaluating on a microscopy image dataset (JUMP-CP) generated under different experimental conditions with a different channel structure than our pretraining data (RPI-93M). Our findings motivate continued research into scaling self-supervised learning on microscopy data in order to create powerful foundation models of cellular biology that have the potential to catalyze advancements in drug discovery and beyond. |
| title | Masked Autoencoders for Microscopy are Scalable Learners of Cellular Biology |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2404.10242 |