Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2412.16247 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911062820913152 |
|---|---|
| author | Donhauser, Konstantin Ulicna, Kristina Moran, Gemma Elyse Ravuri, Aditya Kenyon-Dean, Kian Eastwood, Cian Hartford, Jason |
| author_facet | Donhauser, Konstantin Ulicna, Kristina Moran, Gemma Elyse Ravuri, Aditya Kenyon-Dean, Kian Eastwood, Cian Hartford, Jason |
| contents | Sparse dictionary learning (DL) has emerged as a powerful approach to extract semantically meaningful concepts from the internals of large language models (LLMs) trained mainly in the text domain. In this work, we explore whether DL can extract meaningful concepts from less human-interpretable scientific data, such as vision foundation models trained on cell microscopy images, where limited prior knowledge exists about which high-level concepts should arise. We propose a novel combination of a sparse DL algorithm, Iterative Codebook Feature Learning (ICFL), with a PCA whitening pre-processing step derived from control data. Using this combined approach, we successfully retrieve biologically meaningful concepts, such as cell types and genetic perturbations. Moreover, we demonstrate how our method reveals subtle morphological changes arising from human-interpretable interventions, offering a promising new direction for scientific discovery via mechanistic interpretability in bioimaging. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_16247 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Towards scientific discovery with dictionary learning: Extracting biological concepts from microscopy foundation models Donhauser, Konstantin Ulicna, Kristina Moran, Gemma Elyse Ravuri, Aditya Kenyon-Dean, Kian Eastwood, Cian Hartford, Jason Machine Learning Artificial Intelligence Computer Vision and Pattern Recognition Sparse dictionary learning (DL) has emerged as a powerful approach to extract semantically meaningful concepts from the internals of large language models (LLMs) trained mainly in the text domain. In this work, we explore whether DL can extract meaningful concepts from less human-interpretable scientific data, such as vision foundation models trained on cell microscopy images, where limited prior knowledge exists about which high-level concepts should arise. We propose a novel combination of a sparse DL algorithm, Iterative Codebook Feature Learning (ICFL), with a PCA whitening pre-processing step derived from control data. Using this combined approach, we successfully retrieve biologically meaningful concepts, such as cell types and genetic perturbations. Moreover, we demonstrate how our method reveals subtle morphological changes arising from human-interpretable interventions, offering a promising new direction for scientific discovery via mechanistic interpretability in bioimaging. |
| title | Towards scientific discovery with dictionary learning: Extracting biological concepts from microscopy foundation models |
| topic | Machine Learning Artificial Intelligence Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2412.16247 |