Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2507.06458 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916834130788352 |
|---|---|
| author | Banerjee, Arjun Martinez, David Dang, Camille Tam, Ethan |
| author_facet | Banerjee, Arjun Martinez, David Dang, Camille Tam, Ethan |
| contents | Protein language models (PLMs) encode rich biological information, yet their internal neuron representations are poorly understood. We introduce the first automated framework for labeling every neuron in a PLM with biologically grounded natural language descriptions. Unlike prior approaches relying on sparse autoencoders or manual annotation, our method scales to hundreds of thousands of neurons, revealing individual neurons are selectively sensitive to diverse biochemical and structural properties. We then develop a novel neuron activation-guided steering method to generate proteins with desired traits, enabling convergence to target biochemical properties like molecular weight and instability index as well as secondary and tertiary structural motifs, including alpha helices and canonical Zinc Fingers. We finally show that analysis of labeled neurons in different model sizes reveals PLM scaling laws and a structured neuron space distribution. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_06458 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Automated Neuron Labelling Enables Generative Steering and Interpretability in Protein Language Models Banerjee, Arjun Martinez, David Dang, Camille Tam, Ethan Machine Learning Biomolecules Protein language models (PLMs) encode rich biological information, yet their internal neuron representations are poorly understood. We introduce the first automated framework for labeling every neuron in a PLM with biologically grounded natural language descriptions. Unlike prior approaches relying on sparse autoencoders or manual annotation, our method scales to hundreds of thousands of neurons, revealing individual neurons are selectively sensitive to diverse biochemical and structural properties. We then develop a novel neuron activation-guided steering method to generate proteins with desired traits, enabling convergence to target biochemical properties like molecular weight and instability index as well as secondary and tertiary structural motifs, including alpha helices and canonical Zinc Fingers. We finally show that analysis of labeled neurons in different model sizes reveals PLM scaling laws and a structured neuron space distribution. |
| title | Automated Neuron Labelling Enables Generative Steering and Interpretability in Protein Language Models |
| topic | Machine Learning Biomolecules |
| url | https://arxiv.org/abs/2507.06458 |