Cover Image

Contextual Counting: A Mechanistic Study of Transformers on a Quantitative Task

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Golkar, Siavash, Bietti, Alberto, Pettee, Mariel, Eickenberg, Michael, Cranmer, Miles, Hirashima, Keiya, Krawezik, Geraud, Lourie, Nicholas, McCabe, Michael, Morel, Rudy, Ohana, Ruben, Parker, Liam Holden, Blancard, Bruno Régaldo-Saint, Cho, Kyunghyun, Ho, Shirley
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Multiple Physics Pretraining for Physical Surrogate Models
by: McCabe, Michael, et al.
Published: (2023)

xVal: A Continuous Numerical Tokenization for Scientific Language Models
by: Golkar, Siavash, et al.
Published: (2023)

AstroCLIP: A Cross-Modal Foundation Model for Galaxies
by: Parker, Liam, et al.
Published: (2023)

AION-1: Omnimodal Foundation Model for Astronomical Sciences
by: Parker, Liam, et al.
Published: (2025)

Universal Spectral Tokenization via Self-Supervised Panchromatic Representation Learning
by: Shen, Jeff, et al.
Published: (2025)

Walrus: A Cross-Domain Foundation Model for Continuum Dynamics
by: McCabe, Michael, et al.
Published: (2025)

Statistical Component Separation for Targeted Signal Recovery in Noisy Mixtures
by: Blancard, Bruno Régaldo-Saint, et al.
Published: (2023)

What's In Your Field? Mapping Scientific Research with Knowledge Graphs and Large Language Models
by: Das, Abhipsha, et al.
Published: (2025)

Predicting partially observable dynamical systems via diffusion models with a multiscale inference scheme
by: Morel, Rudy, et al.
Published: (2025)

Overtone: Cyclic Patch Modulation for Clean, Efficient, and Flexible Physics Emulators
by: Mukhopadhyay, Payel, et al.
Published: (2025)

Emergent Transfer of a Physics Foundation Model from Simulation to Laboratory Turbulence
by: Mukhopadhyay, Payel, et al.
Published: (2026)

Mitigating Model Misspecification in Simulation-Based Inference for Galaxy Clustering
by: Pierre, Sébastien, et al.
Published: (2025)

Physics Steering: Causal Control of Cross-Domain Concepts in a Physics Foundation Model
by: Fear, Rio Alexa, et al.
Published: (2025)

On the Value of Tokeniser Pretraining in Physics Foundation Models
by: Sotoudeh, Hadi, et al.
Published: (2026)

Listening to the Noise: Blind Denoising with Gibbs Diffusion
by: Heurtel-Depeiges, David, et al.
Published: (2024)

Galaxy-challenge_SC25_Artifact2
by: Hirashima, Keiya
Published: (2025)

MIMIC: A Generative Multimodal Foundation Model for Biomolecules
by: Golkar, Siavash, et al.
Published: (2026)

{\sc SimBIG}: Cosmological Constraints using Simulation-Based Inference of Galaxy Clustering with Marked Power Spectra
by: Massara, Elena, et al.
Published: (2024)

${\rm S{\scriptsize IM}BIG}$: Cosmological Constraints from the Redshift-Space Galaxy Skew Spectra
by: Hou, Jiamin, et al.
Published: (2024)

Simulation-based stacking
by: Yao, Yuling, et al.
Published: (2023)

Representation Learning for Spatiotemporal Physical Systems
by: Qu, Helen, et al.
Published: (2026)

Galaxy Clustering Analysis with SimBIG and the Wavelet Scattering Transform
by: Blancard, Bruno Régaldo-Saint, et al.
Published: (2023)

The Well: a Large-Scale Collection of Diverse Physics Simulations for Machine Learning
by: Ohana, Ruben, et al.
Published: (2024)

Lost in Latent Space: An Empirical Study of Latent Diffusion Models for Physics Emulation
by: Rozet, François, et al.
Published: (2025)

Se extrae la energía humana de Venezuela : Creación de nuevas asociaciones para el desarrollo / Michael McCabe
by: McCabe, Michael
Published: (1995)

Scaling Laws Are Unreliable for Downstream Tasks: A Reality Check
by: Lourie, Nicholas, et al.
Published: (2025)

Self-supervised Synthetic Pretraining for Inference of Stellar Mass Embedded in Dense Gas
by: Hirashima, Keiya, et al.
Published: (2025)

Editorial Commentary: Timely Treatment of Femoroacetabular Impingement Syndrome Both Treats Ongoing Hip Symptoms and Mitigates Future Cartilage Damage and Osteoarthritis
by: Michael P. McCabe
Published: (2024)

Editorial Commentary: Over‐resection, or Under‐Resection, of the Femoral Cam Lesion in Patients Undergoing Hip Arthroscopy for Femoroacetabular Impingement Syndrome Indicates Different Possible Solutions
by: Michael P. McCabe
Published: (2025)

Multimodal Datasets with Controllable Mutual Information
by: Hashmani, Raheem Karim, et al.
Published: (2025)

Joint Embeddings Go Temporal
by: Ennadir, Sofiane, et al.
Published: (2025)

Nerve Models of Subdivision Bifiltrations
by: Lesnick, Michael, et al.
Published: (2024)

Sparse Approximation of the Subdivision-Rips Bifiltration for Doubling Metrics
by: Lesnick, Michael, et al.
Published: (2024)

MoMo: Momentum Models for Adaptive Learning Rates
by: Schaipp, Fabian, et al.
Published: (2023)

Show Your Work with Confidence: Confidence Bands for Tuning Curves
by: Lourie, Nicholas, et al.
Published: (2023)

Hyperparameter Loss Surfaces Are Simple Near their Optima
by: Lourie, Nicholas, et al.
Published: (2025)

The Coming Generation of Computer Proficient Students: What It May Mean for Libraries.
by: McCabe, Gerard B., et al.
Published: (1994)

one-dimensional axial strain
by: McCabe, David
Published: (2025)

A Square Matrix Classification Scheme
by: McCabe, David
Published: (2025)

A Smoking-Gun Resolution of the Hubble Tension: Parameter-Free Local-Global Mapping
by: McCabe, Luke
Published: (2026)