Image de couverture de livre

Foldable SuperNets: Scalable Merging of Transformers with Different Initializations and Tasks

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Kinderman, Edan, Hubara, Itay, Maron, Haggai, Soudry, Daniel
Format:	Preprint
Publié:	2024
Sujets:	Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Towards Cheaper Inference in Deep Networks with Lower Bit-Width Accumulators
par: Blumenfeld, Yaniv, et autres
Publié: (2024)

Minimum Variance Unbiased N:M Sparsity for the Neural Gradients
par: Chmiel, Brian, et autres
Publié: (2022)

Retrieval from Within: An Intrinsic Capability of Attention-Based Models
par: Hoffer, Elad, et autres
Publié: (2026)

Block Sparse Flash Attention
par: Ohayon, Daniel, et autres
Publié: (2025)

Subgraphormer: Unifying Subgraph GNNs and Graph Transformers via Graph Products
par: Bar-Shalom, Guy, et autres
Publié: (2024)

The Joint Effect of Task Similarity and Overparameterization on Catastrophic Forgetting -- An Analytical Model
par: Goldfarb, Daniel, et autres
Publié: (2024)

Provable Tempered Overfitting of Minimal Nets and Typical Nets
par: Harel, Itamar, et autres
Publié: (2024)

It Takes a Graph to Know a Graph: Rewiring for Homophily with a Reference Graph
par: Mendelman, Harel, et autres
Publié: (2025)

On the Expressive Power of Permutation-Equivariant Weight-Space Networks
par: Dayan, Adir, et autres
Publié: (2026)

Training Transformers for KV Cache Compressibility
par: Gelberg, Yoav, et autres
Publié: (2026)

Are Greedy Task Orderings Better Than Random in Continual Linear Regression?
par: Tsipory, Matan, et autres
Publié: (2025)

A Graph Meta-Network for Learning on Kolmogorov-Arnold Networks
par: Bar-Shalom, Guy, et autres
Publié: (2026)

On the Reconstruction of Training Data from Group Invariant Networks
par: Elbaz, Ran, et autres
Publié: (2024)

Merging Text Transformer Models from Different Initializations
par: Verma, Neha, et autres
Publié: (2024)

Tensor-Parallelism with Partially Synchronized Activations
par: Lamprecht, Itay, et autres
Publié: (2025)

GradMetaNet: An Equivariant Architecture for Learning on Gradients
par: Gelberg, Yoav, et autres
Publié: (2025)

A Flexible, Equivariant Framework for Subgraph GNNs via Graph Products and Graph Coarsening
par: Bar-Shalom, Guy, et autres
Publié: (2024)

Neural Message-Passing on Attention Graphs for Hallucination Detection
par: Frasca, Fabrizio, et autres
Publié: (2025)

Learning from Historical Activations in Graph Neural Networks
par: Galron, Yaniv, et autres
Publié: (2026)

Efficient GNN Training Through Structure-Aware Randomized Mini-Batching
par: Balaji, Vignesh, et autres
Publié: (2025)

On the Expressive Power of Spectral Invariant Graph Neural Networks
par: Zhang, Bohang, et autres
Publié: (2024)

Learning on LoRAs: GL-Equivariant Processing of Low-Rank Weight Spaces for Large Finetuned Models
par: Putterman, Theo, et autres
Publié: (2024)

Homomorphism Expressivity of Spectral Invariant Graph Neural Networks
par: Gai, Jingchu, et autres
Publié: (2025)

Efficient Subgraph GNNs by Learning Effective Selection Policies
par: Bevilacqua, Beatrice, et autres
Publié: (2023)

GRANOLA: Adaptive Normalization for Graph Neural Networks
par: Eliasof, Moshe, et autres
Publié: (2024)

Optimal L2 Regularization in High-dimensional Continual Linear Regression
par: Karpel, Gilad, et autres
Publié: (2026)

Beyond Token Probes: Hallucination Detection via Activation Tensors with ACT-ViT
par: Bar-Shalom, Guy, et autres
Publié: (2025)

FS-KAN: Permutation Equivariant Kolmogorov-Arnold Networks via Function Sharing
par: Elbaz, Ran, et autres
Publié: (2025)

PLUMAGE: Probabilistic Low rank Unbiased Min Variance Gradient Estimator for Efficient Large Model Training
par: Haroush, Matan, et autres
Publié: (2025)

Understanding and Improving Laplacian Positional Encodings For Temporal GNNs
par: Galron, Yaniv, et autres
Publié: (2025)

Graph Metanetworks for Processing Diverse Neural Architectures
par: Lim, Derek, et autres
Publié: (2023)

Equivariant Deep Weight Space Alignment
par: Navon, Aviv, et autres
Publié: (2023)

Optimal Rates in Continual Linear Regression via Increasing Regularization
par: Levinstein, Ran, et autres
Publié: (2025)

From Continual Learning to SGD and Back: Better Rates for Continual Linear Models
par: Evron, Itay, et autres
Publié: (2025)

The Empirical Impact of Neural Parameter Symmetries, or Lack Thereof
par: Lim, Derek, et autres
Publié: (2024)

On The Expressive Power of GNN Derivatives
par: Eitan, Yam, et autres
Publié: (2025)

ZipIt! Merging Models from Different Tasks without Training
par: Stoica, George, et autres
Publié: (2023)

When Diffusion Models Memorize: Inductive Biases in Probability Flow of Minimum-Norm Shallow Neural Nets
par: Zeno, Chen, et autres
Publié: (2025)

Topological Blindspots: Understanding and Extending Topological Deep Learning Through the Lens of Expressivity
par: Eitan, Yam, et autres
Publié: (2024)

MergeNet: Knowledge Migration across Heterogeneous Models, Tasks, and Modalities
par: Li, Kunxi, et autres
Publié: (2024)