Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2409.12255 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909319672365056 |
|---|---|
| author | Jain, Eeshaan Nandy, Tushar Aggarwal, Gaurav Tendulkar, Ashish Iyer, Rishabh De, Abir |
| author_facet | Jain, Eeshaan Nandy, Tushar Aggarwal, Gaurav Tendulkar, Ashish Iyer, Rishabh De, Abir |
| contents | Existing subset selection methods for efficient learning predominantly employ discrete combinatorial and model-specific approaches which lack generalizability. For an unseen architecture, one cannot use the subset chosen for a different model. To tackle this problem, we propose $\texttt{SubSelNet}$, a trainable subset selection framework, that generalizes across architectures. Here, we first introduce an attention-based neural gadget that leverages the graph structure of architectures and acts as a surrogate to trained deep neural networks for quick model prediction. Then, we use these predictions to build subset samplers. This naturally provides us two variants of $\texttt{SubSelNet}$. The first variant is transductive (called as Transductive-$\texttt{SubSelNet}$) which computes the subset separately for each model by solving a small optimization problem. Such an optimization is still super fast, thanks to the replacement of explicit model training by the model approximator. The second variant is inductive (called as Inductive-$\texttt{SubSelNet}$) which computes the subset using a trained subset selector, without any optimization. Our experiments show that our model outperforms several methods across several real datasets |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_12255 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Efficient Data Subset Selection to Generalize Training Across Models: Transductive and Inductive Networks Jain, Eeshaan Nandy, Tushar Aggarwal, Gaurav Tendulkar, Ashish Iyer, Rishabh De, Abir Machine Learning Existing subset selection methods for efficient learning predominantly employ discrete combinatorial and model-specific approaches which lack generalizability. For an unseen architecture, one cannot use the subset chosen for a different model. To tackle this problem, we propose $\texttt{SubSelNet}$, a trainable subset selection framework, that generalizes across architectures. Here, we first introduce an attention-based neural gadget that leverages the graph structure of architectures and acts as a surrogate to trained deep neural networks for quick model prediction. Then, we use these predictions to build subset samplers. This naturally provides us two variants of $\texttt{SubSelNet}$. The first variant is transductive (called as Transductive-$\texttt{SubSelNet}$) which computes the subset separately for each model by solving a small optimization problem. Such an optimization is still super fast, thanks to the replacement of explicit model training by the model approximator. The second variant is inductive (called as Inductive-$\texttt{SubSelNet}$) which computes the subset using a trained subset selector, without any optimization. Our experiments show that our model outperforms several methods across several real datasets |
| title | Efficient Data Subset Selection to Generalize Training Across Models: Transductive and Inductive Networks |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2409.12255 |