Disentangled and Distilled Encoder for Out-of-Distribution Reasoning with Rademacher Guarantees
Fuente:
arXiv
Guardado en:
| Autores principales: | , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866915669150269440 |
|---|---|
| author | Rahiminasab, Zahra Yuhas, Michael Easwaran, Arvind |
| author_facet | Rahiminasab, Zahra Yuhas, Michael Easwaran, Arvind |
| contents | Recently, the disentangled latent space of a variational autoencoder (VAE) has been used to reason about multi-label out-of-distribution (OOD) test samples that are derived from different distributions than training samples. Disentangled latent space means having one-to-many maps between latent dimensions and generative factors or important characteristics of an image. This paper proposes a disentangled distilled encoder (DDE) framework to decrease the OOD reasoner size for deployment on resource-constrained devices while preserving disentanglement. DDE formalizes student-teacher distillation for model compression as a constrained optimization problem while preserving disentanglement with disentanglement constraints. Theoretical guarantees for disentanglement during distillation based on Rademacher complexity are established. The approach is evaluated empirically by deploying the compressed model on an NVIDIA |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_10522 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Disentangled and Distilled Encoder for Out-of-Distribution Reasoning with Rademacher Guarantees Rahiminasab, Zahra Yuhas, Michael Easwaran, Arvind Machine Learning Recently, the disentangled latent space of a variational autoencoder (VAE) has been used to reason about multi-label out-of-distribution (OOD) test samples that are derived from different distributions than training samples. Disentangled latent space means having one-to-many maps between latent dimensions and generative factors or important characteristics of an image. This paper proposes a disentangled distilled encoder (DDE) framework to decrease the OOD reasoner size for deployment on resource-constrained devices while preserving disentanglement. DDE formalizes student-teacher distillation for model compression as a constrained optimization problem while preserving disentanglement with disentanglement constraints. Theoretical guarantees for disentanglement during distillation based on Rademacher complexity are established. The approach is evaluated empirically by deploying the compressed model on an NVIDIA |
| title | Disentangled and Distilled Encoder for Out-of-Distribution Reasoning with Rademacher Guarantees |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2512.10522 |