Has Your Pretrained Model Improved? A Multi-head Posterior Based Approach
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866916125829234688 |
|---|---|
| author | Aboagye, Prince Zheng, Yan Wang, Junpeng Saini, Uday Singh Dai, Xin Yeh, Michael Fan, Yujie Zhuang, Zhongfang Jain, Shubham Wang, Liang Zhang, Wei |
| author_facet | Aboagye, Prince Zheng, Yan Wang, Junpeng Saini, Uday Singh Dai, Xin Yeh, Michael Fan, Yujie Zhuang, Zhongfang Jain, Shubham Wang, Liang Zhang, Wei |
| contents | The emergence of pre-trained models has significantly impacted Natural Language Processing (NLP) and Computer Vision to relational datasets. Traditionally, these models are assessed through fine-tuned downstream tasks. However, this raises the question of how to evaluate these models more efficiently and more effectively. In this study, we explore a novel approach where we leverage the meta-features associated with each entity as a source of worldly knowledge and employ entity representations from the models. We propose using the consistency between these representations and the meta-features as a metric for evaluating pre-trained models. Our method's effectiveness is demonstrated across various domains, including models with relational datasets, large language models and image models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_02987 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Has Your Pretrained Model Improved? A Multi-head Posterior Based Approach Aboagye, Prince Zheng, Yan Wang, Junpeng Saini, Uday Singh Dai, Xin Yeh, Michael Fan, Yujie Zhuang, Zhongfang Jain, Shubham Wang, Liang Zhang, Wei Computation and Language Artificial Intelligence The emergence of pre-trained models has significantly impacted Natural Language Processing (NLP) and Computer Vision to relational datasets. Traditionally, these models are assessed through fine-tuned downstream tasks. However, this raises the question of how to evaluate these models more efficiently and more effectively. In this study, we explore a novel approach where we leverage the meta-features associated with each entity as a source of worldly knowledge and employ entity representations from the models. We propose using the consistency between these representations and the meta-features as a metric for evaluating pre-trained models. Our method's effectiveness is demonstrated across various domains, including models with relational datasets, large language models and image models. |
| title | Has Your Pretrained Model Improved? A Multi-head Posterior Based Approach |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2401.02987 |