Re-evaluating the Need for Multimodal Signals in Unsupervised Grammar Induction
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2022
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929312345620480 |
|---|---|
| author | Li, Boyi Corona, Rodolfo Mangalam, Karttikeya Chen, Catherine Flaherty, Daniel Belongie, Serge Weinberger, Kilian Q. Malik, Jitendra Darrell, Trevor Klein, Dan |
| author_facet | Li, Boyi Corona, Rodolfo Mangalam, Karttikeya Chen, Catherine Flaherty, Daniel Belongie, Serge Weinberger, Kilian Q. Malik, Jitendra Darrell, Trevor Klein, Dan |
| contents | Are multimodal inputs necessary for grammar induction? Recent work has shown that multimodal training inputs can improve grammar induction. However, these improvements are based on comparisons to weak text-only baselines that were trained on relatively little textual data. To determine whether multimodal inputs are needed in regimes with large amounts of textual training data, we design a stronger text-only baseline, which we refer to as LC-PCFG. LC-PCFG is a C-PFCG that incorporates em-beddings from text-only large language models (LLMs). We use a fixed grammar family to directly compare LC-PCFG to various multi-modal grammar induction methods. We compare performance on four benchmark datasets. LC-PCFG provides an up to 17% relative improvement in Corpus-F1 compared to state-of-the-art multimodal grammar induction methods. LC-PCFG is also more computationally efficient, providing an up to 85% reduction in parameter count and 8.8x reduction in training time compared to multimodal approaches. These results suggest that multimodal inputs may not be necessary for grammar induction, and emphasize the importance of strong vision-free baselines for evaluating the benefit of multimodal approaches. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2212_10564 |
| institution | arXiv |
| publishDate | 2022 |
| record_format | arxiv |
| spellingShingle | Re-evaluating the Need for Multimodal Signals in Unsupervised Grammar Induction Li, Boyi Corona, Rodolfo Mangalam, Karttikeya Chen, Catherine Flaherty, Daniel Belongie, Serge Weinberger, Kilian Q. Malik, Jitendra Darrell, Trevor Klein, Dan Computation and Language Artificial Intelligence Machine Learning Are multimodal inputs necessary for grammar induction? Recent work has shown that multimodal training inputs can improve grammar induction. However, these improvements are based on comparisons to weak text-only baselines that were trained on relatively little textual data. To determine whether multimodal inputs are needed in regimes with large amounts of textual training data, we design a stronger text-only baseline, which we refer to as LC-PCFG. LC-PCFG is a C-PFCG that incorporates em-beddings from text-only large language models (LLMs). We use a fixed grammar family to directly compare LC-PCFG to various multi-modal grammar induction methods. We compare performance on four benchmark datasets. LC-PCFG provides an up to 17% relative improvement in Corpus-F1 compared to state-of-the-art multimodal grammar induction methods. LC-PCFG is also more computationally efficient, providing an up to 85% reduction in parameter count and 8.8x reduction in training time compared to multimodal approaches. These results suggest that multimodal inputs may not be necessary for grammar induction, and emphasize the importance of strong vision-free baselines for evaluating the benefit of multimodal approaches. |
| title | Re-evaluating the Need for Multimodal Signals in Unsupervised Grammar Induction |
| topic | Computation and Language Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2212.10564 |