MLEM: Generative and Contrastive Learning as Distinct Modalities for Event Sequences
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914856939028480 |
|---|---|
| author | Moskvoretskii, Viktor Osin, Dmitry Shvetsov, Egor Udovichenko, Igor Zhelnin, Maxim Dukhovny, Andrey Zhimerikina, Anna Burnaev, Evgeny |
| author_facet | Moskvoretskii, Viktor Osin, Dmitry Shvetsov, Egor Udovichenko, Igor Zhelnin, Maxim Dukhovny, Andrey Zhimerikina, Anna Burnaev, Evgeny |
| contents | This study explores the application of self-supervised learning techniques for event sequences. It is a key modality in various applications such as banking, e-commerce, and healthcare. However, there is limited research on self-supervised learning for event sequences, and methods from other domains like images, texts, and speech may not easily transfer. To determine the most suitable approach, we conduct a detailed comparative analysis of previously identified best-performing methods. We find that neither the contrastive nor generative method is superior. Our assessment includes classifying event sequences, predicting the next event, and evaluating embedding quality. These results further highlight the potential benefits of combining both methods. Given the lack of research on hybrid models in this domain, we initially adapt the baseline model from another domain. However, upon observing its underperformance, we develop a novel method called the Multimodal-Learning Event Model (MLEM). MLEM treats contrastive learning and generative modeling as distinct yet complementary modalities, aligning their embeddings. The results of our study demonstrate that combining contrastive and generative approaches into one procedure with MLEM achieves superior performance across multiple metrics. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_15935 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | MLEM: Generative and Contrastive Learning as Distinct Modalities for Event Sequences Moskvoretskii, Viktor Osin, Dmitry Shvetsov, Egor Udovichenko, Igor Zhelnin, Maxim Dukhovny, Andrey Zhimerikina, Anna Burnaev, Evgeny Machine Learning Artificial Intelligence This study explores the application of self-supervised learning techniques for event sequences. It is a key modality in various applications such as banking, e-commerce, and healthcare. However, there is limited research on self-supervised learning for event sequences, and methods from other domains like images, texts, and speech may not easily transfer. To determine the most suitable approach, we conduct a detailed comparative analysis of previously identified best-performing methods. We find that neither the contrastive nor generative method is superior. Our assessment includes classifying event sequences, predicting the next event, and evaluating embedding quality. These results further highlight the potential benefits of combining both methods. Given the lack of research on hybrid models in this domain, we initially adapt the baseline model from another domain. However, upon observing its underperformance, we develop a novel method called the Multimodal-Learning Event Model (MLEM). MLEM treats contrastive learning and generative modeling as distinct yet complementary modalities, aligning their embeddings. The results of our study demonstrate that combining contrastive and generative approaches into one procedure with MLEM achieves superior performance across multiple metrics. |
| title | MLEM: Generative and Contrastive Learning as Distinct Modalities for Event Sequences |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2401.15935 |