Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2410.08243 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910644391903232 |
|---|---|
| author | Delestre, Cyrile Sola, Yoann |
| author_facet | Delestre, Cyrile Sola, Yoann |
| contents | Banking Transaction Flow (BTF) is a sequential data found in a number of banking activities such as marketing, credit risk or banking fraud. It is a multimodal data composed of three modalities: a date, a numerical value and a wording. We propose in this work an application of self-attention mechanism to the processing of BTFs. We trained two general models on a large amount of BTFs in a self-supervised way: one RNN-based model and one Transformer-based model. We proposed a specific tokenization in order to be able to process BTFs. The performance of these two models was evaluated on two banking downstream tasks: a transaction categorization task and a credit risk task. The results show that fine-tuning these two pre-trained models allowed to perform better than the state-of-the-art approaches for both tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_08243 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Self-Attention Mechanism in Multimodal Context for Banking Transaction Flow Delestre, Cyrile Sola, Yoann Machine Learning Artificial Intelligence Banking Transaction Flow (BTF) is a sequential data found in a number of banking activities such as marketing, credit risk or banking fraud. It is a multimodal data composed of three modalities: a date, a numerical value and a wording. We propose in this work an application of self-attention mechanism to the processing of BTFs. We trained two general models on a large amount of BTFs in a self-supervised way: one RNN-based model and one Transformer-based model. We proposed a specific tokenization in order to be able to process BTFs. The performance of these two models was evaluated on two banking downstream tasks: a transaction categorization task and a credit risk task. The results show that fine-tuning these two pre-trained models allowed to perform better than the state-of-the-art approaches for both tasks. |
| title | Self-Attention Mechanism in Multimodal Context for Banking Transaction Flow |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2410.08243 |