Saved in:
| Main Authors: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2507.23267 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916872661762048 |
|---|---|
| author | Braithwaite, D. T. Cavalcanti, Misael McEver, R. Austin Udagawa, Hiroto Silva, Daniel Ramanath, Rohan Meneses, Felipe Yoshida, Arissa Wingert, Evan Ramos, Matheus Zanfelice, Brian Gupta, Aman |
| author_facet | Braithwaite, D. T. Cavalcanti, Misael McEver, R. Austin Udagawa, Hiroto Silva, Daniel Ramanath, Rohan Meneses, Felipe Yoshida, Arissa Wingert, Evan Ramos, Matheus Zanfelice, Brian Gupta, Aman |
| contents | Predictive models play a crucial role in the financial industry, enabling risk prediction, fraud detection, and personalized recommendations, where slight changes in core model performance can result in billions of dollars in revenue or losses. While financial institutions have access to enormous amounts of user data (e.g., bank transactions, in-app events, and customer support logs), leveraging this data effectively remains challenging due to its complexity and scale. Thus, in many financial institutions, most production models follow traditional machine learning (ML) approaches by converting unstructured data into manually engineered tabular features. Conversely, other domains (e.g., natural language processing) have effectively utilized self-supervised learning (SSL) to learn rich representations from raw data, removing the need for manual feature extraction. In this paper, we investigate using transformer-based representation learning models for transaction data, hypothesizing that these models, trained on massive data, can provide a novel and powerful approach to understanding customer behavior. We propose a new method enabling the use of SSL with transaction data by adapting transformer-based models to handle both textual and structured attributes. Our approach, denoted nuFormer, includes an end-to-end fine-tuning method that integrates user embeddings with existing tabular features. Our experiments demonstrate improvements for large-scale recommendation problems at Nubank. Notably, these gains are achieved solely through enhanced representation learning rather than incorporating new data sources. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_23267 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Your Spending Needs Attention: Modeling Financial Habits with Transformers Braithwaite, D. T. Cavalcanti, Misael McEver, R. Austin Udagawa, Hiroto Silva, Daniel Ramanath, Rohan Meneses, Felipe Yoshida, Arissa Wingert, Evan Ramos, Matheus Zanfelice, Brian Gupta, Aman Information Retrieval Predictive models play a crucial role in the financial industry, enabling risk prediction, fraud detection, and personalized recommendations, where slight changes in core model performance can result in billions of dollars in revenue or losses. While financial institutions have access to enormous amounts of user data (e.g., bank transactions, in-app events, and customer support logs), leveraging this data effectively remains challenging due to its complexity and scale. Thus, in many financial institutions, most production models follow traditional machine learning (ML) approaches by converting unstructured data into manually engineered tabular features. Conversely, other domains (e.g., natural language processing) have effectively utilized self-supervised learning (SSL) to learn rich representations from raw data, removing the need for manual feature extraction. In this paper, we investigate using transformer-based representation learning models for transaction data, hypothesizing that these models, trained on massive data, can provide a novel and powerful approach to understanding customer behavior. We propose a new method enabling the use of SSL with transaction data by adapting transformer-based models to handle both textual and structured attributes. Our approach, denoted nuFormer, includes an end-to-end fine-tuning method that integrates user embeddings with existing tabular features. Our experiments demonstrate improvements for large-scale recommendation problems at Nubank. Notably, these gains are achieved solely through enhanced representation learning rather than incorporating new data sources. |
| title | Your Spending Needs Attention: Modeling Financial Habits with Transformers |
| topic | Information Retrieval |
| url | https://arxiv.org/abs/2507.23267 |