Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2312.03523 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909094689898496 |
|---|---|
| author | Tseriotou, Talia Chan, Ryan Sze-Yin Tsakalidis, Adam Bilal, Iman Munire Kochkina, Elena Lyons, Terry Liakata, Maria |
| author_facet | Tseriotou, Talia Chan, Ryan Sze-Yin Tsakalidis, Adam Bilal, Iman Munire Kochkina, Elena Lyons, Terry Liakata, Maria |
| contents | We present an open-source, pip installable toolkit, Sig-Networks, the first of its kind for longitudinal language modelling. A central focus is the incorporation of Signature-based Neural Network models, which have recently shown success in temporal tasks. We apply and extend published research providing a full suite of signature-based models. Their components can be used as PyTorch building blocks in future architectures. Sig-Networks enables task-agnostic dataset plug-in, seamless pre-processing for sequential data, parameter flexibility, automated tuning across a range of models. We examine signature networks under three different NLP tasks of varying temporal granularity: counselling conversations, rumour stance switch and mood changes in social media threads, showing SOTA performance in all three, and provide guidance for future tasks. We release the Toolkit as a PyTorch package with an introductory video, Git repositories for preprocessing and modelling including sample notebooks on the modeled NLP tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2312_03523 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Sig-Networks Toolkit: Signature Networks for Longitudinal Language Modelling Tseriotou, Talia Chan, Ryan Sze-Yin Tsakalidis, Adam Bilal, Iman Munire Kochkina, Elena Lyons, Terry Liakata, Maria Computation and Language We present an open-source, pip installable toolkit, Sig-Networks, the first of its kind for longitudinal language modelling. A central focus is the incorporation of Signature-based Neural Network models, which have recently shown success in temporal tasks. We apply and extend published research providing a full suite of signature-based models. Their components can be used as PyTorch building blocks in future architectures. Sig-Networks enables task-agnostic dataset plug-in, seamless pre-processing for sequential data, parameter flexibility, automated tuning across a range of models. We examine signature networks under three different NLP tasks of varying temporal granularity: counselling conversations, rumour stance switch and mood changes in social media threads, showing SOTA performance in all three, and provide guidance for future tasks. We release the Toolkit as a PyTorch package with an introductory video, Git repositories for preprocessing and modelling including sample notebooks on the modeled NLP tasks. |
| title | Sig-Networks Toolkit: Signature Networks for Longitudinal Language Modelling |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2312.03523 |