MoMoE: A Mixture of Expert Agent Model for Financial Sentiment Analysis
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866917086672977920 |
|---|---|
| author | Shu, Peng Chen, Junhao Liu, Zhengliang Jiang, Hanqi Pan, Yi Nguyen, Khanh Nhu Wu, Zihao Zhao, Huaqin Li, Yiwei Shi, Enze Xu, ShaoChen |
| author_facet | Shu, Peng Chen, Junhao Liu, Zhengliang Jiang, Hanqi Pan, Yi Nguyen, Khanh Nhu Wu, Zihao Zhao, Huaqin Li, Yiwei Shi, Enze Xu, ShaoChen |
| contents | We present a novel approach called Mixture of Mixture of Expert (MoMoE) that combines the strengths of Mixture-of-Experts (MoE) architectures with collaborative multi-agent frameworks. By modifying the LLaMA 3.1 8B architecture to incorporate MoE layers in each agent of a layered collaborative structure, we create an ensemble of specialized expert agents that iteratively refine their outputs. Each agent leverages an MoE layer in its final attention block, enabling efficient task decomposition while maintaining computational feasibility. This hybrid approach creates specialized pathways through both the model architecture and the agent collaboration layers. Experimental results demonstrate significant improvements across multiple language understanding and generation benchmarks, highlighting the synergistic benefits of combining expert routing at both the neural and agent levels. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_13983 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | MoMoE: A Mixture of Expert Agent Model for Financial Sentiment Analysis Shu, Peng Chen, Junhao Liu, Zhengliang Jiang, Hanqi Pan, Yi Nguyen, Khanh Nhu Wu, Zihao Zhao, Huaqin Li, Yiwei Shi, Enze Xu, ShaoChen Computational Engineering, Finance, and Science We present a novel approach called Mixture of Mixture of Expert (MoMoE) that combines the strengths of Mixture-of-Experts (MoE) architectures with collaborative multi-agent frameworks. By modifying the LLaMA 3.1 8B architecture to incorporate MoE layers in each agent of a layered collaborative structure, we create an ensemble of specialized expert agents that iteratively refine their outputs. Each agent leverages an MoE layer in its final attention block, enabling efficient task decomposition while maintaining computational feasibility. This hybrid approach creates specialized pathways through both the model architecture and the agent collaboration layers. Experimental results demonstrate significant improvements across multiple language understanding and generation benchmarks, highlighting the synergistic benefits of combining expert routing at both the neural and agent levels. |
| title | MoMoE: A Mixture of Expert Agent Model for Financial Sentiment Analysis |
| topic | Computational Engineering, Finance, and Science |
| url | https://arxiv.org/abs/2511.13983 |