Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2404.18255 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916274236293120 |
|---|---|
| author | Bai, Zilong Zhang, Ruiji Chen, Linqing Cai, Qijun Zhong, Yuan Wang, Cong Fang, Yan Fang, Jie Sun, Jing Wang, Weikuan Zhou, Lizhi Hua, Haoran Qiu, Tian Wang, Chaochao Sun, Cheng Lu, Jianping Wang, Yixin Xia, Yubin Hu, Meng Liu, Haowen Xu, Peng Xu, Licong Bian, Fu Gu, Xiaolong Zhang, Lisha Wang, Weilei Tu, Changyang |
| author_facet | Bai, Zilong Zhang, Ruiji Chen, Linqing Cai, Qijun Zhong, Yuan Wang, Cong Fang, Yan Fang, Jie Sun, Jing Wang, Weikuan Zhou, Lizhi Hua, Haoran Qiu, Tian Wang, Chaochao Sun, Cheng Lu, Jianping Wang, Yixin Xia, Yubin Hu, Meng Liu, Haowen Xu, Peng Xu, Licong Bian, Fu Gu, Xiaolong Zhang, Lisha Wang, Weilei Tu, Changyang |
| contents | In recent years, large language models(LLMs) have attracted significant attention due to their exceptional performance across a multitude of natural language process tasks, and have been widely applied in various fields. However, the application of large language models in the Intellectual Property (IP) domain is challenging due to the strong need for specialized knowledge, privacy protection, processing of extremely long text in this field. In this technical report, we present for the first time a low-cost, standardized procedure for training IP-oriented LLMs, meeting the unique requirements of the IP domain. Using this standard process, we have trained the PatentGPT series models based on open-source pretrained models. By evaluating them on the open-source IP-oriented benchmark MOZIP, our domain-specific LLMs outperforms GPT-4, indicating the effectiveness of the proposed training procedure and the expertise of the PatentGPT models in the IP domain. Remarkably, our model surpassed GPT-4 on the 2019 China Patent Agent Qualification Examination, scoring 65 and matching human expert levels. Additionally, the PatentGPT model, which utilizes the SMoE architecture, achieves performance comparable to that of GPT-4 in the IP domain and demonstrates a better cost-performance ratio on long-text tasks, potentially serving as an alternative to GPT-4 within the IP domain. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_18255 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | PatentGPT: A Large Language Model for Intellectual Property Bai, Zilong Zhang, Ruiji Chen, Linqing Cai, Qijun Zhong, Yuan Wang, Cong Fang, Yan Fang, Jie Sun, Jing Wang, Weikuan Zhou, Lizhi Hua, Haoran Qiu, Tian Wang, Chaochao Sun, Cheng Lu, Jianping Wang, Yixin Xia, Yubin Hu, Meng Liu, Haowen Xu, Peng Xu, Licong Bian, Fu Gu, Xiaolong Zhang, Lisha Wang, Weilei Tu, Changyang Computation and Language Artificial Intelligence I.2.7 In recent years, large language models(LLMs) have attracted significant attention due to their exceptional performance across a multitude of natural language process tasks, and have been widely applied in various fields. However, the application of large language models in the Intellectual Property (IP) domain is challenging due to the strong need for specialized knowledge, privacy protection, processing of extremely long text in this field. In this technical report, we present for the first time a low-cost, standardized procedure for training IP-oriented LLMs, meeting the unique requirements of the IP domain. Using this standard process, we have trained the PatentGPT series models based on open-source pretrained models. By evaluating them on the open-source IP-oriented benchmark MOZIP, our domain-specific LLMs outperforms GPT-4, indicating the effectiveness of the proposed training procedure and the expertise of the PatentGPT models in the IP domain. Remarkably, our model surpassed GPT-4 on the 2019 China Patent Agent Qualification Examination, scoring 65 and matching human expert levels. Additionally, the PatentGPT model, which utilizes the SMoE architecture, achieves performance comparable to that of GPT-4 in the IP domain and demonstrates a better cost-performance ratio on long-text tasks, potentially serving as an alternative to GPT-4 within the IP domain. |
| title | PatentGPT: A Large Language Model for Intellectual Property |
| topic | Computation and Language Artificial Intelligence I.2.7 |
| url | https://arxiv.org/abs/2404.18255 |