PaECTER: Patent-level Representation Learning using Citation-informed Transformers
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912619027234816 |
|---|---|
| author | Ghosh, Mainak Rose, Michael E. Erhardt, Sebastian Buunk, Erik Harhoff, Dietmar |
| author_facet | Ghosh, Mainak Rose, Michael E. Erhardt, Sebastian Buunk, Erik Harhoff, Dietmar |
| contents | PaECTER is an open-source document-level encoder specific for patents. We fine-tune BERT for Patents with examiner-added citation information to generate numerical representations for patent documents. PaECTER performs better in similarity tasks than current state-of-the-art models used in the patent domain. More specifically, our model outperforms the patent specific pre-trained language model (BERT for Patents) and general-purpose text embedding models (e.g., E5, GTE, and BGE) on our patent citation prediction test dataset on different rank evaluation metrics. PaECTER predicts at least one most similar patent at a rank of 1.32 on average when compared against 25 irrelevant patents. Numerical representations generated by PaECTER from patent text can be used for downstream tasks such as classification, tracing knowledge flows, or semantic similarity search. Semantic similarity search is especially relevant in the context of prior art search for both inventors and patent examiners. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_19411 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | PaECTER: Patent-level Representation Learning using Citation-informed Transformers Ghosh, Mainak Rose, Michael E. Erhardt, Sebastian Buunk, Erik Harhoff, Dietmar Information Retrieval Computation and Language Machine Learning PaECTER is an open-source document-level encoder specific for patents. We fine-tune BERT for Patents with examiner-added citation information to generate numerical representations for patent documents. PaECTER performs better in similarity tasks than current state-of-the-art models used in the patent domain. More specifically, our model outperforms the patent specific pre-trained language model (BERT for Patents) and general-purpose text embedding models (e.g., E5, GTE, and BGE) on our patent citation prediction test dataset on different rank evaluation metrics. PaECTER predicts at least one most similar patent at a rank of 1.32 on average when compared against 25 irrelevant patents. Numerical representations generated by PaECTER from patent text can be used for downstream tasks such as classification, tracing knowledge flows, or semantic similarity search. Semantic similarity search is especially relevant in the context of prior art search for both inventors and patent examiners. |
| title | PaECTER: Patent-level Representation Learning using Citation-informed Transformers |
| topic | Information Retrieval Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2402.19411 |