Utilizing Large Language Models to Synthesize Product Desirability Datasets
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866929750178529280 |
|---|---|
| author | Hastings, John D. Weitl-Harms, Sherri Doty, Joseph Myers, Zachary J. Thompson, Warren |
| author_facet | Hastings, John D. Weitl-Harms, Sherri Doty, Joseph Myers, Zachary J. Thompson, Warren |
| contents | This research explores the application of large language models (LLMs) to generate synthetic datasets for Product Desirability Toolkit (PDT) testing, a key component in evaluating user sentiment and product experience. Utilizing gpt-4o-mini, a cost-effective alternative to larger commercial LLMs, three methods, Word+Review, Review+Word, and Supply-Word, were each used to synthesize 1000 product reviews. The generated datasets were assessed for sentiment alignment, textual diversity, and data generation cost. Results demonstrated high sentiment alignment across all methods, with Pearson correlations ranging from 0.93 to 0.97. Supply-Word exhibited the highest diversity and coverage of PDT terms, although with increased generation costs. Despite minor biases toward positive sentiments, in situations with limited test data, LLM-generated synthetic data offers significant advantages, including scalability, cost savings, and flexibility in dataset production. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_13485 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Utilizing Large Language Models to Synthesize Product Desirability Datasets Hastings, John D. Weitl-Harms, Sherri Doty, Joseph Myers, Zachary J. Thompson, Warren Computation and Language Artificial Intelligence Machine Learning I.2.7; H.3.3; I.2.6; H.5.2 This research explores the application of large language models (LLMs) to generate synthetic datasets for Product Desirability Toolkit (PDT) testing, a key component in evaluating user sentiment and product experience. Utilizing gpt-4o-mini, a cost-effective alternative to larger commercial LLMs, three methods, Word+Review, Review+Word, and Supply-Word, were each used to synthesize 1000 product reviews. The generated datasets were assessed for sentiment alignment, textual diversity, and data generation cost. Results demonstrated high sentiment alignment across all methods, with Pearson correlations ranging from 0.93 to 0.97. Supply-Word exhibited the highest diversity and coverage of PDT terms, although with increased generation costs. Despite minor biases toward positive sentiments, in situations with limited test data, LLM-generated synthetic data offers significant advantages, including scalability, cost savings, and flexibility in dataset production. |
| title | Utilizing Large Language Models to Synthesize Product Desirability Datasets |
| topic | Computation and Language Artificial Intelligence Machine Learning I.2.7; H.3.3; I.2.6; H.5.2 |
| url | https://arxiv.org/abs/2411.13485 |