QUIS: Question-guided Insights Generation for Automated Exploratory Data Analysis
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910657758101504 |
|---|---|
| author | Manatkar, Abhijit Akella, Ashlesha Gupta, Parthivi Narayanam, Krishnasuri |
| author_facet | Manatkar, Abhijit Akella, Ashlesha Gupta, Parthivi Narayanam, Krishnasuri |
| contents | Discovering meaningful insights from a large dataset, known as Exploratory Data Analysis (EDA), is a challenging task that requires thorough exploration and analysis of the data. Automated Data Exploration (ADE) systems use goal-oriented methods with Large Language Models and Reinforcement Learning towards full automation. However, these methods require human involvement to anticipate goals that may limit insight extraction, while fully automated systems demand significant computational resources and retraining for new datasets. We introduce QUIS, a fully automated EDA system that operates in two stages: insight generation (ISGen) driven by question generation (QUGen). The QUGen module generates questions in iterations, refining them from previous iterations to enhance coverage without human intervention or manually curated examples. The ISGen module analyzes data to produce multiple relevant insights in response to each question, requiring no prior training and enabling QUIS to adapt to new datasets. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_10270 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | QUIS: Question-guided Insights Generation for Automated Exploratory Data Analysis Manatkar, Abhijit Akella, Ashlesha Gupta, Parthivi Narayanam, Krishnasuri Artificial Intelligence Computation and Language Databases Machine Learning Discovering meaningful insights from a large dataset, known as Exploratory Data Analysis (EDA), is a challenging task that requires thorough exploration and analysis of the data. Automated Data Exploration (ADE) systems use goal-oriented methods with Large Language Models and Reinforcement Learning towards full automation. However, these methods require human involvement to anticipate goals that may limit insight extraction, while fully automated systems demand significant computational resources and retraining for new datasets. We introduce QUIS, a fully automated EDA system that operates in two stages: insight generation (ISGen) driven by question generation (QUGen). The QUGen module generates questions in iterations, refining them from previous iterations to enhance coverage without human intervention or manually curated examples. The ISGen module analyzes data to produce multiple relevant insights in response to each question, requiring no prior training and enabling QUIS to adapt to new datasets. |
| title | QUIS: Question-guided Insights Generation for Automated Exploratory Data Analysis |
| topic | Artificial Intelligence Computation and Language Databases Machine Learning |
| url | https://arxiv.org/abs/2410.10270 |