Copertina

Reframing Spatial Reasoning Evaluation in Language Models: A Real-World Simulation Benchmark for Qualitative Reasoning

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Li, Fangjun, Hogg, David C., Cohn, Anthony G.
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Computation and Language Artificial Intelligence Databases
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Advancing Spatial Reasoning in Large Language Models: An In-Depth Evaluation and Enhancement Using the StepGame Benchmark
di: Li, Fangjun, et al.
Pubblicazione: (2024)

Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL Workflows
di: Lei, Fangyu, et al.
Pubblicazione: (2024)

QSTRBench: a New Benchmark to Evaluate the Ability of Language Models to Reason with Qualitative Spatial and Temporal Calculi
di: Cohn, Anthony G., et al.
Pubblicazione: (2026)

Schema Lineage Extraction at Scale: Multilingual Pipelines, Composite Evaluation, and Language-Model Benchmarks
di: Yin, Jiaqi, et al.
Pubblicazione: (2025)

RelationalFactQA: A Benchmark for Evaluating Tabular Fact Retrieval from Large Language Models
di: Satriani, Dario, et al.
Pubblicazione: (2025)

MMTU: A Massive Multi-Task Table Understanding and Reasoning Benchmark
di: Xing, Junjie, et al.
Pubblicazione: (2025)

Optimization Techniques for Unsupervised Complex Table Reasoning via Self-Training Framework
di: Li, Zhenyu, et al.
Pubblicazione: (2022)

HES-SQL: Hybrid Reasoning for Efficient Text-to-SQL with Structural Skeleton Guidance
di: Qiu, Suming, et al.
Pubblicazione: (2025)

Language-Native Materials Processing Design by Lightly Structured Text Database and Reasoning Large Language Model
di: Liu, Yuze, et al.
Pubblicazione: (2025)

A State-of-the-Art SQL Reasoning Model using RLVR
di: Ali, Alnur, et al.
Pubblicazione: (2025)

Evaluating the Data Model Robustness of Text-to-SQL Systems Based on Real User Queries
di: Fürst, Jonathan, et al.
Pubblicazione: (2024)

MoNaCo: More Natural and Complex Questions for Reasoning Across Dozens of Documents
di: Wolfson, Tomer, et al.
Pubblicazione: (2025)

Natural Language Interfaces for Spatial and Temporal Databases: A Comprehensive Overview of Methods, Taxonomy, and Future Directions
di: Acharja, Samya, et al.
Pubblicazione: (2026)

IndicDB -- Benchmarking Multilingual Text-to-SQL Capabilities in Indian Languages
di: Dawar, Aviral, et al.
Pubblicazione: (2026)

AtomWorld: A Benchmark for Evaluating Spatial Reasoning in Large Language Models on Crystalline Materials
di: Lv, Taoyuze, et al.
Pubblicazione: (2025)

Explaining Black-box Language Models with Knowledge Probing Systems: A Post-hoc Explanation Perspective
di: Zhao, Yunxiao, et al.
Pubblicazione: (2025)

BEAVER: An Enterprise Benchmark for Text-to-SQL
di: Chen, Peter Baile, et al.
Pubblicazione: (2024)

RUST-BENCH: Benchmarking LLM Reasoning on Unstructured Text within Structured Tables
di: Abhyankar, Nikhil, et al.
Pubblicazione: (2025)

DeepAnalyze: Agentic Large Language Models for Autonomous Data Science
di: Zhang, Shaolei, et al.
Pubblicazione: (2025)

DataVisT5: A Pre-trained Language Model for Jointly Understanding Text and Data Visualization
di: Wan, Zhuoyue, et al.
Pubblicazione: (2024)

OsmT: Bridging OpenStreetMap Queries and Natural Language with Open-source Tag-aware Language Models
di: Wan, Zhuoyue, et al.
Pubblicazione: (2025)

CHASE-SQL: Multi-Path Reasoning and Preference Optimized Candidate Selection in Text-to-SQL
di: Pourreza, Mohammadreza, et al.
Pubblicazione: (2024)

Automated Data Visualization from Natural Language via Large Language Models: An Exploratory Study
di: Wu, Yang, et al.
Pubblicazione: (2024)

A Survey of Large Language Models on Generative Graph Analytics: Query, Learning, and Applications
di: Shang, Wenbo, et al.
Pubblicazione: (2024)

Feather-SQL: A Lightweight NL2SQL Framework with Dual-Model Collaboration Paradigm for Small Language Models
di: Pei, Wenqi, et al.
Pubblicazione: (2025)

SPENCE: A Syntactic Probe for Detecting Contamination in NL2SQL Benchmarks
di: Safarzadeh, Mohammadtaher, et al.
Pubblicazione: (2026)

Structure Guided Large Language Model for SQL Generation
di: Zhang, Qinggang, et al.
Pubblicazione: (2024)

Are Large Language Models the New Interface for Data Pipelines?
di: Junior, Sylvio Barbon, et al.
Pubblicazione: (2024)

Generating Tables from the Parametric Knowledge of Language Models
di: Berkovitch, Yevgeni, et al.
Pubblicazione: (2024)

Are Large Language Models a Good Replacement of Taxonomies?
di: Sun, Yushi, et al.
Pubblicazione: (2024)

GraLMatch: Matching Groups of Entities with Graphs and Language Models
di: Pardo, Fernando De Meer, et al.
Pubblicazione: (2024)

H-STAR: LLM-driven Hybrid SQL-Text Adaptive Reasoning on Tables
di: Abhyankar, Nikhil, et al.
Pubblicazione: (2024)

NormTab: Improving Symbolic Reasoning in LLMs Through Tabular Data Normalization
di: Nahid, Md Mahadi Hasan, et al.
Pubblicazione: (2024)

PURPLE: Making a Large Language Model a Better SQL Writer
di: Ren, Tonghui, et al.
Pubblicazione: (2024)

GradeSQL: Test-Time Inference with Outcome Reward Models for Text-to-SQL Generation from Large Language Models
di: Tritto, Mattia, et al.
Pubblicazione: (2025)

Discovery Agents for Real-Time Analytics: Toward Proactive Insight Systems
di: Rossiello, Gaetano, et al.
Pubblicazione: (2026)

Generating Knowledge Graphs from Large Language Models: A Comparative Study of GPT-4, LLaMA 2, and BERT
di: Bhatt, Ahan, et al.
Pubblicazione: (2024)

SQL-PaLM: Improved Large Language Model Adaptation for Text-to-SQL (extended)
di: Sun, Ruoxi, et al.
Pubblicazione: (2023)

Multi-hop Question Answering over Knowledge Graphs using Large Language Models
di: Chakraborty, Abir
Pubblicazione: (2024)

AnyMatch -- Efficient Zero-Shot Entity Matching with a Small Language Model
di: Zhang, Zeyu, et al.
Pubblicazione: (2024)