Copertina

PUB: A Pragmatics Understanding Benchmark for Assessing LLMs' Pragmatics Capabilities

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Sravanthi, Settaluri Lakshmi, Doshi, Meet, Kalyan, Tankala Pavan, Murthy, Rudra, Bhattacharyya, Pushpak, Dabre, Raj
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Computation and Language
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Understand the Implication: Learning to Think for Pragmatic Understanding
di: Sravanthi, Settaluri Lakshmi, et al.
Pubblicazione: (2025)

Pretraining Language Models Using Translationese
di: Doshi, Meet, et al.
Pubblicazione: (2024)

How effective is Multi-source pivoting for Translation of Low Resource Indian Languages?
di: Gaikwad, Pranav, et al.
Pubblicazione: (2024)

Are Language Models Agnostic to Linguistically Grounded Perturbations? A Case Study of Indic Languages
di: Ghosh, Poulami, et al.
Pubblicazione: (2024)

P-ReMIS: Pragmatic Reasoning in Mental Health and a Social Implication
di: Oram, Sneha, et al.
Pubblicazione: (2025)

Stereotype Detection as a Catalyst for Enhanced Bias Detection: A Multi-Task Learning Approach
di: Tomar, Aditya, et al.
Pubblicazione: (2025)

Mistral-SPLADE: LLMs for better Learned Sparse Retrieval
di: Doshi, Meet, et al.
Pubblicazione: (2024)

A Morphology-Based Investigation of Positional Encodings
di: Ghosh, Poulami, et al.
Pubblicazione: (2024)

Mathematics Isn't Culture-Free: Probing Cultural Gaps via Entity and Scenario Perturbations
di: Tomar, Aditya, et al.
Pubblicazione: (2025)

Pragmatic Theories Enhance Understanding of Implied Meanings in LLMs
di: Sato, Takuma, et al.
Pubblicazione: (2025)

Developing a Pragmatic Benchmark for Assessing Korean Legal Language Understanding in Large Language Models
di: Kim, Yeeun, et al.
Pubblicazione: (2024)

Pragmatic inference of scalar implicature by LLMs
di: Cho, Ye-eun, et al.
Pubblicazione: (2024)

Beyond Understanding: Evaluating the Pragmatic Gap in LLMs' Cultural Processing of Figurative Language
di: Attia, Mena, et al.
Pubblicazione: (2025)

Pragmatics beyond humans: meaning, communication, and LLMs
di: Gvoždiak, Vít
Pubblicazione: (2025)

The Pragmatic Mind of Machines: Tracing the Emergence of Pragmatic Competence in Large Language Models
di: Yu, Kefan, et al.
Pubblicazione: (2025)

RiddleBench: A New Generative Reasoning Benchmark for LLMs
di: Halder, Deepon, et al.
Pubblicazione: (2025)

Towards an Analysis of Discourse and Interactional Pragmatic Reasoning Capabilities of Large Language Models
di: Robrecht, Amelie, et al.
Pubblicazione: (2024)

From Polyester Girlfriends to Blind Mice: Creating the First Pragmatics Understanding Benchmarks for Slovene
di: Brglez, Mojca, et al.
Pubblicazione: (2025)

An Empirical Study of In-context Learning in LLMs for Machine Translation
di: Chitale, Pranjal A., et al.
Pubblicazione: (2024)

CEI: A Benchmark for Evaluating Pragmatic Reasoning in Language Models
di: Chun, Jon, et al.
Pubblicazione: (2026)

CycleDistill: Bootstrapping Machine Translation using LLMs with Cyclical Distillation
di: Halder, Deepon, et al.
Pubblicazione: (2025)

MILU: A Multi-task Indic Language Understanding Benchmark
di: Verma, Sshubam, et al.
Pubblicazione: (2024)

Assessing and Improving Punctuation Robustness in English-Marathi Machine Translation
di: Shejole, Kaustubh Shivshankar, et al.
Pubblicazione: (2025)

Top-b: Entropic Regulation of Relative Probability Bands in Autoregressive Language Processes
di: Halder, Deepon, et al.
Pubblicazione: (2026)

Understanding Understanding: A Pragmatic Framework Motivated by Large Language Models
di: Leyton-Brown, Kevin, et al.
Pubblicazione: (2024)

Cross-Lingual Auto Evaluation for Assessing Multilingual LLMs
di: Doddapaneni, Sumanth, et al.
Pubblicazione: (2024)

BharatBBQ: A Multilingual Bias Benchmark for Question Answering in the Indian Context
di: Tomar, Aditya, et al.
Pubblicazione: (2025)

Reconsidering SMT Over NMT for Closely Related Languages: A Case Study of Persian-Hindi Pair
di: Yousofi, Waisullah, et al.
Pubblicazione: (2024)

Facts-and-Feelings: Capturing both Objectivity and Subjectivity in Table-to-Text Generation
di: Dey, Tathagata, et al.
Pubblicazione: (2024)

We Care: Multimodal Depression Detection and Knowledge Infused Mental Health Therapeutic Response Generation
di: Moon, Palash, et al.
Pubblicazione: (2024)

Main Predicate and Their Arguments as Explanation Signals For Intent Classification
di: Pimparkhede, Sameer, et al.
Pubblicazione: (2025)

Class Distillation with Mahalanobis Contrast: An Efficient Training Paradigm for Pragmatic Language Understanding Tasks
di: Wang, Chenlu, et al.
Pubblicazione: (2025)

Hindi-BEIR : A Large Scale Retrieval Benchmark in Hindi
di: Acharya, Arkadeep, et al.
Pubblicazione: (2024)

PETra: A Multilingual Corpus of Pragmatic Explicitation in Translation
di: Osmelak, Doreen, et al.
Pubblicazione: (2025)

ManagerBench: Evaluating the Safety-Pragmatism Trade-off in Autonomous LLMs
di: Simhi, Adi, et al.
Pubblicazione: (2025)

Scripts Through Time: A Survey of the Evolving Role of Transliteration in NLP
di: Jayakumar, Thanmay, et al.
Pubblicazione: (2026)

RoundTripOCR: A Data Generation Technique for Enhancing Post-OCR Error Correction in Low-Resource Devanagari Languages
di: Kashid, Harshvivek, et al.
Pubblicazione: (2024)

IndicRAGSuite: Large-Scale Datasets and a Benchmark for Indian Language RAG Systems
di: Prasanjith, Pasunuti, et al.
Pubblicazione: (2025)

PUB: Plot Understanding Benchmark and Dataset for Evaluating Large Language Models on Synthetic Visual Data Interpretation
di: Pawelec, Aneta, et al.
Pubblicazione: (2024)

Topics in the Study of the Pragmatic Functions of Phonetic Reduction in Dialog
di: Ward, Nigel G., et al.
Pubblicazione: (2024)