Imagen de Portada

Fine-Grained Benchmark Generation for Comprehensive Evaluation of Foundation Models

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Islam, Mohammed Saidul, Baghbanzadeh, Negin, Kohankhaki, Farnaz, Cheraghi, Afshin, Kore, Ali, Mehdi, Shayaan, Dolatabadi, Elham, Afkanpour, Arash
Formato:	Preprint
Publicado:	2026
Materias:	Machine Learning Artificial Intelligence Computation and Language
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Automated Capability Evaluation of Foundation Models
por: Afkanpour, Arash, et al.
Publicado: (2025)

Open-PMC-18M: A High-Fidelity Large Scale Medical Dataset for Multimodal Representation Learning
por: Baghbanzadeh, Negin, et al.
Publicado: (2025)

Consistency-Guided Asynchronous Contrastive Tuning for Few-Shot Class-Incremental Tuning of Foundation Models
por: Roy, Shuvendu, et al.
Publicado: (2024)

Similarity-Aware Token Pruning: Your VLM but Faster
por: Jeddi, Ahmadreza, et al.
Publicado: (2025)

Practical Guide for Causal Pathways and Sub-group Disparity Analysis
por: Kohankhaki, Farnaz, et al.
Publicado: (2024)

A Shared Encoder Approach to Multimodal Representation Learning
por: Roy, Shuvendu, et al.
Publicado: (2025)

Natural Language Generation for Visualizations: State of the Art, Challenges and Future Directions
por: Hoque, Enamul, et al.
Publicado: (2024)

Advancing Medical Representation Learning Through High-Quality Data
por: Baghbanzadeh, Negin, et al.
Publicado: (2025)

Exploring Bias and Prediction Metrics to Characterise the Fairness of Machine Learning for Equity-Centered Public Health Decision-Making: A Narrative Review
por: Raza, Shaina, et al.
Publicado: (2024)

Benchmarking Vision-Language Contrastive Methods for Medical Representation Learning
por: Roy, Shuvendu, et al.
Publicado: (2024)

When Does RL Help Medical VLMs? Disentangling Vision, SFT, and RL Gains
por: Jeddi, Ahmadreza, et al.
Publicado: (2026)

BanglaSummEval: Reference-Free Factual Consistency Evaluation for Bangla Summarization
por: Rafid, Ahmed, et al.
Publicado: (2026)

BenLLMEval: A Comprehensive Evaluation into the Potentials and Pitfalls of Large Language Models on Bengali NLP
por: Kabir, Mohsinul, et al.
Publicado: (2023)

Template-Based Probes Are Imperfect Lenses for Counterfactual Bias Evaluation in LLMs
por: Kohankhaki, Farnaz, et al.
Publicado: (2024)

Are Large Vision Language Models up to the Challenge of Chart Comprehension and Reasoning? An Extensive Investigation into the Capabilities and Limitations of LVLMs
por: Islam, Mohammed Saidul, et al.
Publicado: (2024)

Factcheck-Bench: Fine-Grained Evaluation Benchmark for Automatic Fact-checkers
por: Wang, Yuxia, et al.
Publicado: (2023)

Aligning Text, Code, and Vision: A Multi-Objective Reinforcement Learning Framework for Text-to-Visualization
por: Rahman, Mizanur, et al.
Publicado: (2026)

FAIR Enough: How Can We Develop and Assess a FAIR-Compliant Dataset for Large Language Models' Training?
por: Raza, Shaina, et al.
Publicado: (2024)

Judging the Judges: Can Large Vision-Language Models Fairly Evaluate Chart Comprehension and Reasoning?
por: Laskar, Md Tahmid Rahman, et al.
Publicado: (2025)

Improving Automatic Evaluation of Large Language Models (LLMs) in Biomedical Relation Extraction via LLMs-as-the-Judge
por: Laskar, Md Tahmid Rahman, et al.
Publicado: (2025)

DataNarrative: Automated Data-Driven Storytelling with Visualizations and Texts
por: Islam, Mohammed Saidul, et al.
Publicado: (2024)

When Can We Trust LLMs in Mental Health? Large-Scale Benchmarks for Reliable LLM Evaluation
por: Badawi, Abeer, et al.
Publicado: (2025)

DashboardQA: Benchmarking Multimodal Agents for Question Answering on Interactive Dashboards
por: Kartha, Aaryaman, et al.
Publicado: (2025)

Lost in Translation: Do LVLM Judges Generalize Across Languages?
por: Laskar, Md Tahmid Rahman, et al.
Publicado: (2026)

FineMath: A Fine-Grained Mathematical Evaluation Benchmark for Chinese Large Language Models
por: Liu, Yan, et al.
Publicado: (2024)

The Perils of Chart Deception: How Misleading Visualizations Affect Vision-Language Models
por: Mahbub, Ridwan, et al.
Publicado: (2025)

LLMs Are Not Intelligent Thinkers: Introducing Mathematical Topic Tree Benchmark for Comprehensive Evaluation of LLMs
por: Davoodi, Arash Gholami, et al.
Publicado: (2024)

FREB-TQA: A Fine-Grained Robustness Evaluation Benchmark for Table Question Answering
por: Zhou, Wei, et al.
Publicado: (2024)

C-FAITH: A Chinese Fine-Grained Benchmark for Automated Hallucination Evaluation
por: Zhang, Xu, et al.
Publicado: (2025)

Evaluating Progress in Graph Foundation Models: A Comprehensive Benchmark and New Insights
por: Yu, Xingtong, et al.
Publicado: (2026)

P2P: Automated Paper-to-Poster Generation and Fine-Grained Benchmark
por: Sun, Tao, et al.
Publicado: (2025)

Fine-Grained Prediction of Reading Comprehension from Eye Movements
por: Shubi, Omer, et al.
Publicado: (2024)

CFVBench: A Comprehensive Video Benchmark for Fine-grained Multimodal Retrieval-Augmented Generation
por: Wei, Kaiwen, et al.
Publicado: (2025)

SKA-Bench: A Fine-Grained Benchmark for Evaluating Structured Knowledge Understanding of LLMs
por: Liu, Zhiqiang, et al.
Publicado: (2025)

From Charts to Fair Narratives: Uncovering and Mitigating Geo-Economic Biases in Chart-to-Text
por: Mahbub, Ridwan, et al.
Publicado: (2025)

IslamicMMLU: A Benchmark for Evaluating LLMs on Islamic Knowledge
por: Abdelaal, Ali, et al.
Publicado: (2026)

SparseTransX: Efficient Training of Translation-Based Knowledge Graph Embeddings Using Sparse Matrix Operations
por: Anik, Md Saidul Hoque, et al.
Publicado: (2025)

Academic case reports lack diversity: Assessing the presence and diversity of sociodemographic and behavioral factors related to Post COVID-19 Condition
por: Florez, Juan Andres Medina, et al.
Publicado: (2025)

VLDBench Evaluating Multimodal Disinformation with Regulatory Alignment
por: Raza, Shaina, et al.
Publicado: (2025)

A Flexible Fairness Framework with Surrogate Loss Reweighting for Addressing Sociodemographic Disparities
por: Xu, Wen, et al.
Publicado: (2025)