Cover Image

SailCompass: Towards Reproducible and Robust Evaluation for Southeast Asian Languages

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Guo, Jia, Dou, Longxu, Zeng, Guangtao, Kok, Stanley, Lu, Wei, Liu, Qian
Format:	Preprint
Published:	2024
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Sailor: Open Language Models for South-East Asia
by: Dou, Longxu, et al.
Published: (2024)

RegMix: Data Mixture as Regression for Language Model Pre-training
by: Liu, Qian, et al.
Published: (2024)

SEA-LION: Southeast Asian Languages in One Network
by: Ng, Raymond, et al.
Published: (2025)

Investigating the Effects of Cognitive Biases in Prompts on Large Language Model Outputs
by: Sun, Yan, et al.
Published: (2025)

Sentiment-Aware Extractive and Abstractive Summarization for Unstructured Text Mining
by: Liu, Junyi, et al.
Published: (2025)

SEA-HELM: Southeast Asian Holistic Evaluation of Language Models
by: Susanto, Yosephine, et al.
Published: (2025)

Reasoning Does Not Necessarily Improve Role-Playing Ability
by: Feng, Xiachong, et al.
Published: (2025)

A Survey of Table Reasoning with Large Language Models
by: Zhang, Xuanliang, et al.
Published: (2024)

TinyLlama: An Open-Source Small Language Model
by: Zhang, Peiyuan, et al.
Published: (2024)

SMARTe: Slot-based Method for Accountable Relational Triple extraction
by: Tan, Xue Wen, et al.
Published: (2025)

Improving Demonstration Diversity by Human-Free Fusing for Text-to-SQL
by: Wang, Dingzirui, et al.
Published: (2024)

Enhancing Numerical Reasoning with the Guidance of Reliable Reasoning Processes
by: Wang, Dingzirui, et al.
Published: (2024)

DAC: Decomposed Automation Correction for Text-to-SQL
by: Wang, Dingzirui, et al.
Published: (2024)

MURRE: Multi-Hop Table Retrieval with Removal for Open-Domain Text-to-SQL
by: Zhang, Xuanliang, et al.
Published: (2024)

Sailor2: Sailing in South-East Asia with Inclusive Multilingual LLMs
by: Dou, Longxu, et al.
Published: (2025)

SEA-BED: How Do Embedding Models Represent Southeast Asian Languages?
by: Ponwitayarat, Wuttikorn, et al.
Published: (2025)

Training Optimal Large Diffusion Language Models
by: Ni, Jinjie, et al.
Published: (2025)

SeaLLMs 3: Open Foundation and Chat Multilingual Large Language Models for Southeast Asian Languages
by: Zhang, Wenxuan, et al.
Published: (2024)

CompassVerifier: A Unified and Robust Verifier for LLMs Evaluation and Outcome Reward
by: Liu, Shudong, et al.
Published: (2025)

SEACrowd: A Multilingual Multimodal Data Hub and Benchmark Suite for Southeast Asian Languages
by: Lovenia, Holy, et al.
Published: (2024)

Compass-v3: Scaling Domain-Specific LLMs for Multilingual E-Commerce in Southeast Asia
by: Maria, Sophia
Published: (2025)

A Survey on Large Language Model-Based Social Agents in Game-Theoretic Scenarios
by: Feng, Xiachong, et al.
Published: (2024)

AgentCompass: Towards Reliable Evaluation of Agentic Workflows in Production
by: Kartik, NVJK, et al.
Published: (2025)

SEALGuard: Safeguarding the Multilingual Conversations in Southeast Asian Languages for LLM Software Systems
by: Shan, Wenliang, et al.
Published: (2025)

OpenCompass: A Universal Evaluation Platform for Large Language Models
by: Cao, Maosong, et al.
Published: (2026)

Scaling Laws with Vocabulary: Larger Models Deserve Larger Vocabularies
by: Tao, Chaofan, et al.
Published: (2024)

Political Compass or Spinning Arrow? Towards More Meaningful Evaluations for Values and Opinions in Large Language Models
by: Röttger, Paul, et al.
Published: (2024)

To Err Is Human; To Annotate, SILICON? Toward Robust Reproducibility in LLM Annotation
by: Cheng, Xiang, et al.
Published: (2024)

Talk Less, Verify More: Improving LLM Assistants with Semantic Checks and Execution Feedback
by: Sun, Yan, et al.
Published: (2026)

Lessons from the Trenches on Reproducible Evaluation of Language Models
by: Biderman, Stella, et al.
Published: (2024)

Compass-Embedding v4: Robust Contrastive Learning for Multilingual E-commerce Embeddings
by: Ueareeworakul, Pakorn, et al.
Published: (2025)

MLPs Compass: What is learned when MLPs are combined with PLMs?
by: Zhou, Li, et al.
Published: (2024)

FLEXTAF: Enhancing Table Reasoning with Flexible Tabular Formats
by: Zhang, Xuanliang, et al.
Published: (2024)

Towards Reproducible LLM Evaluation: Quantifying Uncertainty in LLM Benchmark Scores
by: Blackwell, Robert E., et al.
Published: (2024)

SEADialogues: A Multilingual Culturally Grounded Multi-turn Dialogue Dataset on Southeast Asian Languages
by: Kautsar, Muhammad Dehan Al, et al.
Published: (2025)

QoNext: Towards Next-generation QoE for Foundation Models
by: Guo, Yijin, et al.
Published: (2025)

SCITAT: A Question Answering Benchmark for Scientific Tables and Text Covering Diverse Reasoning Types
by: Zhang, Xuanliang, et al.
Published: (2024)

Unnatural Languages Are Not Bugs but Features for LLMs
by: Duan, Keyu, et al.
Published: (2025)

MHPP: Exploring the Capabilities and Limitations of Language Models Beyond Basic Code Generation
by: Dai, Jianbo, et al.
Published: (2024)

PPTC-R benchmark: Towards Evaluating the Robustness of Large Language Models for PowerPoint Task Completion
by: Zhang, Zekai, et al.
Published: (2024)