Copertina

SmartBench: Is Your LLM Truly a Good Chinese Smartphone Assistant?

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Lu, Xudong, Gao, Haohao, Wu, Renshou, Ren, Shuai, Chen, Xiaoxin, Li, Hongsheng, Li, Fangyuan
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Computation and Language Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

GenieBlue: Integrating both Linguistic and Multimodal Capabilities for Large Language Models on Mobile Devices
di: Lu, Xudong, et al.
Pubblicazione: (2025)

SmartBench: Evaluating LLMs in Smart Homes with Anomalous Device States and Behavioral Contexts
di: Zou, Qingsong, et al.
Pubblicazione: (2026)

EdgeInfinite: A Memory-Efficient Infinite-Context Transformer for Edge Devices
di: Chen, Jiyu, et al.
Pubblicazione: (2025)

MathVerse: Does Your Multi-modal LLM Truly See the Diagrams in Visual Math Problems?
di: Zhang, Renrui, et al.
Pubblicazione: (2024)

SPP: Sparsity-Preserved Parameter-Efficient Fine-Tuning for Large Language Models
di: Lu, Xudong, et al.
Pubblicazione: (2024)

LiveClawBench: Benchmarking LLM Agents on Complex, Real-World Assistant Tasks
di: Long, Xiang, et al.
Pubblicazione: (2026)

VoiceBench: Benchmarking LLM-Based Voice Assistants
di: Chen, Yiming, et al.
Pubblicazione: (2024)

From Solver to Tutor: Evaluating the Pedagogical Intelligence of LLMs with KMP-Bench
di: Shi, Weikang, et al.
Pubblicazione: (2026)

MJ-Bench: Is Your Multimodal Reward Model Really a Good Judge for Text-to-Image Generation?
di: Chen, Zhaorun, et al.
Pubblicazione: (2024)

CroissantLLM: A Truly Bilingual French-English Language Model
di: Faysse, Manuel, et al.
Pubblicazione: (2024)

VoiceAssistant-Eval: Benchmarking AI Assistants across Listening, Speaking, and Viewing
di: Wang, Ke, et al.
Pubblicazione: (2025)

Learn Your Reference Model for Real Good Alignment
di: Gorbatovski, Alexey, et al.
Pubblicazione: (2024)

Your Agent May Misevolve: Emergent Risks in Self-evolving LLM Agents
di: Shao, Shuai, et al.
Pubblicazione: (2025)

MIR-Bench: Can Your LLM Recognize Complicated Patterns via Many-Shot In-Context Reasoning?
di: Yan, Kai, et al.
Pubblicazione: (2025)

Does Unlearning Truly Unlearn? A Black Box Evaluation of LLM Unlearning Methods
di: Doshi, Jai, et al.
Pubblicazione: (2024)

Your Mixture-of-Experts LLM Is Secretly an Embedding Model For Free
di: Li, Ziyue, et al.
Pubblicazione: (2024)

MMSciBench: Benchmarking Language Models on Chinese Multimodal Scientific Problems
di: Ye, Xinwu, et al.
Pubblicazione: (2025)

VoiceAgentBench: Are Voice Assistants ready for agentic tasks?
di: Jain, Dhruv, et al.
Pubblicazione: (2025)

Lever: Speculative LLM Inference on Smartphones
di: Wang, Tuowei, et al.
Pubblicazione: (2026)

BlueLM-V-3B: Algorithm and System Co-Design for Multimodal Large Language Models on Mobile Devices
di: Lu, Xudong, et al.
Pubblicazione: (2024)

When and What to Ask: AskBench and Rubric-Guided RLVR for LLM Clarification
di: Zhao, Jiale, et al.
Pubblicazione: (2026)

Are Your Generated Instances Truly Useful? GenBench-MILP: A Benchmark Suite for MILP Instance Generation
di: Luo, Yidong, et al.
Pubblicazione: (2025)

Khayyam Challenge (PersianMMLU): Is Your LLM Truly Wise to The Persian Language?
di: Ghahroodi, Omid, et al.
Pubblicazione: (2024)

Not All Experts are Equal: Efficient Expert Pruning and Skipping for Mixture-of-Experts Large Language Models
di: Lu, Xudong, et al.
Pubblicazione: (2024)

11Plus-Bench: Demystifying Multimodal LLM Spatial Reasoning with Cognitive-Inspired Analysis
di: Li, Chengzu, et al.
Pubblicazione: (2025)

ELITR-Bench: A Meeting Assistant Benchmark for Long-Context Language Models
di: Thonet, Thibaut, et al.
Pubblicazione: (2024)

UI-Genie: A Self-Improving Approach for Iteratively Boosting MLLM-based Mobile GUI Agents
di: Xiao, Han, et al.
Pubblicazione: (2025)

Does Machine Unlearning Truly Remove Knowledge?
di: Chen, Haokun, et al.
Pubblicazione: (2025)

StockBench: Can LLM Agents Trade Stocks Profitably In Real-world Markets?
di: Chen, Yanxu, et al.
Pubblicazione: (2025)

AfroBench: How Good are Large Language Models on African Languages?
di: Ojo, Jessica, et al.
Pubblicazione: (2023)

Save the Good Prefix: Precise Error Penalization via Process-Supervised RL to Enhance LLM Reasoning
di: Liu, Haolin, et al.
Pubblicazione: (2026)

Divide-or-Conquer? Which Part Should You Distill Your LLM?
di: Wu, Zhuofeng, et al.
Pubblicazione: (2024)

Adaptive Markup Language Generation for Contextually-Grounded Visual Document Understanding
di: Xiao, Han, et al.
Pubblicazione: (2025)

AgentCollabBench: Diagnosing When Good Agents Make Bad Collaborators
di: Mazumder, Aritra, et al.
Pubblicazione: (2026)

ChineseHarm-Bench: A Chinese Harmful Content Detection Benchmark
di: Liu, Kangwei, et al.
Pubblicazione: (2025)

To Believe or Not to Believe Your LLM
di: Yadkori, Yasin Abbasi, et al.
Pubblicazione: (2024)

Adversarial Preference Optimization: Enhancing Your Alignment via RM-LLM Game
di: Cheng, Pengyu, et al.
Pubblicazione: (2023)

AlignBench: Benchmarking Chinese Alignment of Large Language Models
di: Liu, Xiao, et al.
Pubblicazione: (2023)

Online DPO: Online Direct Preference Optimization with Fast-Slow Chasing
di: Qi, Biqing, et al.
Pubblicazione: (2024)

PhoStream: Benchmarking Real-World Streaming for Omnimodal Assistants in Mobile Scenarios
di: Lu, Xudong, et al.
Pubblicazione: (2026)