Saved in:
| Main Authors: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2603.27476 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917365858435072 |
|---|---|
| author | Wang, Wei Shi, Tianyu Zhang, Shuai Xia, Boyang Xie, Zequn Zeng, Chenyu Zhang, Qi Ai, Lynn Yu, Yaqi Zhang, Kaiming Tang, Feiyue |
| author_facet | Wang, Wei Shi, Tianyu Zhang, Shuai Xia, Boyang Xie, Zequn Zeng, Chenyu Zhang, Qi Ai, Lynn Yu, Yaqi Zhang, Kaiming Tang, Feiyue |
| contents | AI-powered people search platforms are increasingly used in recruiting, sales prospecting, and professional networking, yet no widely accepted benchmark exists for evaluating their performance. We introduce PeopleSearchBench, an open-source benchmark that compares four people search platforms on 119 real-world queries across four use cases: corporate recruiting, B2B sales prospecting, expert search with deterministic answers, and influencer/KOL discovery. A key contribution is Criteria-Grounded Verification, a factual relevance pipeline that extracts explicit, verifiable criteria from each query and uses live web search to determine whether returned people satisfy them. This produces binary relevance judgments grounded in factual verification rather than subjective holistic LLM-as-judge scores. We evaluate systems on three dimensions: Relevance Precision (padded nDCG@10), Effective Coverage (task completion and qualified result yield), and Information Utility (profile completeness and usefulness), averaged equally into an overall score. Lessie, a specialized AI people search agent, performs best overall, scoring 65.2, 18.5% higher than the second-ranked system, and is the only system to achieve 100% task completion across all 119 queries. We also report confidence intervals, human validation of the verification pipeline (Cohen's kappa = 0.84), ablations, and full documentation of queries, prompts, and normalization procedures. Code, query definitions, and aggregated results are available on GitHub. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_27476 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | PeopleSearchBench: A Multi-Dimensional Benchmark for Evaluating AI-Powered People Search Platforms Wang, Wei Shi, Tianyu Zhang, Shuai Xia, Boyang Xie, Zequn Zeng, Chenyu Zhang, Qi Ai, Lynn Yu, Yaqi Zhang, Kaiming Tang, Feiyue Artificial Intelligence Machine Learning AI-powered people search platforms are increasingly used in recruiting, sales prospecting, and professional networking, yet no widely accepted benchmark exists for evaluating their performance. We introduce PeopleSearchBench, an open-source benchmark that compares four people search platforms on 119 real-world queries across four use cases: corporate recruiting, B2B sales prospecting, expert search with deterministic answers, and influencer/KOL discovery. A key contribution is Criteria-Grounded Verification, a factual relevance pipeline that extracts explicit, verifiable criteria from each query and uses live web search to determine whether returned people satisfy them. This produces binary relevance judgments grounded in factual verification rather than subjective holistic LLM-as-judge scores. We evaluate systems on three dimensions: Relevance Precision (padded nDCG@10), Effective Coverage (task completion and qualified result yield), and Information Utility (profile completeness and usefulness), averaged equally into an overall score. Lessie, a specialized AI people search agent, performs best overall, scoring 65.2, 18.5% higher than the second-ranked system, and is the only system to achieve 100% task completion across all 119 queries. We also report confidence intervals, human validation of the verification pipeline (Cohen's kappa = 0.84), ablations, and full documentation of queries, prompts, and normalization procedures. Code, query definitions, and aggregated results are available on GitHub. |
| title | PeopleSearchBench: A Multi-Dimensional Benchmark for Evaluating AI-Powered People Search Platforms |
| topic | Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2603.27476 |