Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2509.06774 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908937908912128 |
|---|---|
| author | Dutta, Hridoy Sankar Ansari, Sana Kumari, Swati Bhalerao, Shounak Ravi |
| author_facet | Dutta, Hridoy Sankar Ansari, Sana Kumari, Swati Bhalerao, Shounak Ravi |
| contents | Organizations and educational institutions use time-bound assessment tasks to evaluate coding and problem-solving skills. These assessments measure not only the correctness of the solutions, but also their efficiency. Problem setters (educator/interviewer) are responsible for crafting these challenges, carefully balancing difficulty and relevance to create meaningful evaluation experiences. Conversely, problem solvers (student/interviewee) apply critical and logical thinking to arrive at correct solutions. In the era of Large Language Models (LLMs), LLMs assist problem setters in generating diverse and challenging questions, but they can undermine assessment integrity for problem solvers by providing easy access to solutions. We introduce OpenCoderRank, a lightweight, self-hosted platform that emulates real-world timed technical assessments in resource-constrained environments. OpenCoderRank is intentionally model-agnostic: it facilitates the creation, deployment and automatic grading of problems while offering fine-grained control over time limits, input-output pairs and execution constraints. OpenCoderRank is evaluated using two methods: 1. BERTScore, 2. LLM evaluation. Our findings indicate that OpenCoderRank connects problem setters and solvers by supporting time-constrained preparation and self-hosted, customizable assessments in resource-constrained settings. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_06774 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | OpenCoderRank: Personalized Technical Assessments with Generative AI Dutta, Hridoy Sankar Ansari, Sana Kumari, Swati Bhalerao, Shounak Ravi Software Engineering Organizations and educational institutions use time-bound assessment tasks to evaluate coding and problem-solving skills. These assessments measure not only the correctness of the solutions, but also their efficiency. Problem setters (educator/interviewer) are responsible for crafting these challenges, carefully balancing difficulty and relevance to create meaningful evaluation experiences. Conversely, problem solvers (student/interviewee) apply critical and logical thinking to arrive at correct solutions. In the era of Large Language Models (LLMs), LLMs assist problem setters in generating diverse and challenging questions, but they can undermine assessment integrity for problem solvers by providing easy access to solutions. We introduce OpenCoderRank, a lightweight, self-hosted platform that emulates real-world timed technical assessments in resource-constrained environments. OpenCoderRank is intentionally model-agnostic: it facilitates the creation, deployment and automatic grading of problems while offering fine-grained control over time limits, input-output pairs and execution constraints. OpenCoderRank is evaluated using two methods: 1. BERTScore, 2. LLM evaluation. Our findings indicate that OpenCoderRank connects problem setters and solvers by supporting time-constrained preparation and self-hosted, customizable assessments in resource-constrained settings. |
| title | OpenCoderRank: Personalized Technical Assessments with Generative AI |
| topic | Software Engineering |
| url | https://arxiv.org/abs/2509.06774 |