Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | https://arxiv.org/abs/2509.19708 |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866912602405208064 |
|---|---|
| author | Kumar, Anand Khare, Vishal Sharma, Deepak Kumar, Satyam Saini, Vijay Yadav, Anshul Jain, Sachendra Rana, Ankit Verma, Pratham Meena, Vaibhav Edubilli, Avinash |
| author_facet | Kumar, Anand Khare, Vishal Sharma, Deepak Kumar, Satyam Saini, Vijay Yadav, Anshul Jain, Sachendra Rana, Ankit Verma, Pratham Meena, Vaibhav Edubilli, Avinash |
| contents | We present a comprehensive real-world evaluation of AI-assisted software development tools deployed at enterprise scale. Over one year, 300 engineers across multiple teams integrated an in-house AI platform (DeputyDev) that combines code generation and automated review capabilities into their daily workflows. Through rigorous cohort analysis, our study demonstrates statistically significant productivity improvements, including an overall 31.8% reduction in PR review cycle time.
Developer adoption was strong, with 85% satisfaction for code review features and 93% expressing a desire to continue using the platform. Adoption patterns showed systematic scaling from 4% engagement in month 1 to 83% peak usage by month 6, stabilizing at 60% active engagement. Top adopters achieved a 61% increase in code volume pushed to production, contributing to approximately 30 to 40% of code shipped to production through this tool, accounting for an overall 28% increase in code shipment volume.
Unlike controlled benchmark evaluations, our longitudinal analysis provides empirical evidence from production environments, revealing both the transformative potential and practical deployment challenges of integrating AI into enterprise software development workflows. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_19708 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Intuition to Evidence: Measuring AI's True Impact on Developer Productivity Kumar, Anand Khare, Vishal Sharma, Deepak Kumar, Satyam Saini, Vijay Yadav, Anshul Jain, Sachendra Rana, Ankit Verma, Pratham Meena, Vaibhav Edubilli, Avinash Software Engineering Artificial Intelligence Machine Learning We present a comprehensive real-world evaluation of AI-assisted software development tools deployed at enterprise scale. Over one year, 300 engineers across multiple teams integrated an in-house AI platform (DeputyDev) that combines code generation and automated review capabilities into their daily workflows. Through rigorous cohort analysis, our study demonstrates statistically significant productivity improvements, including an overall 31.8% reduction in PR review cycle time. Developer adoption was strong, with 85% satisfaction for code review features and 93% expressing a desire to continue using the platform. Adoption patterns showed systematic scaling from 4% engagement in month 1 to 83% peak usage by month 6, stabilizing at 60% active engagement. Top adopters achieved a 61% increase in code volume pushed to production, contributing to approximately 30 to 40% of code shipped to production through this tool, accounting for an overall 28% increase in code shipment volume. Unlike controlled benchmark evaluations, our longitudinal analysis provides empirical evidence from production environments, revealing both the transformative potential and practical deployment challenges of integrating AI into enterprise software development workflows. |
| title | Intuition to Evidence: Measuring AI's True Impact on Developer Productivity |
| topic | Software Engineering Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2509.19708 |