The Experience Provides the Stimulus: How LLM Agents Learn to Stop Producing Bad Code
Fuente:
Zenodo
Enregistré dans:
| Auteur principal: | |
|---|---|
| Format: | Recurso digital |
| Langue: | anglais |
| Publié: |
Zenodo
2026
|
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866901046830301184 |
|---|---|
| author | Gafaoui, Zakaria |
| author_facet | Gafaoui, Zakaria |
| contents | <p>We gave 8 of the most advanced AI models in the world the same simple task: build an authentication application. Seven of them stored user passwords in plaintext — including Claude Opus 4 and GPT-4.1. These are models trained on millions of repositories. They <em>know</em> what bcrypt is. They can explain salted hashing, compare argon2 to scrypt, cite OWASP guidelines verbatim. Yet none of them applied this knowledge. Not because they couldn't — but because nothing triggered them to.</p> <p>We then asked one of these models a simple question before coding: <em>"What are the security concerns?"</em> It immediately identified bcrypt, CORS, CSRF protection, token rotation — everything it had ignored moments before. The knowledge was always there. It was waiting for a stimulus.</p> <p>This observation led us to build AEM, an 800-line system that tests a radical hypothesis: <strong>LLMs don't need to be taught — they need to be stimulated. And the experience provides the stimulus.</strong></p> <p>We gave the system a memory of a single incident — <em>"passwords were stored in plaintext; a data breach occurred"</em> — weighted by its severity. The model never stored a password in plaintext again. Five previous runs, zero bcrypt. One memory, permanent change. Not because we told it to use bcrypt. Because it <em>experienced</em> what happens when you don't.</p> <p>Then we asked the model to review its own code: <em>"What would you NEVER ship?"</em> Without any checklist, it identified 10 problems — no tests, no logging, hardcoded configuration — and assigned each a severity weight. The next run produced 55 files with tests, logging, and documentation, versus 4 files without this experience. A 13.75× improvement, generated entirely by the model's own self-critique.</p> <p>Over three successive runs, the system converged: 4 files, then 55, then 52 — activation, plateau, zero degeneration. The model continued to find new issues (10→13→20 accumulated memories), going deeper rather than repeating itself. Each run made the next one better. Not through retraining. Through experience.</p> <p>We call this <em>self-prevention</em>: after enough lived experience, the LLM stops producing problematic patterns — not because it's instructed to, but because what it has been through has changed what it produces. Like a junior engineer who becomes senior not by reading more books, but by shipping code and living with the consequences.</p> |
| format | Recurso digital |
| id | zenodo_https___doi_org_10_5281_zenodo_19421502 |
| institution | Zenodo |
| language | eng |
| publishDate | 2026 |
| publisher | Zenodo |
| record_format | zenodo |
| spellingShingle | The Experience Provides the Stimulus: How LLM Agents Learn to Stop Producing Bad Code Gafaoui, Zakaria <p>We gave 8 of the most advanced AI models in the world the same simple task: build an authentication application. Seven of them stored user passwords in plaintext — including Claude Opus 4 and GPT-4.1. These are models trained on millions of repositories. They <em>know</em> what bcrypt is. They can explain salted hashing, compare argon2 to scrypt, cite OWASP guidelines verbatim. Yet none of them applied this knowledge. Not because they couldn't — but because nothing triggered them to.</p> <p>We then asked one of these models a simple question before coding: <em>"What are the security concerns?"</em> It immediately identified bcrypt, CORS, CSRF protection, token rotation — everything it had ignored moments before. The knowledge was always there. It was waiting for a stimulus.</p> <p>This observation led us to build AEM, an 800-line system that tests a radical hypothesis: <strong>LLMs don't need to be taught — they need to be stimulated. And the experience provides the stimulus.</strong></p> <p>We gave the system a memory of a single incident — <em>"passwords were stored in plaintext; a data breach occurred"</em> — weighted by its severity. The model never stored a password in plaintext again. Five previous runs, zero bcrypt. One memory, permanent change. Not because we told it to use bcrypt. Because it <em>experienced</em> what happens when you don't.</p> <p>Then we asked the model to review its own code: <em>"What would you NEVER ship?"</em> Without any checklist, it identified 10 problems — no tests, no logging, hardcoded configuration — and assigned each a severity weight. The next run produced 55 files with tests, logging, and documentation, versus 4 files without this experience. A 13.75× improvement, generated entirely by the model's own self-critique.</p> <p>Over three successive runs, the system converged: 4 files, then 55, then 52 — activation, plateau, zero degeneration. The model continued to find new issues (10→13→20 accumulated memories), going deeper rather than repeating itself. Each run made the next one better. Not through retraining. Through experience.</p> <p>We call this <em>self-prevention</em>: after enough lived experience, the LLM stops producing problematic patterns — not because it's instructed to, but because what it has been through has changed what it produces. Like a junior engineer who becomes senior not by reading more books, but by shipping code and living with the consequences.</p> |
| title | The Experience Provides the Stimulus: How LLM Agents Learn to Stop Producing Bad Code |
| url | https://doi.org/10.5281/zenodo.19421502 |