Machine-Readable Behavioural Compliance Evidence for AI Systems: A Specification Profiling Framework
Fuente:
Zenodo
Salvato in:
| Autore principale: | |
|---|---|
| Natura: | Recurso digital |
| Lingua: | inglese |
| Pubblicazione: |
Zenodo
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866901163555684352 |
|---|---|
| author | Caprazli, Kafkas M. |
| author_facet | Caprazli, Kafkas M. |
| contents | Existing methods for evaluating AI behaviour conflate personality measurement with specification compliance. This paper presents the Specification Profiling Framework (SPF), a specification-verification method that produces machine-readable evidence of whether an AI system's observable output conforms to an explicit behavioural specification. SPF evaluates systems across eight behavioural constraints using a two-turn protocol that isolates specification effects from baseline behaviour. Methodology validation with four commercial AI systems reveals significant per-system variation: compliance ranges from 0/8 to 6/8 constraints. A specification reversal anomaly (D8 DomainStrictness) demonstrates that multi-dimensional separated assessment surfaces structural failures invisible to scalar scoring. All evidence artefacts are structured (JSON), reproducible, and mapped to EU AI Act conformity assessment requirements (Annex A). |
| format | Recurso digital |
| id | zenodo_https___doi_org_10_5281_zenodo_18984072 |
| institution | Zenodo |
| language | eng |
| publishDate | 2026 |
| publisher | Zenodo |
| record_format | zenodo |
| spellingShingle | Machine-Readable Behavioural Compliance Evidence for AI Systems: A Specification Profiling Framework Caprazli, Kafkas M. AI conformity assessment behavioural specification EU AI Act specification compliance LLM evaluation machine-readable evidence CEN/CENELEC AI standardisation specification profiling DGD framework Existing methods for evaluating AI behaviour conflate personality measurement with specification compliance. This paper presents the Specification Profiling Framework (SPF), a specification-verification method that produces machine-readable evidence of whether an AI system's observable output conforms to an explicit behavioural specification. SPF evaluates systems across eight behavioural constraints using a two-turn protocol that isolates specification effects from baseline behaviour. Methodology validation with four commercial AI systems reveals significant per-system variation: compliance ranges from 0/8 to 6/8 constraints. A specification reversal anomaly (D8 DomainStrictness) demonstrates that multi-dimensional separated assessment surfaces structural failures invisible to scalar scoring. All evidence artefacts are structured (JSON), reproducible, and mapped to EU AI Act conformity assessment requirements (Annex A). |
| title | Machine-Readable Behavioural Compliance Evidence for AI Systems: A Specification Profiling Framework |
| topic | AI conformity assessment behavioural specification EU AI Act specification compliance LLM evaluation machine-readable evidence CEN/CENELEC AI standardisation specification profiling DGD framework |
| url | https://doi.org/10.5281/zenodo.18984072 |