| _version_ | 1866901667072442368 |
|---|---|
| author | Li, Meng |
| author_facet | Li, Meng |
| contents | Metamorphic Testing addresses the test-oracle problem in scientific computing, but classical Mutation Score (MS) operates on syntactic AST mutations and misses domain semantics. We propose the Semantic Mutation Score (SMS), built on five domain-semantic operators (Conservation Erosion, Operator Substitution, Hyperparameter, Trajectory Flip, Structural Injection). SMS degenerates almost everywhere to MS in a characterised limit, preserving backward compatibility with the classical mutation-testing lineage. A 12-PUT x 5-MP design over four single-output float-to-float classes (numeric, probabilistic, surrogate, ML) is paired with a three-layer attribution classifier separating true semantic faults from tolerance, OOD, statistical, and artefact categories. The pre-registered large-effect threshold for Cliff's delta is not met under the point-estimate criterion; the observed effect lies in the medium-effect range. Cross-source pooling under an identical prompt does not appreciably shift delta. AST-level overlap between LLM-generated and default cosmic-ray syntactic mutants is small; the Hyperparameter, Structural Injection, and Trajectory Flip classes are unreachable under default first-order syntactic configurations. |
| format | Recurso digital |
| id | zenodo_https___doi_org_10_5281_zenodo_20250665 |
| institution | Zenodo |
| language | |
| publishDate | 2026 |
| publisher | Zenodo |
| record_format | zenodo |
| spellingShingle | A semantic mutation metric for metamorphic relation adequacy in scientific computing programs Li, Meng metamorphic testing mutation testing semantic mutation operators metamorphic relation adequacy LLM-generated mutants Cliff's delta scientific computing kernels Metamorphic Testing addresses the test-oracle problem in scientific computing, but classical Mutation Score (MS) operates on syntactic AST mutations and misses domain semantics. We propose the Semantic Mutation Score (SMS), built on five domain-semantic operators (Conservation Erosion, Operator Substitution, Hyperparameter, Trajectory Flip, Structural Injection). SMS degenerates almost everywhere to MS in a characterised limit, preserving backward compatibility with the classical mutation-testing lineage. A 12-PUT x 5-MP design over four single-output float-to-float classes (numeric, probabilistic, surrogate, ML) is paired with a three-layer attribution classifier separating true semantic faults from tolerance, OOD, statistical, and artefact categories. The pre-registered large-effect threshold for Cliff's delta is not met under the point-estimate criterion; the observed effect lies in the medium-effect range. Cross-source pooling under an identical prompt does not appreciably shift delta. AST-level overlap between LLM-generated and default cosmic-ray syntactic mutants is small; the Hyperparameter, Structural Injection, and Trajectory Flip classes are unreachable under default first-order syntactic configurations. |
| title | A semantic mutation metric for metamorphic relation adequacy in scientific computing programs |
| topic | metamorphic testing mutation testing semantic mutation operators metamorphic relation adequacy LLM-generated mutants Cliff's delta scientific computing kernels |
| url | https://doi.org/10.5281/zenodo.20250665 |