LNE-Blocking: An Efficient Framework for Contamination Mitigation Evaluation on Large Language Models
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866916956586639360 |
|---|---|
| author | Hou, Ruijie Jiao, Yueyang Hu, Hanxu Li, Yingming Lam, Wai Zhang, Huajian Lu, Hongyuan |
| author_facet | Hou, Ruijie Jiao, Yueyang Hu, Hanxu Li, Yingming Lam, Wai Zhang, Huajian Lu, Hongyuan |
| contents | The problem of data contamination is now almost inevitable during the development of large language models (LLMs), with the training data commonly integrating those evaluation benchmarks even unintentionally. This problem subsequently makes it hard to benchmark LLMs fairly. Instead of constructing contamination-free datasets (quite hard), we propose a novel framework, \textbf{LNE-Blocking}, to restore model performance prior to contamination on potentially leaked datasets. Our framework consists of two components: contamination detection and disruption operation. For the prompt, the framework first uses the contamination detection method, \textbf{LNE}, to assess the extent of contamination in the model. Based on this, it adjusts the intensity of the disruption operation, \textbf{Blocking}, to elicit non-memorized responses from the model. Our framework is the first to efficiently restore the model's greedy decoding performance. This comes with a strong performance on multiple datasets with potential leakage risks, and it consistently achieves stable recovery results across different models and varying levels of data contamination. We release the code at https://github.com/RuijieH/LNE-Blocking to facilitate research. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_15218 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | LNE-Blocking: An Efficient Framework for Contamination Mitigation Evaluation on Large Language Models Hou, Ruijie Jiao, Yueyang Hu, Hanxu Li, Yingming Lam, Wai Zhang, Huajian Lu, Hongyuan Computation and Language The problem of data contamination is now almost inevitable during the development of large language models (LLMs), with the training data commonly integrating those evaluation benchmarks even unintentionally. This problem subsequently makes it hard to benchmark LLMs fairly. Instead of constructing contamination-free datasets (quite hard), we propose a novel framework, \textbf{LNE-Blocking}, to restore model performance prior to contamination on potentially leaked datasets. Our framework consists of two components: contamination detection and disruption operation. For the prompt, the framework first uses the contamination detection method, \textbf{LNE}, to assess the extent of contamination in the model. Based on this, it adjusts the intensity of the disruption operation, \textbf{Blocking}, to elicit non-memorized responses from the model. Our framework is the first to efficiently restore the model's greedy decoding performance. This comes with a strong performance on multiple datasets with potential leakage risks, and it consistently achieves stable recovery results across different models and varying levels of data contamination. We release the code at https://github.com/RuijieH/LNE-Blocking to facilitate research. |
| title | LNE-Blocking: An Efficient Framework for Contamination Mitigation Evaluation on Large Language Models |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2509.15218 |