IBI-001104 - Severepublie
Benchmark Overfitting Bias
Biais de sur-optimisation du benchmark
IBI Score
7.2
out of 10
Severity4_grave
Frequencyfrequent
Scope groups4
Detection difficulty3
🇫🇷 Définition (FR)
Optimisation excessive sur un benchmark, réduisant la généralisation en production.
🇬🇧 Definition (EN)
Excessive optimization on a benchmark, reducing generalization in production.
TriAxis Taxonomy
Structural Classification
O
Origin
O2
Computational & Statistical
L
Lifecycle
L6
Evaluation
S
Specific
S17
Measurement error bias
💡
Concrete Example
Un modèle atteint 99% sur SQuAD mais échoue sur des questions hors distribution.
📊 Detection Metrics
Demographic Parity DifferenceDisparate Impact RatioEqualized Odds Difference
⚙ Alert Thresholds
Disparate Impact Ratio0.8
Equalized Odds Difference0.1
Demographic Parity Difference0.1
📚 Sources & Scientific References
- •Beyond Accuracy: Behavioral Testing of NLP Models with CheckListRibeiro, M.T., Wu, T., Guestrin, C., Singh, S. (2020)
🛡 Impact & Mitigation
🎯 Vulnerable groups impacted
No specific groups.
🛠 Mitigation methodologies
- AUDIT_TIERS : Third-party Audit
🏢 Sectors concerned
Education & Training (Risk: /5)Media & Social Platforms (Risk: /5)
⚖ Applicable regulations
🌍 International
- •ISO_42001 : ISO/IEC 42001:2023
- •UNESCO_2021 : UNESCO - Recommandation sur l'éthique de l'IA
- •OECD_AI_PRINCIPLES : OCDE - Principes IA
- •COE_AI_CONVENTION_INT : Conseil de l'Europe - Convention IA
📋 Metadata
Contributor
Mohamadou SIDIBE
Institution
AI BIAIS INITIATIVE
Version
1
Created on
October 14, 2026
Updated on
November 4, 2026
UUID
fb31a3d5-b1c4-4817-ad81-34ccd98a8a3a
Explore other biases
250 documented biases, classified by the TriAxis taxonomy.