Home›Biases›IBI-00110
IBI-001104 - Severepublie

Benchmark Overfitting Bias

Biais de sur-optimisation du benchmark

IBI Score
7.2
out of 10
Severity4_grave
Frequencyfrequent
Scope groups4
Detection difficulty3
🇫🇷 Définition (FR)

Optimisation excessive sur un benchmark, réduisant la généralisation en production.

🇬🇧 Definition (EN)

Excessive optimization on a benchmark, reducing generalization in production.

TriAxis Taxonomy

Structural Classification

O
Origin
O2
Computational & Statistical
L
Lifecycle
L6
Evaluation
S
Specific
S17
Measurement error bias
💡
Concrete Example

Un modèle atteint 99% sur SQuAD mais échoue sur des questions hors distribution.

📊 Detection Metrics

Demographic Parity DifferenceDisparate Impact RatioEqualized Odds Difference

⚙ Alert Thresholds

Disparate Impact Ratio0.8
Equalized Odds Difference0.1
Demographic Parity Difference0.1

📚 Sources & Scientific References

🛡 Impact & Mitigation

🎯 Vulnerable groups impacted

No specific groups.

🛠 Mitigation methodologies

  • AUDIT_TIERS : Third-party Audit

🏢 Sectors concerned

Education & Training (Risk: /5)Media & Social Platforms (Risk: /5)

⚖ Applicable regulations

🌍 International
  • •
    ISO_42001 : ISO/IEC 42001:2023
  • •
    UNESCO_2021 : UNESCO - Recommandation sur l'éthique de l'IA
  • •
    OECD_AI_PRINCIPLES : OCDE - Principes IA
  • •
    COE_AI_CONVENTION_INT : Conseil de l'Europe - Convention IA

📋 Metadata

Contributor
Mohamadou SIDIBE
Institution
AI BIAIS INITIATIVE
Version
1
Created on
October 14, 2026
Updated on
November 4, 2026
UUID
fb31a3d5-b1c4-4817-ad81-34ccd98a8a3a

Explore other biases

250 documented biases, classified by the TriAxis taxonomy.