To evaluate the effectiveness of adversarial debiasing with a gradient reversal layer (GRL) in improving age-equitable diabetes prediction and to assess fairness across different data partitions, including stratified analysis.
Approach:
Dataset: Utilized the Pima Indians Diabetes Database (n = 768) with predictors standardized using a scaler fitted on training data. The train-test split was stratified by diabetes outcome.
Key Findings:
The adversarial model improved recall for the >50 years age group from 0.5556 to 0.7778 (+22.22 percentage points).
Overall discrimination remained comparable with a slight increase in ROC-AUC from 0.7852 to 0.7896 (+0.45 pp).
The recall parity gap increased from 0.0996 to 0.2153 (+11.57 pp), indicating a decline in recall for the <30-year group (−6.25 pp).
Mean recall parity gap showed a modest reduction across five seeds (0.3282 to 0.3033, −2.49 pp) but with high variability (SD > 0.27).
The adversarial model reduced the fairness gap in three of five seeds, increased it in one, and produced no change in one.
Interpretation:
Adversarial debiasing can enhance predictive recall for underrepresented demographic subgroups but does not ensure consistent fairness improvements across different data partitions.
Limitations:
Single train–test splits may not provide reliable fairness assessments due to potential biases in subgroup representation.
Small subgroup sizes can lead to high variability in fairness metrics, complicating the assessment of model performance.
Conclusion:
Multi-seed evaluation is essential for reliable fairness assessment in machine learning models for diabetes prediction.
