@LGroves Hey Lincoln, thank you very much for your feedback. I'm glad you asked, since being in a hurry and not being video editing wizards, we skipped showing a lot of our analysis. We agree that Legacy Admissions was indeed highly predictive; we built a page in SAS VA basically for each feature that showed the different admission rates for Legacy Admission values. Once we found out there was bias against some applicants, we dropped the sensitive features (gender, cultural identity, and country region). At that point, there was still some bias, mainly for cultural identity. We had two options (actually, they are not mutually exclusive): dropping other features or mitigating the bias with exponentiated gradient reduction through the mitigate bias action. We tried both and also combinations of both. Our results showed that dropping the Legacy Admission feature would not have a high impact both before and after the mitigation. Example: Before mitigation, keeping Legacy Admission Before mitigation, dropping Legacy Admission As you can see, there was a 3% drop in prediction parity bias for Cultural Identity, but an increase for Country Region and Gender. Probably the features we built with feature engineering on the Mission Statement really offset the effect of Legacy Admission in terms of predictive power. After mitigating the bias, dropping Legacy Admission only slightly increased the misclassification rate but had basically no effect on bias metrics (actually the model mitigated for demographic parity had a lower prediction bias when keeping Legacy Admission). The dataset was really small, so the results might vary with a different split (we used a 60-20-20 split stratified on Admissions and Cultural Identity). At the same time, since it was so small, we preferred to mitigate the bias rather than dropping too many features. For our suggestion to predict the future students performance instead of predicting the admission based on historical data, Legacy Admission could be a good predictor (in econometrics there are studies that show how parents educations affects students performances), anyway it'd be important to check its effect on bias metrics. We still have access to our pipelines so if you have any other question feel free to ask. Andrea & Pasquale
... View more