TheItalianJob - Ethical Data Analysis | Graduate School Admission (Student track option 2)
| Team Name | TheItalianJob |
| Track | Student Track 2 |
| Use Case | Ethical Data Analysis |
| Technology | SAS Viya, Python |
| Region | EMEA |
| Team lead | Pasquale Maritato |
| Team members | @AAlessandrelli |
| Social media handles | *all team members' social media links here* |
| Is your team interested in participating in an interview? | N |
| Optional: Expand on your technology expertise |
Pitch Video
Jury Video
Team photo
Hey Lincoln, thank you very much for your feedback.
I'm glad you asked, since being in a hurry and not being video editing wizards, we skipped showing a lot of our analysis.
We agree that Legacy Admissions was indeed highly predictive; we built a page in SAS VA basically for each feature that showed the different admission rates for Legacy Admission values.
Once we found out there was bias against some applicants, we dropped the sensitive features (gender, cultural identity, and country region).
At that point, there was still some bias, mainly for cultural identity. We had two options (actually, they are not mutually exclusive): dropping other features or mitigating the bias with exponentiated gradient reduction through the mitigate bias action. We tried both and also combinations of both.
Our results showed that dropping the Legacy Admission feature would not have a high impact both before and after the mitigation.
Example:
- Before mitigation, keeping Legacy Admission
- Before mitigation, dropping Legacy Admission
As you can see, there was a 3% drop in prediction parity bias for Cultural Identity, but an increase for Country Region and Gender.
Probably the features we built with feature engineering on the Mission Statement really offset the effect of Legacy Admission in terms of predictive power.
After mitigating the bias, dropping Legacy Admission only slightly increased the misclassification rate but had basically no effect on bias metrics (actually the model mitigated for demographic parity had a lower prediction bias when keeping Legacy Admission).
The dataset was really small, so the results might vary with a different split (we used a 60-20-20 split stratified on Admissions and Cultural Identity). At the same time, since it was so small, we preferred to mitigate the bias rather than dropping too many features.
For our suggestion to predict the future students performance instead of predicting the admission based on historical data, Legacy Admission could be a good predictor (in econometrics there are studies that show how parents educations affects students performances), anyway it'd be important to check its effect on bias metrics.
We still have access to our pipelines so if you have any other question feel free to ask.
Andrea & Pasquale
