Detecting Bias with SHAP: What do Developer Salaries tell us about the Gender Pay Gap?
We hear about "model bias," but models simply reflect the data they trained on. Can we use them to detect instances of bias in data?
Join Data Scientist Sean Owen, as he shares how standard models can be augmented with SHAP (SHapley Additive exPlanations) to detect predictions that may be concerning and how to dig deeper into the reasons behind those predictions. To illustrate this, he'll examine the results of the 2019 StackOverflow Developer Survey, and apply Apache Spark and SHAP to study whether attributes like gender have outsized effects on developer salaries.
During this session you'll learn:
Why bias is an important topic in data, and how models can help detect bias
How to apply SHAP to explain what factor influence each prediction from a model
How to apply Apache Spark to scale up SHAP and detect anomalous predictions