Algorithm explanation about shap HOT 1 CLOSED

Hey, good question! For Kernel SHAP the mapping is defined where the composition f(h_x(z')) is the expectation of the model output conditioned on the variables set to 1 in the simplified input vector z'. In the code this expectation is computed by assuming that all input features are independent and then integrating over the background dataset provided by the user. Integration is done by evaluating the model on each input sample in the background dataset, sometimes this background dataset is just a single reference input sample for sake of speed. This means h_x is never explicitly defined in the code.

Another way to look at the formulation above is that h_x(z') maps to an input vector 'x' with either missing values or the original input value, depending on if the corresponding element of z' is 0 or 1 respectively. Then the conditional expectation is just a way to get around that most models can't handle arbitrary patterns of missing values.

Hope that helps!

from shap.