partial fails with "gbm" objects due to missing n.trees argument.

Hi!

We're coming up to a new release of gbm which considerably tidies the code and introduces a much stricter NAMESPACE. We've also removed the multinomial distribution, which was essentially unsupported and rather buggy code, and it looks like it's breaking your package.

Would it be possible for you to take a look?

00check.log.txt

Hi @bgreenwell thank you so much for your awesome package... Really fantastic..

Nevertheless, we have noticed that it only supports gaussian regression and classification. Is it possible to further implement gamma and poisson regressions? I am using xgboost 0.6.x
Thank in advance for your attention
Giorgio

I'm trying to use pdp with an xgboost model. The training data is mostly text that I am encoding using bag of words. So I start off with a data frame that contains a column of text. That text is tokenized and a sparse term document matrix created. This matrix is what is used to train the xgboost model.

It appears that pdp can't handle this situation yet where the training data for the xgboost model isn't in a data frame, but in a matrix. I tried this:

partial(xgb_fit, train = train_matrix, pred.var = "price")
Error in partial.default(xgb_fit, train = train_matrix, pred.var = "price") : 
  price not found in the training data.

but as you can see it doesn't work. Here train_matrix is a xgb.DMatrix object that was created from the bag of words term-document matrix of type dgCMatrix.

Is it possible to use pdp to generate partial dependence plots in this case?

Borrowing from rpart:

if (missing(method)) {
  method <- if (is.factor(Y) || is.character(Y)) "class"
   else if (inherits(Y, "Surv")) "exp"
  else if (is.matrix(Y)) "poisson"
  else "anova"
 }

Allow column position specification for the pred.grid argument:

partial(fit, pred.var = "x")
partial(fit, pred.var = 1)

Currently the predicted probabilities are not wrapped correctly for XGBoost models. This can be fixed by using reshape = TRUE in the prediction call.

If we let type = c("link", "response"), then do we even need the superType generic?

This is a statistic for assessing the strength of interaction between predictors and is based on the partial dependence functions. Start with gbm's implementation; see ?gbm::interact.gbm.

Currently, partial uses the same algorithm for all objects. However, gbm's plot function is much faster on "gbm" objects. Perhaps it would be more resourceful to simply make partial.gbm a wrapper around gbm::plot.gbm.

Will need the C++ code and methods for the new class "GBMFit" (no more multinomial here).

Support added for e1071::naiveBayes models (classic naive Bayes).

This should suffice:

type <- match.arg(type)
if (type == "auto") {
  if (!is.null(pred.fun)) {
    "regression"  # doesn't matter
  } else {
    superType(object)
  }
}

Not sure this possible right now since it will require knowledge of the response vector. Perhaps add two new options: center and response?

partial(mod, pred.var = "X1", center = TRUE, response = "Y", plot = TRUE, train = trn)

The following will not work for "randomForest" (and similar) objects:

partial(fit, pred.var = "x", type = "prob")

Perhaps revert to super.type = c("regression", "classification")? Again, this can be accomplished via the pred.fun argument, but this will be quicker in many cases.

Very similar in spirit to XGBoost, but faster.

This would automatically reshape predicted probabilities into a matrix alleviating the need to do it manually in both getPDClassLogit and getPDClassProb.

Alternatively, a better default name like "Partial dependence" or something could be used.

All that is needed is the creation of an appropriate X matrix (i.e., pred.grid) and a call to the gbm_plot C++ function.

Example usage:

partial(obj, pred.var = "x", quantiles = TRUE, probs = 1:9/10, plot = TRUE))
partial(obj, pred.var = "x", trim.outliers = TRUE, plot = TRUE))

Fixed!

Looks like classification models contain a lev component.

Following recent work in scikit-learn, it might be useful to have three functions:

• pdBrute - used when approx = FALSE and recursive = FALSE and averages over training data (slow, but accurate);
• pdApprox - used when approx = TRUE and will fix other predictors at their median/mode (fast, but less accurate);
• pdGBM - used when recursive = TRUE and uses Friedman's weighted tree traversal method (only for "gbm" objects).

RStudio keeps crashing when ever using partial with "gbm" objects and a user-supplied data frame for the pred.grid argument.

knnreg is a caret function for k nearest neighbors regression. Formula interface seems broken and does not return anything useful.

For now, partial will throw the following error message:

 Error in partial.default(fit, pred.var = "y") : 
  "y" cannot be a predictor name. 

prediction.type = c("link", "response", "centered.logit")

where,

• response.type = "link" returns predictions on the link scale (original scale for ordinary regression models, log count for Poisson models, two-class logit for binomial models, etc.)
• response.type = "response" returns predictions on the original response scale (original scale for ordinary regression models, count for Poisson models, class probabilities for binomial models, etc.)
• response.type = "centered.logit" is only used for classification and returns predictions on a scale similar to a logit, but with the average log probability as the reference class.

Hence, it might be worthwhile to create a transformPrediction function with arguments yhat, dist, and type. While this can all be accomplished using the pred.fun argument, this option will be quicker in many cases.

Probably most useful if object is of class "train":

topPredictors <- function(object, n = 1L, ...) {
  UseMethod("topPredictors")
}
topPredictors.train <- function(object, n = 1L, ...) {
  imp <- caret::varImp(object)$importance
  imp <- imp[order(imp$Overall, decreasing = TRUE), , drop = FALSE]
  rownames(imp)[seq_len(n)]
}

library(pdp)
important.preds <- topPredictors(mod, 2)
partial(mod, pred.var = iimportant.preds, plot = TRUE,
        chull = TRUE, individual = TRUE, progress = "text")

When recursive = TRUE partial will call pdGBM which relies on GBM's C++ code for Friedman's weighted tree traversal approach (which seems to approximate the brute force approach?).