xgb.DMatrix cbind about xgboost HOT 2 CLOSED

currently, it is not possible, however, you are allowed to start from big DMatrix and slice it.

from xgboost.

Because you make reference to cbind I imagine you use R. If yes, you can use cbind2 to join two sparse Matrix. It works very well. Don't forget to remove the intercept column before joining (with a formulae like ~.-1).

One use case for me is a dataset very big that I need to hot encode. Because two version of the dataset don't fit in the RAM I hot encod one column a time then merge it in a final matrix, delete the column in the dense matrix and do it again until all column are converted.
It works. It's slow (15mn for 4Gb on a recent laptop).

convertToSparseMatrix <- function(dataset, delDataset = T, verbose = F){
  cols <- colnames(dataset) %>% copy
  for(col in cols){
    if(verbose) print(col)
    f <- col %>% paste("~", ., "-1", sep = "", collapse = "") %>% as.formula
    if(exists("X")){
      X1 <- sparse.model.matrix(f, data = dataset)
      X <- cbind2(X, X1)
      rm(X1)    
    } else {
      X <- sparse.model.matrix(f, data = dataset)
    }
    if(delDataset) dataset[,col:=NULL, with=F]
  }
  X
}

@tqchen are you interested in this method for R package?

Kind regards,
Michaël

from xgboost.