Some learners produce errors with discrete Params about mlrmbo HOT 7 CLOSED

Oh ... i just saw, I already posted this a month ago in issue #21
If we want to continue our project, we need a fix for this. As you can see - this happens with MANY (most?) models. Could we talk about this for 10 minutes in the next days? In the best case we can find a technical soluation, which Jakob could implement?

from mlrmbo.

Well, the calls you posted above are faulty. predict.type is an argument of makeLearner and not makeBaggingWrapper. Moreover none of the learners you used in your example does support the prediction of standard error. The following runs fine:

library(mlrMBO)
par.set = makeParamSet(
  makeNumericVectorParam("x", len = 5, lower = 0, upper = 1),
  makeDiscreteParam("z", values = 1:10)
)
f = function(x) sum(x$x) + as.numeric(x$z)
learner = makeBaggingWrapper(makeLearner("regr.nnet"), 10L)
control =  makeMBOControl( init.design.points = 20, iters = 10)
res = mbo(f, par.set, learner = learner, control = control)

from mlrmbo.

I haven't looked at this for months ... i don't know if I posted some crap or if mlr changed, whatever. Good, if it runs fine.

But after applying the bagging wrapper every learner should be able to predict the se?
That's the whole reason for us to use the bagging wrapper - to predict the se and not only the mean response. This should also work somehow ..? We want to use for example a bagged kknn for mbo.

from mlrmbo.

Ok, now I unterstand what you calls should look like. You have to set the predict.type of the wrapped model to se via the setPredictType method. The following code makes what you want and runs without any problems now.

library(mlrMBO)
par.set = makeParamSet(
  makeNumericVectorParam("x", len = 5, lower = 0, upper = 1),
  makeDiscreteParam("z", values = 1:10)
)
f = function(x) sum(x$x) + as.numeric(x$z)
learner = makeBaggingWrapper(makeLearner("regr.nnet"), 10L)
learner = setPredictType(learner, "se")
control =  makeMBOControl( init.design.points = 20, iters = 2)
res = mbo(f, par.set, learner = learner, control = control)

from mlrmbo.

Stupid me ... sometimes it's really helpfull to use code that actually works instead of just using code that looks like it could do what you want it to. Cool, I will have a look at some things today.

So we have 1 extra line code we always have to use here. Perhaps we could talk about how to save this 1 line, but on the other hand it's only 1 line.

from mlrmbo.

Stupid me ... sometimes it's really helpfull to use code that actually works instead of just using code
that looks like it could do what you want it to. Cool, I will have a look at some things today.

Shit happens 😄

I think we can close this, because we discuss this already in #21.

So we have 1 extra line code we always have to use here. Perhaps we could talk about how to save
this 1 line, but on the other hand it's only 1 line.

This should be made a mlr issue.

from mlrmbo.

1. predict.type is not an arg of bagging wrapper because I don't want to bloat the interface.

2. I don't have any unit tests that fail currently. I will use the BaggingWrapper now too in test_different learners and close this.

Post more in #21 or open more issues if you have more models that break.

from mlrmbo.