In dp_rmse.py, the lines

noise = rng.standard_cauchy()
dp_rmse = cast(float, rmse + 2 * (gamma + 1) * sens * noise / epsilon)

may yield (large) negative values – unreasonable for rMSE loss values.
These may be some viable counter measures:

1. Simply skip new trees having negative associated loss
2. Keep track of recent losses to detect outlier (not necessarily just negative) losses, skip those
3. After encountering a negative loss, clip it to 0 and raise the comparison threshold prev_loss < current_loss according to a (yet to be determined) schedule step by step, so that new trees get a chance to join the ensemble again (which would otherwise be highly unlikely).

expose it to the end user

I could use these for example:

1. https://www.kaggle.com/maajdl/yeh-concret-data
2. https://www.kaggle.com/harlfoxem/housesalesprediction
3. https://www.kaggle.com/mirichoi0218/insurance

Integrate https://github.com/loretanr/dp-gbdt by writing a wrapper that exposes DPEnsemble to python, implementing the scikit learn estimator interface.

But only if not following the post processing approach mentioned in #16.
Add a sensitivity variant in dm_rmse.py and provide an (eps, delta)-dp loss function class in losses.py

Filter -> Allocate -> Put filtered data back into pool -> repeat
But keep in mind, that clipping might be required for the last couple of trees, because the data pool might be not large enough to provide sufficient data points for each tree.

That way finding the split point is less dependent on the data (the status quo is to use the data values directly to determine the best split)

Relevant section in the C++-Code:

// sensitivity for leaves
if (params->gradient_filtering && !params->leaf_clipping) {
    // you can only "turn off" leaf clipping if GDF is enabled!
    tree_params.delta_v = params->l2_threshold / (1 + params->l2_lambda);
} else {
    tree_params.delta_v = std::min((double) (params->l2_threshold / (1 + params->l2_lambda)),
            2 * params->l2_threshold * pow(1-params->learning_rate, tree_index));
}

It is strange that the rMSE reaches a local minimum roughly at 0.3 and then increases progressively. I would have expected that with increasing epsilon, the loss decreases.

TODO: Observe single runs at epsilon = 0.3 (outside of GridSearchCV) and view the logs.

Since in other parts of the implementation it is assumed that the gradients are within a specific range.