Classifier predict numerical precision issue with large raw_score about lightgbm HOT 4 OPEN

Thanks for using LightGBM.

Are you able to share some minimal code showing precisely what you mean? I'm unsure how to interpret some of these statements like "a very large number with value 0 or value 1".

from lightgbm.

I cannot share much in the way of specifics, but here is the workflow:

# X = features, y = known outcome

model_pipeline = Pipeline(
    lightgbm.LGBMClassifier(
        objective="binary",
        boosting="rf",
        n_estimators=1000,
        # etc., mostly default values
    )
)

model_pipeline = model_pipeline.fit(X, y)

scores = model_pipeline.predict_proba(X)[:,1]

scores are distributed from about 1e-5 to 1.0 in a bowl shape (high population at the min and max), with a large quantity having a value of exactly 1.0 (loss of ranking information).

If instead I look at:

scores_raw = model_pipeline.predict_proba(X, raw_score=True)

Then scores_raw is distributed from about -11000 to +136000 with a shape more resembling a decaying exponential, and no repeated values at the max score (no loss of ranking information). I can apply the basic sigmoid function to scores_raw to regain scores, which illustrates the numerical precision limit on the upper end. If the scores_raw distribution was first scaled down to something close to 1, the sigmoid distribution would not run into numerical precision limitations. Then the score ranking could again be used as intended.

As I mentioned, a previous version of lightgbm did not behave in the current way, avoiding this problem.

from lightgbm.

Ok, so to clarify:

• you are not actually using Random Forest mode? (boosting="rf")
• you are using the built-in binary loss function (objective="binary")?

from lightgbm.

Apologies, yea I am using boosting="rf", I have edited my previous comment to include that. I am also using the built-in binary loss function.

from lightgbm.