jrzaurin / lightgbm-with-focal-loss Goto Github PK

Hi. Thanks for this implementation.
I had a problem:

/var/folders/tx/5cvg8b910f31k6ttxp5ds9cxbmnldx/T/ipykernel_14194/2624284624.py:26: RuntimeWarning: overflow encountered in exp
  p = 1/(1+np.exp(-x))
/var/folders/tx/5cvg8b910f31k6ttxp5ds9cxbmnldx/T/ipykernel_14194/2624284624.py:27: RuntimeWarning: divide by zero encountered in log
  return -( a*t + (1-a)*(1-t) ) * (( 1 - ( t*p + (1-t)*(1-p)) )**g) * ( t*np.log(p)+(1-t)*np.log(1-p) )
/var/folders/tx/5cvg8b910f31k6ttxp5ds9cxbmnldx/T/ipykernel_14194/2624284624.py:27: RuntimeWarning: invalid value encountered in multiply
  return -( a*t + (1-a)*(1-t) ) * (( 1 - ( t*p + (1-t)*(1-p)) )**g) * ( t*np.log(p)+(1-t)*np.log(1-p) )

Do you know what could it be?
The problem is a multiclass classification with 30 classes.

great post and code
but what is about
scale_pos_weight is used to adjust the imbalance will spoil probability?

microsoft/LightGBM#2107

scikit-learn/scikit-learn#14735

by the way
did you found really unbalanced data?

hi, when I use lightgbm with focal loss, it can not open multithreading. The CPU only use 100%.

Hey there! Just wanted to let you know I put some work into deriving exact formulas, which avoids the need for numeric differentiation. I also found out that to be correct, the initialization value of the gradient boosting algorithm has to be accounted for. I've written about this in a blog post.

Kind regards :)

Can you specify a license for this repository?

I am using my own dataset with X and Y read in as Pandas dataframes (followed by train-valid-split)

I'm getting the following error in best.
best = fmin(fn=objective,
space=space,
algo=tpe.suggest,
max_evals=5,
trials=trials)
ValueError Traceback (most recent call last)
in
7 algo=tpe.suggest,
8 max_evals=5,
----> 9 trials=trials)

--> 729 self.pandas_categorical)
730 label = _label_from_pandas(label)
731 self.data_has_header = False

~\Anaconda3\envs\tensorflow-gpu+SHAP\lib\site-packages\lightgbm\basic.py in _data_from_pandas(data, feature_name, categorical_feature, pandas_categorical)
245 if len(data.shape) != 2 or data.shape[0] < 1:
246 raise ValueError('Input data must be 2 dimensional and non empty.')
--> 247 if feature_name == 'auto' or feature_name is None:
248 data = data.rename(columns=str)
249 cat_cols = data.select_dtypes(include=['category']).columns

all of my features have names; the feature_names are
Index(['acft_make_CESSNA', 'acft_make_PIPER', 'acft_make_BEECH',
'acft_make_LUSCOMBE', 'acft_make_PITTS', 'acft_make_CHAMPION',
'acft_make_MOONEY', 'acft_make_NORTH AMERICAN', 'acft_make_BELLANCA',
'acft_make_AERONCA',
...
'pc_profession_ENGR', 'pc_profession_DOCD', 'pc_profession_PMIL',
'pc_profession_OCP ', 'pc_profession_FARA', 'pc_profession_UNK',
'pc_profession_N ', 'pc_profession_Y ', 'pc_profession_NOCP',
'pc_profession_CLGY'],
dtype='object', length=790)

Any ideas to what might be wrong?