Comments (12)
@braindancer Agreed! @toosyou 's implementation is wrong. The focal weights should be adjusted pixel by pixel given each pixel's prediction and ground truth labels.
In summary, to make it more stable, one should:
- Train the model use cross entropy loss for a few epochs first;
- Add "y_pred = K.clip(y_pred, eps, 1-eps)".
from focal-loss-keras.
Add in clipping to y_pred to be between (eps,1-eps)
from focal-loss-keras.
I think it's because the calculation of cross_entropy is not stable.
I am using
def focal_loss_fixed(y_true, y_pred):
y_true = K.flatten(y_true)
y_pred = K.flatten(y_pred)
alpha_factor = keras.backend.ones_like(y_true) * alpha
alpha_factor = tf.where(keras.backend.equal(y_true, 1), alpha_factor, 1 - alpha_factor)
focal_weight = tf.where(keras.backend.equal(y_true, 1), 1 - y_pred, y_pred)
focal_weight = alpha_factor * focal_weight ** gamma
cls_loss = focal_weight * K.binary_crossentropy(y_true, y_pred)
return K.sum(cls_loss)
then it's working fine.
from focal-loss-keras.
same problem, nan when training.
from focal-loss-keras.
very very not stable ,it is easy to turns to Nan
from focal-loss-keras.
It looks like bad initialization is the problem. What works for me is to run 2 epochs with binary_crossentropy from keras.losses, and then compile model again, using focal loss and train until convergence.
from focal-loss-keras.
You can try alpha=0.25. This make my training much more stable.
from focal-loss-keras.
I've tried adding clipping as suggested above and using alpha=0.25, but sometimes it still fails.
Why default value for alpha is 2? it makes no sense, because it should be less than 1 to weight values properly
from focal-loss-keras.
I've set the alpha=.25
as in the paper.
from focal-loss-keras.
I am still getting nan after a few epochs even with alpha=0.25 and i even added k.epsilon() in both logs.
Any fix recommended?
from focal-loss-keras.
@sarim-zafar I tried to find what causes this nans, but didn't succeed, maybe you can find it out
from focal-loss-keras.
@toosyou I believe yours is not the focal loss implemention since binary_crossentropy is already a scalar
from focal-loss-keras.
Related Issues (18)
- Unknown loss function:focal_loss_fixed HOT 6
- how to implemented it in faster rcnn? HOT 2
- AttributeError: 'function' object has no attribute 'get_shape'
- Apply in multi-label task HOT 3
- 'function' object has no attribute 'get_shape' HOT 5
- Forgot tensorflow import HOT 2
- Nan problem for LOG HOT 6
- Focal Loss is Huge HOT 3
- Multi-label classification HOT 1
- Can you provide a multi-label version about the focal-loss-keras? thanks
- can this loss function be applied to training where output is one-hot encoding?
- How to finetune alpha and gamma?
- is there a version that works with theano backend HOT 1
- alpha is not added in correctly HOT 2
- there are some problems about default alpha value HOT 1
- AttributeError when apply this loss HOT 2
- Should we use Sigmoid instead Softmax? HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from focal-loss-keras.