Baseline results about meld HOT 9 OPEN

I get the same results as @stdoo above and am unable to reproduce the results in the paper in using bcLSTM model to classify Emotion with text/audio/text+audio features.

If I do the Sentiment classification, the F1-scores are closer to the values listed in Table 13, but do not match them exactly.

I am curious to know what was done differently in obtaining the results shown in the paper.

from meld.

I also tried setting the class weights as described in the README:
class_weight = {0:4.0, 1:15.0, 2:15.0, 3:3.0, 4:1.0, 5:6.0, 7:3.0}
but still not able to reproduce the results

from meld.

Hi,

Thanks for notifying this issue. Previously, we have had users who have been able to recreate our results and improve upon them. Updates in the dependent softwares might be a reason behind this.

Nevertheless, class weights were the main strategy we used in providing the baseline results in the paper. We encourage you to try out other variations of these weights too. Meanwhile, we will try to update them as per new packages.

from meld.

@devamanyu can you provide the versions of the dependent software you are referring to.

from meld.

@devamanyu I get the same problem with @stdoo and I also tried setting the class weight as described in the README (like @sanzgiri ). Can you provide the versions of the dependent softwares?

from meld.

@devamanyu I get the same problem with @stdoo and I also tried setting the class weight as described in the README (like @sanzgiri ). Can you provide the versions of the dependent softwares?

from meld.

Emotion - {'neutral': 0, 'surprise': 1, 'fear': 2, 'sadness': 3, 'joy': 4, 'disgust': 5, 'anger': 6}.

Dear @stdoo ，could you please give me some guidance about how to fix it? Thanks

from meld.

Hi,

Thanks for notifying this issue. Previously, we have had users who have been able to recreate our results and improve upon them. Updates in the dependent softwares might be a reason behind this.

Nevertheless, class weights were the main strategy we used in providing the baseline results in the paper. We encourage you to try out other variations of these weights too. Meanwhile, we will try to update them as per new packages.

@devamanyu ， I‘m sorry to disturb you. I'm new to tensorflow, and I don't have cs/ee background. You say the main strategy is to adjust the class weights. However, I don't know how to do it. To be more specific, Which file should I check to modify the weights, what codes do I need to modify?

Thanks.

from meld.

Hi, I have tried the class_weight strategy but found that class_weight in keras must be a dict, not a array. No matter the latest version or the 2.0.2 version your work used.
https://github.com/keras-team/keras/blob/576f8fe8e6a21b7094316d36c315c2f6bdb487cc/keras/engine/training.py#L557

I doubt whether you have tried the strategy. If so, could you give a more specific hint? (like how to change the code to use the class_weight strategy) Thanks a lot.

from meld.