The competitions's discuss from humeai

Calculation of Concordance Correlation Coefficient (CCC) may be incorrect

Hi, there! I think the calculation of Concordance Correlation Coefficient (CCC) given in the baseline codes may be incorrect. The unbiased estimation of covariance should be cov = np.nansum((y_true - x_mean) * (y_pred - y_mean)) / (len(y_true) - 1) instead of mean. e.g.,

x = torch.rand(16, 10)  # (bsz, n_classes)
y = x
=> CCC_given_in_utils = 0.0938  # error
=> CCC_corrected = 1.0  # corrected

Reference:

https://github.com/audeering/audtorch/blob/0.1.1/audtorch/metrics/functional.py#L83
https://en.wikipedia.org/wiki/Covariance
Please also see a correction of the CCC form from the original author Lin in Reference [5] of Wikipedia.

Error on baseline Multitask with DeepSpectrum

Hi ExVo team,

I got an error when training the multitask code with DeepSpectrum features. Other features are OK.
Here are the command and error message.

(ExVo2022) pc060066:ExVo-MultiTask$ python3 main.py -d /data/22_ICML-ExVo22_TeamAtmaja_ITSN/ -l data_info.csv -e 20 -f DeepSpectrum -tn baseline --n_seeds 1 -p 5
ltloss
No stored files found, creating from scratch ...
100%|█████████████████████████████████████| 59201/59201 [15:53<00:00, 62.10it/s]
Saving data ...
Running Model for 1 seeds
Running experiments with DeepSpectrum
Seed 42 | 0.001 | Batch Size 8 | Epochs 20
Traceback (most recent call last):
  File "main.py", line 382, in <module>
    main()
  File "main.py", line 307, in main
    model, hmean, metrics = baseline(
  File "main.py", line 100, in baseline
    age_mae, country_uar, y_country, age_ccc, train_val, loss = train(
  File "/data/github/ExVo2022/ExVo-MultiTask/train.py", line 52, in train
    y_pred = model(inputs_X)
  File "/home/bagus/miniconda3/envs/ExVo2022/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
    return forward_call(*input, **kwargs)
  File "/data/github/ExVo2022/ExVo-MultiTask/models.py", line 38, in forward
    h_shared = self.share_layer(x)
  File "/home/bagus/miniconda3/envs/ExVo2022/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/bagus/miniconda3/envs/ExVo2022/lib/python3.8/site-packages/torch/nn/modules/container.py", line 139, in forward
    input = module(input)
  File "/home/bagus/miniconda3/envs/ExVo2022/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/bagus/miniconda3/envs/ExVo2022/lib/python3.8/site-packages/torch/nn/modules/linear.py", line 96, in forward
    return F.linear(input, self.weight, self.bias)
  File "/home/bagus/miniconda3/envs/ExVo2022/lib/python3.8/site-packages/torch/nn/functional.py", line 1847, in linear
    return torch._C._nn.linear(input, weight, bias)
RuntimeError: mat1 and mat2 shapes cannot be multiplied (8x4096 and 4095x128)

humeai / competitions Goto Github PK

competitions's Issues

Calculation of Concordance Correlation Coefficient (CCC) may be incorrect

Error on baseline Multitask with DeepSpectrum

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent