Code Monkey home page Code Monkey logo

aucoresnet's Introduction

aucoresnet

OFFICIAL REPOSITORY

AUCO ResNet: an end-to-end network for Covid-19 pre-screening from cough and breath

paper

AUCO ResNet: an end-to-end network for Covid-19 pre-screening from cough and breath

Cite

Dentamaro, V., Giglio, P., Impedovo, D., Moretti, L., & Pirlo, G. (2022). AUCO ResNet: an end-to-end network for Covid-19 pre-screening from cough and breath. Pattern Recognition, 108656. doi:10.1016/j.patcog.2022.108656

Highlights

  • The Auditory Cortex ResNet, briefly AUCO ResNet, is proposed and tested. It is a deep neural network architecture especially designed for audio classification trained end-to-end. It is inspired by the architectural organization of rat's auditory cortex, containing also innovations 2 and 3. The network outperforms the state-of-the-art accuracies on a reference audio benchmark dataset without any kind of preprocessing, imbalanced data handling and, most importantly, any kind of data augmentation.

  • A trainable Mel-like spectrogram layer able to finetune the
    Mel-like-Spectrogram for capturing relevant time frequency
    information.

  • A novel sinusoidal learnable attention mechanism which can be
    considered as a technique to weight local and global feature
    descriptors focusing on high frequency details.

  • State of the art cross-dataset testing and related accuracies.

aucoresnet's People

Contributors

vincenzodentamaro avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar

aucoresnet's Issues

Invalid dtype: complex64

Hello!When I run your jupyter notebook,I have met an error.The details are as followes:

RuntimeError Traceback (most recent call last)
Cell In[20], line 176
172 print('AUCS len', len(aucs))
173 #Layer 1 = 13% , layer 2 = 11%, Layer 3 17%, Layer 4 10%, Layer 5 26%, Layer 6 22%
--> 176 AUCOResNetV2_ = AUCOResNetV2(att='att3',gmode='concat',sample_rate = SAMPLING_RATE,compatibilityfunction='cp',#cp, mha, dp
177 datasetname='covid',input_shape=input_shape,
178 outputclasses=2,runs=[1,8,3,7,2],n_mels=150, win_length=140, hop_length=344, strides=(3,3),
179 n_fft=fft, sinusoidal = sinusoidal, trainable=True,
180 metrics = ['accuracy', 'AUC'], pers_act = pers_act,attention_type=attention_type, optimizer=optimizer,
181 subsample_initial_block=subsample, initial_kernel=kernel_size, debug=False)#78 auc medio con cbam
184 AUCOResNetV2_.StandardFit(datasetname='covid',X=X_train.real,Y=y_train.real,patience=5,validation_data=(X_test,y_test),
185 batch_size=batch_size,epochs=100,checkpointcall = model_checkpoint_callback)
188 model2 = AUCOResNetV2_.model

Cell In[18], line 611
601 #Layer 1
602 '''
603 if trainable:
604 fft_base_2 = pow(2, math.ceil(math.log(n_fft)/math.log(2)));
(...)
609 else:
610 '''
--> 611 x = self.get_melspectrogram_layer(name='mel',n_fft=n_fft,sample_rate=sample_rate,n_mels=n_mels, hop_length=hop_length, return_decibel=return_decibel,input_data_format=input_data_format, trainable = trainable, num_classes=outputclasses) (inputs)
613 if debug:
...
Invalid dtype: complex64

Arguments received by STFT.call():
• args=('<KerasTensor shape=(None, 1, 330750), dtype=float32, sparse=None, name=keras_tensor_3>',)
• kwargs=<class 'inspect._empty'>

Could you like help me?

Cannot get tht same accuracy.

Nice Work, I use your code and Coswara-Data, but I just get 0.63 AUC with finetuning parameters. I check the training log, and I found the model overfit. Is your code complete?

How to output the Trained Mel-like spectrogram?

Thank you the share!

I have a questions. How to output the Trained Mel-like spectrogram? When I tried to do it, it got an error “Graph disconnected: cannot obtain value for tensor KerasTensor(type_spec=TensorSpec(shape=(None, 1, 330750), dtype=tf.float32, name='stft_5_input'), name='stft_5_input', description="created by layer 'stft_5_input'") at layer "stft_5". The following previous layers were accessed without issue: []”.

'mel' layer

Thanks you very much for sharing the code!

I have a question: why is there no trainable parameter in 'mel' layer?
Isn't this layer the 'trainable Mel Spectrogram Layer'?
Please see following picture.
model summary

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.