model fit not using GPU about deepconv-dti HOT 10 CLOSED

Hello, Angela Lopez

Thank you for rasing issue

DeepConv-DTI model is built on Keras and Tensorflow backend.

And rest of them is just wrapper for convinient usage of Model.

Would you please check your tensorflow package is based on gpu?

Would you specify your python environment?

Thank you

from deepconv-dti.

Hi,
thank you for your quick response.
As for your questions, I've created a virtual environment specifically for using this package. These are the packages I have installed in the environment (obtained by typing pip3 freeze from within the environment):

absl-py==0.9.0
astor==0.8.1
bleach==1.5.0
gast==0.3.3
google-pasta==0.2.0
grpcio==1.29.0
h5py==2.10.0
html5lib==0.9999999
importlib-metadata==1.6.0
joblib==0.15.1
Keras==2.1.6
Keras-Applications==1.0.8
Keras-Preprocessing==1.1.2
Markdown==3.2.2
numpy==1.18.4
pandas==1.0.3
pkg-resources==0.0.0
protobuf==3.12.1
python-dateutil==2.8.1
pytz==2020.1
PyYAML==5.3.1
scikit-learn==0.23.1
scipy==1.4.1
six==1.15.0
sklearn==0.0
tensorboard==1.14.0
tensorflow-estimator==1.14.0
tensorflow-gpu==1.14.0
termcolor==1.1.0
threadpoolctl==2.0.0
Werkzeug==1.0.1
wrapt==1.12.1
zipp==3.1.0

As you can see, I have tensorflow-gpu installed, but no tensorflow. When I ran the model with my data.

This is the command I'm running:
python3 DeepConvDTI.py ./datos_deepaffinity/train_dti.csv ./datos_deepaffinity/comp_data.csv ./datos_deepaffinity/prot_data.csv --validation -n val_dataset -i ./datos_deepaffinity/val_dti.csv -d ./datos_deepaffinity/comp_data.csv -t ./datos_deepaffinity/prot_data.csv -W --drug-layers 512 128 --window-sizes 10 15 20 25 30 --protein-layers 128 --fc-layers 256 --learning-rate 0.0001 -n 30 --prot-vec Convolution --prot-len 1500 -V fingerprint --drug-len 881 --dropout 0 --activation elu --n-filters 128 --batch-size 128 --decay 0.0001 --output ./results_deepaffinity/0/validation_output.csv --save-model .results_deepaffinity/0/model.model --n-epoch 150

And what I'm obtaining is a model which takes 4h for running an epoch....

Hope you can help me. Thank you in advance

P.S.: I'm sorry, I closed the issue by mistake and had to open it again.

from deepconv-dti.

Hello,

Would you please show me installed package list by using command

pip list

from deepconv-dti.

Hello,

Would you please show me installed package list by using command

pip list

absl-py (0.9.0)
astor (0.8.1)
bleach (1.5.0)
gast (0.3.3)
google-pasta (0.2.0)
grpcio (1.29.0)
h5py (2.10.0)
html5lib (0.9999999)
importlib-metadata (1.6.0)
joblib (0.15.1)
Keras (2.1.6)
Keras-Applications (1.0.8)
Keras-Preprocessing (1.1.2)
Markdown (3.2.2)
numpy (1.18.4)
pandas (1.0.3)
pip (9.0.1)
pkg-resources (0.0.0)
protobuf (3.12.1)
python-dateutil (2.8.1)
pytz (2020.1)
PyYAML (5.3.1)
scikit-learn (0.23.1)
scipy (1.4.1)
setuptools (47.1.0)
six (1.15.0)
sklearn (0.0)
tensorboard (1.14.0)
tensorflow-estimator (1.14.0)
tensorflow-gpu (1.14.0)
termcolor (1.1.0)
threadpoolctl (2.0.0)
Werkzeug (1.0.1)
wheel (0.34.2)
wrapt (1.12.1)
zipp (3.1.0)

from deepconv-dti.

Hello, I don't know why tensorflow-gpu doesn't work in your environment.

I'm sorry to ask, but would you check whether GPU driver and CUDA(+CUDNN) interface is properly set in your virtual environment? Or would you run GPU-checking script in virtual environment?

thank you.

from deepconv-dti.

Hi,
sorry for the delay in the answer but I have been trying different alternatives (none of them worked).

I also think that there is a compatibility problem between CUDA and tensorflow versions. I have CUDA version 10.2 installed and the environment in which I usually work has tensorflow 2.1.0 (in this environment models run on the GPUs and I don't have any problems). However, I cannot run DeepConv-DTI with tensorflow 2.1.0. And now I'm trying to find a version of tensorflow-gpu that is compatible with CUDA 10.2 and also with DeepConv-DTI.

Could you please tell me the CUDA and tensorflow-gpu versions in which you have succesfully run DeepConv-DTI?

Thank you in advance.

from deepconv-dti.

Hello.

I think this problem comes from tensorflow mismatches.

I used tensorflow 1.12 with keras.

As you know, tensorflow 1.xx code is not much compatible with tensorflow 2.x.

But I wonder that when you show me your package list with pip list your tf is 1.14.0, not 2.1.0.

Would you try running DeepConv-DTI with tensorflow 1.12.0 and keras 2.2.4?

from deepconv-dti.

Hi,

sorry, I think you misunderstood me. What I was saying is that I usually work with tensorflow 2.1.0 because it is compatible with my CUDA 10.2. The first time I tried running DeepConv-DTI, I tried it with my usual environment but it didn't work. You only specify as requirement that tensorflow must be >1.0, but it is not stated anywhere that tensorflow should also be <2.0 and I think you should add it.

I then created a virtual environment specifically for running DeepConv-DTI with an older version of tensorflow-gpu (tensorflow-gpu==1.14.0, as I showed you). The problem now is a mismatch between tensorflow AND CUDA. tensorflow-gpu 1.14 (or 1.12 or whatever other version I've tried until now) does not match with CUDA 10.2 and that's why it's not using GPUs. I cannot find a combination of tensorflow-gpu compatible with CUDA 10.2 that works with DeepConv-DTI.

I will try to set up an environment in another machine with a different version of CUDA. Again, could you please tell me which CUDA version are you using?

Thank you.

from deepconv-dti.

Hello,
Firstly, I'm sorry for misunderstanding what you said.
And my CUDA version is 10.2 while my tensorflow version is 1.12.0
However, actually I used CUDA version with 9.x recently I updated CUDA to 10.2, and it works.

As you pointed, I think compatibility between tensorflow and CUDA is the cause of this problem.

And I will specify tensorflow version.

I'm sorry for you have had this kind of problem and thank you for raising issue and suggestion for version specification.

From Ingoo Lee.

from deepconv-dti.

I will close this issue.

from deepconv-dti.