Code Monkey home page Code Monkey logo

cardio's Introduction

CardIO

CardIO is designed to build end-to-end machine learning models for deep research of electrocardiograms.

Main features:

  • load and save signals in various formats: WFDB, DICOM, EDF, XML (Schiller), etc.
  • resample, crop, flip and filter signals
  • detect PQ, QT, QRS segments
  • calculate heart rate and other ECG characteristics
  • perform complex processing like fourier and wavelet transformations
  • apply custom functions to the data
  • recognize heart diseases (e.g. atrial fibrillation)
  • efficiently work with large datasets that do not even fit into memory
  • perform end-to-end ECG processing
  • build, train and test neural networks and other machine learning models

For more details see the documentation and tutorials.

About CardIO

CardIO is based on BatchFlow. You might benefit from reading its documentation. However, it is not required, especially at the beginning.

CardIO has three modules: core, models and pipelines.

core module contains EcgBatch and EcgDataset classes. EcgBatch defines how ECGs are stored and includes actions for ECG processing. These actions might be used to build multi-staged workflows that can also involve machine learning models. EcgDataset is a class that stores indices of ECGs and generates batches of type EcgBatch.

models module provides several ready to use models for important problems in ECG analysis:

  • how to detect specific features of ECG like R-peaks, P-wave, T-wave, etc
  • how to recognize heart diseases from ECG, for example, atrial fibrillation

pipelines module contains predefined workflows to

  • train a model and perform an inference to detect PQ, QT, QRS segments and calculate heart rate
  • train a model and perform an inference to find probabilities of heart diseases, in particular, atrial fibrillation

Basic usage

Here is an example of a pipeline that loads ECG signals, makes preprocessing and trains a model for 50 epochs:

train_pipeline = (
  ds.Pipeline()
    .init_model("dynamic", DirichletModel, name="dirichlet", config=model_config)
    .init_variable("loss_history", init_on_each_run=list)
    .load(components=["signal", "meta"], fmt="wfdb")
    .load(components="target", fmt="csv", src=LABELS_PATH)
    .drop_labels(["~"])
    .rename_labels({"N": "NO", "O": "NO"})
    .flip_signals()
    .random_resample_signals("normal", loc=300, scale=10)
    .random_split_signals(2048, {"A": 9, "NO": 3})
    .binarize_labels()
    .train_model("dirichlet", make_data=concatenate_ecg_batch, fetches="loss", save_to=V("loss_history"), mode="a")
    .run(batch_size=100, shuffle=True, drop_last=True, n_epochs=50)
)

Installation

CardIO module is in the beta stage. Your suggestions and improvements are very welcome.

CardIO supports python 3.5 or higher.

Installation as a python package

With pipenv:

pipenv install git+https://github.com/analysiscenter/cardio.git#egg=cardio

With pip:

pip3 install git+https://github.com/analysiscenter/cardio.git

After that just import cardio:

import cardio

Installation as a project repository

When cloning repo from GitHub use flag --recursive to make sure that batchflow submodule is also cloned.

git clone --recursive https://github.com/analysiscenter/cardio.git

Citing CardIO

Please cite CardIO in your publications if it helps your research.

DOI

Khudorozhkov R., Illarionov E., Kuvaev A., Podvyaznikov D. CardIO library for deep research of heart signals. 2017.
@misc{cardio_2017_1156085,
  author       = {R. Khudorozhkov and E. Illarionov and A. Kuvaev and D. Podvyaznikov},
  title        = {CardIO library for deep research of heart signals},
  year         = 2017,
  doi          = {10.5281/zenodo.1156085},
  url          = {https://doi.org/10.5281/zenodo.1156085}
}

cardio's People

Contributors

alexanderkuvaev avatar dpodvyaznikov avatar roman-kh avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

cardio's Issues

Cardio with TF 1.14

Hi,

I am running Cardio dirichlet_model with Tensorflow 1.14. While there are many warnings (which can be fixed), I am not going past test pipeline with the following error.

ValueError: Cannot feed value of shape (4, 1) for Tensor 'DirichletModel/targets:0', which has shape '(?, 2)'

I have not made any changes to the code in the dirichlet_model Python notebook and trying to run it as is..How do I fix this?

Many thanks
~anoop

pipenv installation doesn't work

Microsoft Windows [Version 10.0.18362.295]
(c) 2019 Microsoft Corporation. All rights reserved.

C:\Users\gsm20>pip install cardio
Collecting cardio
ERROR: Could not find a version that satisfies the requirement cardio (from versions: none)
ERROR: No matching distribution found for cardio

C:\Users\gsm20>pipenv install git+https://github.com/analysiscenter/cardio.git#egg=cardio
Installing git+https://github.com/analysiscenter/cardio.git#egg=cardio…
Warning: You installed a VCS dependency in non-editable mode. This will work fine, but sub-dependencies will not be resolved by $ pipenv lock.
To enable this sub-dependency functionality, specify that this dependency is editable.
Collecting cardio from git+https://github.com/analysiscenter/cardio.git#egg=cardio
Cloning https://github.com/analysiscenter/cardio.git to c:\users\gsm20\appdata\local\temp\pip-install-r0n92tbz\cardio
Collecting numpy>=1.13.1 (from cardio)
Downloading https://files.pythonhosted.org/packages/cb/41/05fbf6944b098eb9d53e8a29a9dbfa20a7448f3254fb71499746a29a1b2d/numpy-1.17.1-cp37-cp37m-win_amd64.whl (12.8MB)
Collecting scipy>=0.19.1 (from cardio)
Using cached https://files.pythonhosted.org/packages/50/eb/defa40367863304e1ef01c6572584c411446a5f29bdd9dc90f91509e9144/scipy-1.3.1-cp37-cp37m-win_amd64.whl
Collecting pandas>=0.21.1 (from cardio)
Using cached https://files.pythonhosted.org/packages/b1/69/fcc29820befae2b96fd0b01225577af653e87cd0914634bb2d372a457bd7/pandas-0.25.1-cp37-cp37m-win_amd64.whl
Collecting scikit-learn==0.19.1 (from cardio)
Using cached https://files.pythonhosted.org/packages/f5/2c/5edf2488897cad4fb8c4ace86369833552615bf264460ae4ef6e1f258982/scikit-learn-0.19.1.tar.gz
Collecting numba>=0.35.0 (from cardio)
Using cached https://files.pythonhosted.org/packages/dd/89/861d76f7a677fc5ba7fafa8085017c43b76e3880e9e365f863f8cc800a46/numba-0.45.1-cp37-cp37m-win_amd64.whl
Collecting pywavelets>=0.5.2 (from cardio)
Using cached https://files.pythonhosted.org/packages/71/57/1b6c06f2b681975e5b43ff13ba127031f668d65b592cc56fae6445b675c7/PyWavelets-1.0.3-cp37-none-win_amd64.whl
Collecting matplotlib>=2.1.0 (from cardio)
Using cached https://files.pythonhosted.org/packages/1a/c0/69e3f695d7384012e90be1e16570c08953baae00fd98094179ef87c7d5a2/matplotlib-3.1.1-cp37-cp37m-win_amd64.whl
Collecting dill>=0.2.7.1 (from cardio)
Using cached https://files.pythonhosted.org/packages/39/7a/70803635c850e351257029089d38748516a280864c97cbc73087afef6d51/dill-0.3.0.tar.gz
Collecting pydicom>=0.9.9 (from cardio)
Using cached https://files.pythonhosted.org/packages/fc/d5/da1fdf3b967e324ee47a7ad9553c9b94c1193b6b98afd9eeda0efb76b9f7/pydicom-1.3.0-py2.py3-none-any.whl
Collecting pyedflib>=0.1.11 (from cardio)
Using cached https://files.pythonhosted.org/packages/a4/c1/11d90aa4d20a1d3df04ea9e1db4e03fea4b199a5b515a057cbe5e35769c0/pyEDFlib-0.1.14.tar.gz

Error: An error occurred while installing git+https://github.com/analysiscenter/cardio.git#egg=cardio!
b' Running command git clone -q https://github.com/analysiscenter/cardio.git 'C:\Users\gsm20\AppData\Local\Temp\pip-install-r0n92tbz\cardio'\n Running command git submodule update --init --recursive -q\n ERROR: Command errored out with exit status 4294967295:\n command: 'c:\users\gsm20\.virtualenvs\gsm20-sprtm5uw\scripts\python.exe' -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\gsm20\\AppData\\Local\\Temp\\pip-install-r0n92tbz\\pyedflib\\setup.py'"'"'; file='"'"'C:\\Users\\gsm20\\AppData\\Local\\Temp\\pip-install-r0n92tbz\\pyedflib\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' egg_info --egg-base pip-egg-info\n cwd: C:\Users\gsm20\AppData\Local\Temp\pip-install-r0n92tbz\pyedflib\\n Complete output (4 lines):\n No module named 'numpy'\n *** package "numpy" not found ***\n pyEDFlib requires a version of NumPy, even for setup.\n Please get it from http://numpy.scipy.org/ or install it through your package manager.\n ----------------------------------------\nERROR: Command errored out with exit status 4294967295: python setup.py egg_info Check the logs for full command output.\n'
This is likely caused by a bug in cardio. Report this to its maintainers.
Installation

i am trying to install using cmd( i am using windows 10 and python3.7)
but i get this error..

can you tell me why..? how can i fix it??

and can i use zip file and comman python setup.py install??

I.CardIO.ipynb Error in line [6]

batch_with_data = batch_with_data.load(src='../cardio/tests/data/REFERENCE.csv', fmt='csv', components='target')
have to replace to
batch_with_data = batch.load(src='../cardio/tests/data/REFERENCE.csv', fmt='csv', components='target')

How to pass train and validation data together?

Hello,

Typically in Keras, when we use fit_generator(), I can pass both training and validation batch generators while tracking validation accuracy.

Is there a way I can do this using batchflow for example? I see all examples suggest we use (dataset.train >> pipeline).run() only. Is there a way of passing dataset.train and dataset.test to KerasModel to get validation accuracy during training?

thanks in advance,
~anoop

different predictions for the same eds.test

Hi! I set np.random.seed for the same random number in random_split_signals, but I start several times - I get different predictions each time
tf_predict_pipeline = (ds.dataset.Pipeline()
.init_model("dynamic", TFConvModel, name="conv_model", config=model_config)
.init_variable("predictions_list", init_on_each_run=list)
.load(fmt="wfdb", components=["signal", "meta"])
.flip_signals()
.random_split_signals(2048,1)
.apply_transform(func=np.transpose, src='signal', dst='signal', axes=[0, 2, 1])
.predict_model('conv_model', make_data=tf_make_data,
fetches='output', save_to=V("predictions_list"), mode="e"))

Error on importing library

I'm importing libraries in this tutorial Dirichlet model training for Atrial fibrillation detection, on importing Dirichlet model:

from cardio.models.dirichlet_model import DirichletModel, concatenate_ecg_batch

I got this error:
AttributeError: module 'tensorflow.python.ops.losses.losses' has no attribute 'huber_loss'

My environment:
Anaconda Python 3.6.5
Tensorflow 1.1.0
Mac Sierra 10.12

'Annotation' object has no attirubte 'annsamp'

Hi all --- I am trying to run the HMM segmentation in this notebook. I've downloaded the QT database, but when I run the pipeline initialization line ppl_inits = (dtst >> pipeline).run(), I get the following output:

     82         if ds.any_action_failed(results):
     83             all_errors = self.get_errors(results)
---> 84             raise RuntimeError("Cannot assemble the batch", all_errors)
     85
     86     @staticmethod

RuntimeError: ('Cannot assemble the batch', [AttributeError("'Annotation' object has no attribute 'annsamp'",), AttributeError("'Annotation' object has no attribute 'annsamp'",), Att
ributeError("'Annotation' object has no attribute 'annsamp'",), AttributeError("'Annotation' object has no attribute 'annsamp'",), AttributeError("'Annotation' object has no attribut
e 'annsamp'",), AttributeError("'Annotation' object has no attribute 'annsamp'",), AttributeError("'Annotation' object has no attribute 'annsamp'",), AttributeError("'Annotation' obj
ect has no attribute 'annsamp'",), AttributeError("'Annotation' object has no attribute 'annsamp'",), AttributeError("'Annotation' object has no attribute 'annsamp'",), AttributeErro
r("'Annotation' object has no attribute 'annsamp'",), AttributeError("'Annotation' object has no attribute 'annsamp'",), AttributeError("'Annotation' object has no attribute 'annsamp
'",), AttributeError("'Annotation' object has no attribute 'annsamp'",), AttributeError("'Annotation' object has no attribute 'annsamp'",), AttributeError("'Annotation' object has no
 attribute 'annsamp'",), AttributeError("'Annotation' object has no attribute 'annsamp'",), AttributeError("'Annotation' object has no attribute 'annsamp'",), AttributeError("'Annota
tion' object has no attribute 'annsamp'",), AttributeError("'Annotation' object has no attribute 'annsamp'",)])

Is there some data file missing from my repository?

install_requires question

HI all :
When I am installing this project ,the install_requires needs scikit-learn=='0.19.1'
but scikit-learn 0.19.1 I can't install correctly
I think it's too old version?
I user annaconda +python3.7
how could I do to work with this project~ thanks a lot
image

how can i load split ECG data

I'm sorry for asking so many questions.
But I haven't been able to solve the problem all night.
Here's the question:

  1. How can I use only one ECG channel not 2-lead of ECG1 and ECG2 in qt-database?

  2. The meta data contains all the qrs information, but according to the survey, there are about 3,700 annotation data entered by experts. Do you use all Meta data in cardio when do training? Or do you use only 3,700 data?

  3. When I train HMM algorithm, I always have to load all the data, so how can I split and load only the data I want?
    I would like to load only data that has an expert annotation.

I'm looking at the subfiles of the all tutorial and all the Cardio .py files.
But I'm frustrated with the continuing problem.

I'd really appreciate it if you could give me a little more help.

best regard
Endrew

Training time with training2017 dataset

I ran the training session on NVIDIA Tegra Jetson TX2 GPU. The training2017 dataset contains ~8500 samples. Just how much training time this is supposed to take?

On above GPU, I needed 18 hours of training time. Is this normal (Ubuntu 16.04 LTS, TF 1.9 with GPU support) ?

~anoop

how to predict with .edf file?

Dear all:
I have train a model with /example/Getting_started.jupyter
it's worked with .hea file
and it can predict hea file
but how to read .edf file
when I change lod .head file to .edf file
code:
index = bf.FilesIndex(path="../cardio/tests/data/sample.edf", no_ext=True, sort=True)
eds = bf.Dataset(index, batch_class=EcgBatch)
batch = eds.next_batch(batch_size=2)
batch_with_data = batch.load(fmt="edf", components=["signal", "meta"])

from cardio.pipelines import dirichlet_predict_pipeline
model_path = "af_model_dump"
gpu_options = tf.GPUOptions(per_process_gpu_memory_fraction=0.7, allow_growth=True)
pipeline = dirichlet_predict_pipeline(model_path, gpu_options=gpu_options)
res = (eds >> pipeline).run()
pred = res.get_variable("predictions_list")

I get problem :
/usr/local/lib/python3.6/dist-packages/cardio/core/ecg_batch.py in _reraise_exceptions(self, results)
197 if bf.any_action_failed(results):
198 all_errors = self.get_errors(results)
--> 199 raise RuntimeError("Cannot assemble the batch", all_errors)
200
201 @staticmethod

RuntimeError: ('Cannot assemble the batch', [FileNotFoundError(2, 'No such file or directory')])

Plz help me~~
thanks a lot

AttributeError: unstack_signals not found in class EcgBatch

Hi
I ran into errors when running I.Cardio tutorial file. The script before this works fine, could you help with that?

`original_batch = batch_with_data.deepcopy()
changed_batch = batch_with_data.deepcopy()

original_batch.random_split_signals(length=3000, n_segments=4)

changed_batch.random_split_signals(length=3000, n_segments=4).unstack_signals()`


AttributeError Traceback (most recent call last)
in ()
4 original_batch.random_split_signals(length=3000, n_segments=4)
5
----> 6 changed_batch.random_split_signals(length=3000, n_segments=4).unstack_signals()

~/anaconda3/lib/python3.6/site-packages/cardio/dataset/dataset/batch.py in getattr(self, name)
258 return attr
259 else:
--> 260 raise AttributeError("%s not found in class %s" % (name, self.class.name))
261
262 def setattr(self, name, value):

AttributeError: unstack_signals not found in class EcgBatch

how can i find "batch_size" function?

I would like to know about "batch_size" in the following code.
There are three things I want to know.

template_ppl_inits = (
bf.Pipeline()
.init_variable("annsamps", init_on_each_run=list)
.init_variable("anntypes", init_on_each_run=list)
.init_variable("hmm_features", init_on_each_run=list)
.load(fmt='wfdb', components=["signal", "annotation", "meta"], ann_ext='pu1')
.cwt(src="signal", dst="hmm_features", scales=[4,8,16], wavelet="mexh")
.standardize(axis=-1, src="hmm_features", dst="hmm_features")
.update_variable("annsamps", bf.F(get_annsamples), mode='e')
.update_variable("anntypes", bf.F(get_anntypes), mode='e')
.update_variable("hmm_features", bf.B("hmm_features"), mode='e')
.run(batch_size=20, shuffle=False, drop_last=False, n_epochs=1, lazy=True)

)

  1. I want to know if " batch_size " is the size to determine file index or the size to determine batch of siganl.

  2. If batch_size is to determine the batch_size of siganl, i would like to know whether it is a nested, batch-free, one-step process after the other.(I want to know the structure of playing batch.)

  3. I couldn't find a method where is batch_size. I would appreciate it if you could help me know the function associated with batch_size.

ECG segmentation perfomance

Hi,
thank you for sharing this!

I've successfully trained the hmm model for the segmentation task. Feeding it with the demo data provided outputs satisfying results. When using different unseen data, it does not though.

I tried to do the segmentation with the Physionet 2016 dataset . Each record contains a heart sound recording and the respective ecg signal. Therefore, I've extracted the ecg first:

filename = "a0001"
record = wfdb.rdrecord('./tests/data/' + filename)
signals, fields = wfdb.rdsamp('./tests/data/' + name, channels=[1])

# Write a wfdb header file and any associated dat files from this object.
wfdb.wrsamp(
  'modified_' + filename,
  fs = fields['fs'],
  units=['mV'],
  sig_name=['ECG'],
  p_signal=signals,
  write_dir='./tests/data/',
  fmt=['16']
)

The prediction is done by:

ECG_MASK = "../cardio/tests/data/modified*.hea"
MODEL_PATH = "./model_dump.dill" 
config_predict = { 'build': False, 'load': {'path': "./model_dump.dill"}}

index = bf.FilesIndex(path="./tests/data/modified*.hea", no_ext=True, sort=True)
dtst = bf.Dataset(index, batch_class=EcgBatch)

pipeline = dtst >> hmm_predict_pipeline(MODEL_PATH, annot="hmm_annotation")
batch = pipeline.next_batch()

for idx, val in enumerate(dtst.indices):
  batch.show_ecg(batch.indices[idx], 0, 20, "hmm_annotation")

But the model output seems to be almost random:

Screenshot 2020-02-24 at 00 19 49

My question is: Did you experience similar results? Do you have any perfomance metrics for the segmentation model? Am I missing something here (like preprocessing)?

Any help is greatly appreciated.

what is difference A00001~A08528 microsoft Access Table Shortcut and sel30~selxx

there is item in " ..\cardio\cardio\tests\data " A0001~A00013 .hea and Microsoft Access Table Shortcut.

but there isn't in qt-database-1.0.0 Microsoft Access Table Shortcut.

what's the difference. and how can i read in qt-database-1.0.0?

using qt-database-1.0.0 i can train HMM algorithm. but i can't find the midium result..

AttributeError: module 'cardio.dataset' has no attribute 'Batch'

I got this while trying to follow the tutorial:

import cardio.dataset as ds
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-2-46c2fbe44381> in <module>()
----> 1 import cardio.dataset as ds

~/Dropbox/Development/cardio/cardio/__init__.py in <module>()
      2 import sys
      3 
----> 4 from .batch import *  # pylint: disable=wildcard-import
      5 from . import dataset  # pylint: disable=wildcard-import
      6 

~/Dropbox/Development/cardio/cardio/batch/__init__.py in <module>()
      1 """ ECG Batch """
----> 2 from .ecg_batch import EcgBatch
      3 from .ecg_dataset import EcgDataset

~/Dropbox/Development/cardio/cardio/batch/ecg_batch.py in <module>()
     16 
     17 
---> 18 class EcgBatch(ds.Batch):  # pylint: disable=too-many-public-methods,too-many-instance-attributes
     19     """Batch class for ECG signals storing.
     20 

AttributeError: module 'cardio.dataset' has no attribute 'Batch'

Here are the install steps I used - I had to manually install numba after getting an error about it being missing:

cd cardio
virtualenv --python=python3 venv
source venv/bin/activate
pip install .
pip install jupyter
jupyter notebook --port=8089
# I got ImportError: No module named 'numba'
pip install numba

I'm running Python 3.5 on OS X.

hmm_preprocessing_pipeline(); FileNotFoundError(2, 'No such file or directory')])

Hi

I ran into the following question when trying to run tutorial III and had the following errors. I checked the ../cardio/tests/data directory, which is directly from github repository, there are files existing. I am not sure where went wrong, could you please help with that?

Thanks!

import sys, os
sys.path.append('..')

from cardio import EcgDataset

SIGNALS_PATH = "../cardio/tests/data" #set path to the QT database
SIGNALS_MASK = os.path.join(SIGNALS_PATH, "*.hea")
dtst = EcgDataset(path=SIGNALS_MASK, no_ext=True, sort=True)

from cardio.pipelines import hmm_preprocessing_pipeline, hmm_train_pipeline
import warnings
warnings.filterwarnings('ignore')

pipeline = hmm_preprocessing_pipeline()
ppl_inits = (dtst >> pipeline).run()


RuntimeError Traceback (most recent call last)
in ()
4
5 pipeline = hmm_preprocessing_pipeline()
----> 6 ppl_inits = (dtst >> pipeline).run()

~/anaconda3/lib/python3.6/site-packages/cardio/dataset/dataset/pipeline.py in run(self, *args, **kwargs)
1086 if len(args) == 0 and len(kwargs) == 0:
1087 args, kwargs = self._lazy_run
-> 1088 for _ in self.gen_batch(*args, **kwargs):
1089 pass
1090 return self

~/anaconda3/lib/python3.6/site-packages/cardio/dataset/dataset/pipeline.py in gen_batch(self, batch_size, shuffle, n_epochs, drop_last, prefetch, on_iter, *args, **kwargs)
1033 for batch in batch_generator:
1034 try:
-> 1035 batch_res = self._exec(batch)
1036 except SkipBatchException:
1037 pass

~/anaconda3/lib/python3.6/site-packages/cardio/dataset/dataset/pipeline.py in _exec(self, batch, new_loop)
577 asyncio.set_event_loop(asyncio.new_event_loop())
578 batch.pipeline = self
--> 579 batch_res = self._exec_all_actions(batch)
580 batch_res.pipeline = self
581 return batch_res

~/anaconda3/lib/python3.6/site-packages/cardio/dataset/dataset/pipeline.py in _exec_all_actions(self, batch, action_list)
563 join_batches = None
564
--> 565 batch = self._exec_one_action(batch, _action, _action_args, _action['kwargs'])
566
567 batch.pipeline = self

~/anaconda3/lib/python3.6/site-packages/cardio/dataset/dataset/pipeline.py in _exec_one_action(self, batch, action, args, kwargs)
514 batch.pipeline = self
515 action_method, _ = self._get_action_method(batch, action['name'])
--> 516 batch = action_method(*args, **kwargs)
517 batch.pipeline = self
518 return batch

~/anaconda3/lib/python3.6/site-packages/cardio/dataset/dataset/decorators.py in _action_wrapper(action_self, *args, **kwargs)
35 action_self.pipeline.get_variable(_lock_name).acquire()
36
---> 37 _res = action_method(action_self, *args, **kwargs)
38
39 if _use_lock is not None:

~/anaconda3/lib/python3.6/site-packages/cardio/core/ecg_batch.py in load(self, src, fmt, components, ann_ext, *args, **kwargs)
254 return self._load_labels(src)
255 elif fmt in ["wfdb", "dicom", "edf", "wav"]:
--> 256 return self._load_data(src=src, fmt=fmt, components=components, ann_ext=ann_ext, *args, **kwargs)
257 else:
258 return super().load(src, fmt, components, *args, **kwargs)

~/anaconda3/lib/python3.6/site-packages/cardio/dataset/dataset/decorators.py in wrapped_method(self, *args, **kwargs)
236 return wrap_with_async(self, args, kwargs)
237 elif target in ['threads', 't']:
--> 238 return wrap_with_threads(self, args, kwargs)
239 elif target in ['mpc', 'm']:
240 return wrap_with_mpc(self, args, kwargs)

~/anaconda3/lib/python3.6/site-packages/cardio/dataset/dataset/decorators.py in wrap_with_threads(self, args, kwargs)
169 cf.wait(futures, timeout=timeout, return_when=cf.ALL_COMPLETED)
170
--> 171 return _call_post_fn(self, post_fn, futures, args, full_kwargs)
172
173 def wrap_with_mpc(self, args, kwargs):

~/anaconda3/lib/python3.6/site-packages/cardio/dataset/dataset/decorators.py in _call_post_fn(self, post_fn, futures, args, kwargs)
134 return self
135 else:
--> 136 return post_fn(all_results, *args, **kwargs)
137
138 def _make_args(init_args, args, kwargs):

~/anaconda3/lib/python3.6/site-packages/cardio/core/ecg_batch.py in _assemble_load(self, results, *args, **kwargs)
310 """
311 _ = args, kwargs
--> 312 self._reraise_exceptions(results)
313 components = kwargs.get("components", None)
314 if components is None:

~/anaconda3/lib/python3.6/site-packages/cardio/core/ecg_batch.py in _reraise_exceptions(self, results)
200 if ds.any_action_failed(results):
201 all_errors = self.get_errors(results)
--> 202 raise RuntimeError("Cannot assemble the batch", all_errors)
203
204 @staticmethod

RuntimeError: ('Cannot assemble the batch', [FileNotFoundError(2, 'No such file or directory'), FileNotFoundError(2, 'No such file or directory'), FileNotFoundError(2, 'No such file or directory'), FileNotFoundError(2, 'No such file or directory'), FileNotFoundError(2, 'No such file or directory'), FileNotFoundError(2, 'No such file or directory')])

Question about how to get the annotation prediction from pipeline

Hello, I wonder how I can get the annotation prediction of the segmentation from the following code?

batch = (eds >> hmm_predict_pipeline("model_dump.dll", annot="hmm_annotation")).next_batch()
batch.show_ecg('A00001',start=0, end=5, annot="hmm_annotation")

The image can show them but I can't get them.

eds = EcgDataset(path='/home/long/cardio/cardio/tests/data/A*.hea',no_ext=True,sort=True)
print(eds.indices)
print('len(eds.indices)=',len(eds.indices))
from cardio.pipelines import hmm_predict_pipeline
print(type(hmm_predict_pipeline("model_dump.dll", annot="hmm_annotation")))
batch = (eds >> hmm_predict_pipeline("model_dump.dll", annot="hmm_annotation"))
print(type(batch))
batch = batch.next_batch()
batch.show_ecg('A00001',start=0, end=5, annot="hmm_annotation")

result:

['A00001' 'A00002' 'A00004' 'A00005' 'A00008' 'A00013']
len(eds.indices)= 6
<class 'cardio.dataset.dataset.pipeline.Pipeline'>
<class 'cardio.dataset.dataset.pipeline.Pipeline'>
#(image here)

I've tried from the 'show_ecg' source code:

import numpy as np
print(np.shape(batch.signal))
import cardio.core.ecg_batch_tools as bt
start = 0
end = 5
annot="hmm_annotation"
i = batch.get_pos(None,"hmm_annotation",'A00001')
print(i)
meta = batch.meta[i]
fs = meta["fs"]
print(fs)
signal_states = getattr(batch,annot)[i][start:end]
print(signal_states)
starts, ends = bt.find_intervals_borders(signal_states,bt.QRS_STATES)
print(starts,ends)
print((starts+start)/fs,(ends+start)/fs)

but I got strange result:

(6,)
0
300
[2 2 2 2 2]
[] []
[] []

Please help, thanks a lot!

Dataset - TypeError: only integer scalar arrays can be converted to a scalar index

Hi,
I'm new to Python but wanted to try ECG segmentation so I tried using cardio. I've installed below packages on Python 3.6.5 in addition to all the other packages that were needed in the process. I tried executing the example code of the ECG segmentation in the III. Models tutorial, and got an error message "TypeError: only integer scalar arrays can be converted to a scalar index". I've listed the codes that were executed followed by the error message.

pip3 install git+https://github.com/analysiscenter/cardio.git
pip3 install git+https://github.com/analysiscenter/dataset.git
pip install hmmlearn-0.2.1-cp36-cp36m-win_amd64.whl (after I downloaded the file from https://www.lfd.uci.edu/~gohlke/pythonlibs/#hmmlearn - installing hmmlearn from GitHub gave an error while installing)

Code:

import sys, os
sys.path.append('..')

from cardio import EcgDataset

SIGNALS_PATH = "/notebooks/data/ECG/QT/" #set path to the QT database
SIGNALS_MASK = os.path.join(SIGNALS_PATH, "*.hea")

dtst = EcgDataset(path=SIGNALS_MASK, no_ext=True, sort=True)

from cardio.pipelines import hmm_preprocessing_pipeline, hmm_train_pipeline
import warnings
warnings.filterwarnings('ignore')

pipeline = hmm_preprocessing_pipeline()
ppl_inits = (dtst >> pipeline).run()

Error message:
ppl_inits = (dtst >> pipeline).run()

Traceback (most recent call last):

File "", line 1, in
ppl_inits = (dtst >> pipeline).run()

File "C:\Anaconda3\lib\site-packages\cardio\dataset\dataset\pipeline.py", line 1088, in run
for _ in self.gen_batch(*args, **kwargs):

File "C:\Anaconda3\lib\site-packages\cardio\dataset\dataset\pipeline.py", line 1033, in gen_batch
for batch in batch_generator:

File "C:\Anaconda3\lib\site-packages\cardio\dataset\dataset\base.py", line 162, in gen_batch
for ix_batch in self.index.gen_batch(batch_size, shuffle, n_epochs, drop_last):

File "C:\Anaconda3\lib\site-packages\cardio\dataset\dataset\dsindex.py", line 276, in gen_batch
batch = self.next_batch(batch_size, shuffle, n_epochs, drop_last, iter_params)

File "C:\Anaconda3\lib\site-packages\cardio\dataset\dataset\dsindex.py", line 250, in next_batch
return self.create_batch(rest_items, pos=True)

File "C:\Anaconda3\lib\site-packages\cardio\dataset\dataset\dsindex.py", line 292, in create_batch
batch = self.subset_by_pos(_batch_indices)

File "C:\Anaconda3\lib\site-packages\cardio\dataset\dataset\dsindex.py", line 80, in subset_by_pos
return self.index[pos]

TypeError: only integer scalar arrays can be converted to a scalar index

I'm not sure what is the problem. I appreciate your help in advance.

TypeError: '<' not supported between instances of 'float' and 'str'

Hi
I ran into trouble when trying the II.pipelines script. I used the tests/data dataset as my input. Could you please help with that?

import os
from cardio import EcgDataset
PATH_TO_DATA = "../cardio/tests/data" #set path to data
pds = EcgDataset(path=os.path.join(PATH_TO_DATA, "*.hea"), no_ext=True, sort=True)
pds.cv_split(0.8, shuffle=True)

from cardio.pipelines import dirichlet_train_pipeline
%env CUDA_VISIBLE_DEVICES=0
AF_SIGNALS_REF = os.path.join(PATH_TO_DATA, "REFERENCE.csv")
pipeline = dirichlet_train_pipeline(AF_SIGNALS_REF, batch_size=256, n_epochs=500)

trained = (pds.train >> pipeline).run()


TypeError Traceback (most recent call last)
in ()
----> 1 trained = (pds.train >> pipeline).run()

~/anaconda3/lib/python3.6/site-packages/cardio/dataset/dataset/pipeline.py in run(self, *args, **kwargs)
1086 if len(args) == 0 and len(kwargs) == 0:
1087 args, kwargs = self._lazy_run
-> 1088 for _ in self.gen_batch(*args, **kwargs):
1089 pass
1090 return self

~/anaconda3/lib/python3.6/site-packages/cardio/dataset/dataset/pipeline.py in gen_batch(self, batch_size, shuffle, n_epochs, drop_last, prefetch, on_iter, *args, **kwargs)
1033 for batch in batch_generator:
1034 try:
-> 1035 batch_res = self._exec(batch)
1036 except SkipBatchException:
1037 pass

~/anaconda3/lib/python3.6/site-packages/cardio/dataset/dataset/pipeline.py in _exec(self, batch, new_loop)
577 asyncio.set_event_loop(asyncio.new_event_loop())
578 batch.pipeline = self
--> 579 batch_res = self._exec_all_actions(batch)
580 batch_res.pipeline = self
581 return batch_res

~/anaconda3/lib/python3.6/site-packages/cardio/dataset/dataset/pipeline.py in _exec_all_actions(self, batch, action_list)
563 join_batches = None
564
--> 565 batch = self._exec_one_action(batch, _action, _action_args, _action['kwargs'])
566
567 batch.pipeline = self

~/anaconda3/lib/python3.6/site-packages/cardio/dataset/dataset/pipeline.py in _exec_one_action(self, batch, action, args, kwargs)
514 batch.pipeline = self
515 action_method, _ = self._get_action_method(batch, action['name'])
--> 516 batch = action_method(*args, **kwargs)
517 batch.pipeline = self
518 return batch

~/anaconda3/lib/python3.6/site-packages/cardio/dataset/dataset/decorators.py in _action_wrapper(action_self, *args, **kwargs)
35 action_self.pipeline.get_variable(_lock_name).acquire()
36
---> 37 _res = action_method(action_self, *args, **kwargs)
38
39 if _use_lock is not None:

~/anaconda3/lib/python3.6/site-packages/cardio/core/ecg_batch.py in load(self, src, fmt, components, ann_ext, *args, **kwargs)
252 components = np.asarray(components).ravel()
253 if (fmt == "csv" or fmt is None and isinstance(src, pd.Series)) and np.all(components == "target"):
--> 254 return self._load_labels(src)
255 elif fmt in ["wfdb", "dicom", "edf", "wav"]:
256 return self._load_data(src=src, fmt=fmt, components=components, ann_ext=ann_ext, *args, **kwargs)

~/anaconda3/lib/python3.6/site-packages/cardio/core/ecg_batch.py in _load_labels(self, src)
353 raise RuntimeError("Batch with undefined unique_labels must be created in a pipeline")
354 ds_indices = self.pipeline.dataset.indices
--> 355 self.unique_labels = np.sort(src[ds_indices].unique())
356 return self
357

~/anaconda3/lib/python3.6/site-packages/numpy/core/fromnumeric.py in sort(a, axis, kind, order)
845 else:
846 a = asanyarray(a).copy(order="K")
--> 847 a.sort(axis=axis, kind=kind, order=order)
848 return a
849

TypeError: '<' not supported between instances of 'float' and 'str'

Errors on running tutorials

Hello
I'm getting errors running Tutorials notebooks.

In Notebook 2 (Pipelines) I got this
Screenshot from 2019-05-04 11-43-34

And in Notebook 3 (Models) I got this error
Screenshot from 2019-05-04 11-46-33

I'm using Ubuntu 18.04 with latest anaconda

'NoneType' object has no attribute 'ndim'

I got this error whilst following the tutorial when trying to display the ecg signal using:
sample.show_ecg('A00001')

File "<ipython-input-42-3742dca0684b>", line 1, in <module> sample.show_ecg('A00001')

File "/anaconda3/lib/python3.6/site-packages/cardio/core/ecg_batch.py", line 391, in show_ecg self._check_2d(signal)
File "/anaconda3/lib/python3.6/site-packages/cardio/core/ecg_batch.py", line 218, in _check_2d if signal.ndim != 2:

AttributeError: 'NoneType' object has no attribute 'ndim'

Quite new to python so not sure if its something to do with me cloning the repo etc?

Any help would be much appreciated!

reading labeled data

Hello all,

The flow of "dataset => batch => batch with data" is understood. "next_batch" can be called with an arbitrary batch_size.

My question is while loading the "target" component in "batch with data" how I can extract the same indices from the larger labeled data file?

e.g. consider the following
eds = EcgDataset(path="*.hea")
batch = eds.next_batch (batch_size = 8)
batch_with_data = batch.load (fmt="wfdb", components = ["signal", "meta"]

Now, suppose I have totally 1000 .hea files and a csv file with 1000 outcome labels. How do I fetch exactly those 8 labels in "batch_with_data" that correspond to same indices as files in the batch from this larger 1000 labels array?

thanks

~anoop

get error when import cardio in Anaconda python 3.6.3

import cardio
Traceback (most recent call last):
File "", line 1, in
File "C:\Users\HTTU\Anaconda3\lib\site-packages\cardio_init_.py", line 3, in
from . import dataset # pylint: disable=wildcard-import
File "C:\Users\HTTU\Anaconda3\lib\site-packages\cardio\dataset_init_.py", line 4, in
sys.modules[package] = importlib.import_module('.dataset', package)
File "C:\Users\HTTU\Anaconda3\lib\importlib_init_.py", line 126, in import_module
return bootstrap.gcd_import(name[level:], package, level)
File "C:\Users\HTTU\Anaconda3\lib\site-packages\cardio\dataset\dataset_init
.py", line 9, in
from .batch import Batch, ArrayBatch, DataFrameBatch
File "C:\Users\HTTU\Anaconda3\lib\site-packages\cardio\dataset\dataset\batch.py", line 21, in
import dask.dataframe as dd
File "C:\Users\HTTU\Anaconda3\lib\site-packages\dask\dataframe_init
.py", line 3, in
from .core import (DataFrame, Series, Index, _Frame, map_partitions,
File "C:\Users\HTTU\Anaconda3\lib\site-packages\dask\dataframe\core.py", line 40, in
pd.core.computation.expressions.set_use_numexpr(False)
AttributeError: module 'pandas.core.computation' has no attribute 'expressions'

Installation Failed ( pyEDFlib requires a version of NumPy, even for setup )

Привет. Не получается установить библиотеку cardIO. Пробовал устанавливать через pipenv, pip и через python setup.py install. В всех случаях ругается на pyEDFlib и NumPy.
Лог прилагается.

Hello. I am having a problem installing the cardIO library using pipenv or pip. Maybe the problem is in setup.py or pyEDFlib?

It is log:
Microsoft Windows [Version 6.1.7601]
(c) Корпорация Майкрософт (Microsoft Corp.), 2009. Все права защищены.

C:\Users\user>pipenv install git+https://github.com/analysiscenter/cardio.git#eg
g=cardio
Creating a virtualenv for this project:
Pipfile: C:\Users\user\Pipfile
Using c:\python\python.exe (3.7.4) to create virtualenv:
[ =] Creating virtual environment...Already using interpreter c:\python\python
.exe
Using base prefix 'c:\python'
New python executable in C:\Users\user.virtualenvs\user-KgVmIwbj\Scripts\python
.exe
Installing setuptools, pip, wheel...
done.

Successfully created virtual environment!
Virtualenv location: C:\Users\user.virtualenvs\user-KgVmIwbj
Creating a Pipfile for this project:
Installing git+https://github.com/analysiscenter/cardio.git#egg=cardio:
Warning: You installed a VCS dependency in non-editable mode. This will work fin
e, but sub-dependencies will not be resolved by $ pipenv lock.
To enable this sub-dependency functionality, specify that this dependency is e
ditable.
Collecting cardio from git+https://github.com/analysiscenter/cardio.git#egg=card
io
Cloning https://github.com/analysiscenter/cardio.git to c:\users\user\appdata
local\temp\pip-install-g2sfymp2\cardio
Collecting numpy>=1.13.1 (from cardio)
Downloading https://files.pythonhosted.org/packages/26/26/73ba03b2206371cdef62
afebb877e9ba90a1f0dc3d9de22680a3970f5a50/numpy-1.17.0-cp37-cp37m-win_amd64.whl (
12.8MB)
Collecting scipy>=0.19.1 (from cardio)
Downloading https://files.pythonhosted.org/packages/50/eb/defa40367863304e1ef0
1c6572584c411446a5f29bdd9dc90f91509e9144/scipy-1.3.1-cp37-cp37m-win_amd64.whl (3
0.3MB)
Collecting pandas>=0.21.1 (from cardio)
Downloading https://files.pythonhosted.org/packages/c1/cf/58ccaa38d5670dd4d2ae
e5df90aa03d670ede3947b7148e72391c80d4f91/pandas-0.25.0-cp37-cp37m-win_amd64.whl
(9.2MB)
Collecting scikit-learn==0.19.1 (from cardio)
Downloading https://files.pythonhosted.org/packages/f5/2c/5edf2488897cad4fb8c4
ace86369833552615bf264460ae4ef6e1f258982/scikit-learn-0.19.1.tar.gz (9.5MB)
Collecting numba>=0.35.0 (from cardio)
Downloading https://files.pythonhosted.org/packages/dd/89/861d76f7a677fc5ba7fa
fa8085017c43b76e3880e9e365f863f8cc800a46/numba-0.45.1-cp37-cp37m-win_amd64.whl (
2.0MB)
Collecting pywavelets>=0.5.2 (from cardio)
Downloading https://files.pythonhosted.org/packages/71/57/1b6c06f2b681975e5b43
ff13ba127031f668d65b592cc56fae6445b675c7/PyWavelets-1.0.3-cp37-none-win_amd64.wh
l (4.2MB)
Collecting matplotlib>=2.1.0 (from cardio)
Downloading https://files.pythonhosted.org/packages/1a/c0/69e3f695d7384012e90b
e1e16570c08953baae00fd98094179ef87c7d5a2/matplotlib-3.1.1-cp37-cp37m-win_amd64.w
hl (9.1MB)
Collecting dill>=0.2.7.1 (from cardio)
Downloading https://files.pythonhosted.org/packages/39/7a/70803635c850e3512570
29089d38748516a280864c97cbc73087afef6d51/dill-0.3.0.tar.gz (151kB)
Collecting pydicom>=0.9.9 (from cardio)
Downloading https://files.pythonhosted.org/packages/fc/d5/da1fdf3b967e324ee47a
7ad9553c9b94c1193b6b98afd9eeda0efb76b9f7/pydicom-1.3.0-py2.py3-none-any.whl (7.1
MB)
Collecting pyedflib>=0.1.11 (from cardio)
Downloading https://files.pythonhosted.org/packages/a4/c1/11d90aa4d20a1d3df04e
a9e1db4e03fea4b199a5b515a057cbe5e35769c0/pyEDFlib-0.1.14.tar.gz (1.1MB)

Error: An error occurred while installing git+https://github.com/analysiscenter
/cardio.git#egg=cardio!
b' Running command git clone -q https://github.com/analysiscenter/cardio.git '
C:\Users\user\AppData\Local\Temp\pip-install-g2sfymp2\cardio'\n Running
command git submodule update --init --recursive -q\n ERROR: Command errored
out with exit status 4294967295:\n command: 'c:\users\user.virtualenvs
\user-kgvmiwbj\scripts\python.exe' -c 'import sys, setuptools, tokenize; sys
.argv[0] = '"'"'C:\Users\user\AppData\Local\Temp\pip-instal
l-g2sfymp2\pyedflib\setup.py'"'"'; file='"'"'C:\Users\user
\AppData\Local\Temp\pip-install-g2sfymp2\pyedflib\setup.py'"
'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().rep
lace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, _
file_, '"'"'exec'"'"'))' egg_info --egg-base pip-egg-info\n cwd
: C:\Users\user\AppData\Local\Temp\pip-install-g2sfymp2\pyedflib\n C
omplete output (4 lines):\n No module named 'numpy'\n *** package "numpy
" not found ***\n pyEDFlib requires a version of NumPy, even for setup.\n
Please get it from http://numpy.scipy.org/ or install it through your package ma
nager.\n ----------------------------------------\nERROR: Command errored out
with exit status 4294967295: python setup.py egg_info Check the logs for full c
ommand output.\n'
This is likely caused by a bug in cardio. Report this to its maintainers.
Installation Failed

syntax error when importing batch.py

import os import sys sys.path.append("..") import cardio.dataset as ds from cardio import EcgBatch from cardio.models.metrics import classification_report %env CUDA_VISIBLE_DEVICES=0 import tensorflow as tf


Error message as the following:

User/ydee/anaconda2/lib/python2.7/site-packages/cardio/dataset/dataset/batch.py", line 403
_args = tuple([src_attr, *args])
^
SyntaxError: invalid syntax

HMM Jupyter notebook gives batchflow import errors

Dear developers,

Thanks a lot for sharing your code on GitHub! I installed CardIO as a Git project repository (following your instructions, so including --recursive), on my Ubuntu Linux machine with Python 3.7 and Miniconda.

When trying to work through the HMM Jupyter notebook, I'm getting a lot of errors following the cardio import commands. The first one is AttributeError: module 'cardio.batchflow' has no attribute 'Batch' (not sure whether the others will be resolved once this one is, so I won't list these for now).

Am I doing something wrong? Would it be better to install CardIO through pip3?

Thanks in advance for your help!
Best wishes,
Sita

EDIT: In the meantime I tried installing through pip3, but that gives a lot of errors. I'm copying the last error below, if you need the other errors as well, please let me know.

ERROR: Command errored out with exit status 1: /home/sita/miniconda3/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-4be4jd4c/scikit-learn/setup.py'"'"'; __file__='"'"'/tmp/pip-install-4be4jd4c/scikit-learn/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-47pzagft/install-record.txt --single-version-externally-managed --compile --install-headers /home/sita/miniconda3/include/python3.7m/scikit-learn Check the logs for full command output.

can not install

python setup.py install

running install
running bdist_egg
running egg_info
writing requirements to cardio.egg-info/requires.txt
writing cardio.egg-info/PKG-INFO
writing top-level names to cardio.egg-info/top_level.txt
writing dependency_links to cardio.egg-info/dependency_links.txt
reading manifest file 'cardio.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no previously-included files matching '' found under directory 'docs/_build'
warning: no previously-included files matching '
.pyc' found anywhere in distribution
warning: no previously-included files matching '.pyo' found anywhere in distribution
warning: no previously-included files matching '
.pyd' found anywhere in distribution
warning: no previously-included files matching '.git' found anywhere in distribution
warning: no previously-included files matching '
.so' found anywhere in distribution
warning: no previously-included files matching '~' found anywhere in distribution
warning: no previously-included files matching '#
' found anywhere in distribution
warning: no previously-included files matching '.DS_Store' found anywhere in distribution
writing manifest file 'cardio.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_py
creating build/bdist.linux-x86_64/egg
creating build/bdist.linux-x86_64/egg/cardio
creating build/bdist.linux-x86_64/egg/cardio/pipelines
copying build/lib/cardio/pipelines/pipelines.py -> build/bdist.linux-x86_64/egg/cardio/pipelines
copying build/lib/cardio/pipelines/init.py -> build/bdist.linux-x86_64/egg/cardio/pipelines
creating build/bdist.linux-x86_64/egg/cardio/core
copying build/lib/cardio/core/ecg_batch_tools.py -> build/bdist.linux-x86_64/egg/cardio/core
copying build/lib/cardio/core/kernels.py -> build/bdist.linux-x86_64/egg/cardio/core
copying build/lib/cardio/core/ecg_dataset.py -> build/bdist.linux-x86_64/egg/cardio/core
copying build/lib/cardio/core/utils.py -> build/bdist.linux-x86_64/egg/cardio/core
copying build/lib/cardio/core/init.py -> build/bdist.linux-x86_64/egg/cardio/core
copying build/lib/cardio/core/ecg_batch.py -> build/bdist.linux-x86_64/egg/cardio/core
creating build/bdist.linux-x86_64/egg/cardio/models
copying build/lib/cardio/models/keras_custom_objects.py -> build/bdist.linux-x86_64/egg/cardio/models
creating build/bdist.linux-x86_64/egg/cardio/models/dirichlet_model
copying build/lib/cardio/models/dirichlet_model/dirichlet_model.py -> build/bdist.linux-x86_64/egg/cardio/models/dirichlet_model
copying build/lib/cardio/models/dirichlet_model/init.py -> build/bdist.linux-x86_64/egg/cardio/models/dirichlet_model
copying build/lib/cardio/models/layers.py -> build/bdist.linux-x86_64/egg/cardio/models
creating build/bdist.linux-x86_64/egg/cardio/models/fft_model
copying build/lib/cardio/models/fft_model/fft_model.py -> build/bdist.linux-x86_64/egg/cardio/models/fft_model
copying build/lib/cardio/models/fft_model/init.py -> build/bdist.linux-x86_64/egg/cardio/models/fft_model
creating build/bdist.linux-x86_64/egg/cardio/models/hmm
copying build/lib/cardio/models/hmm/hmm.py -> build/bdist.linux-x86_64/egg/cardio/models/hmm
copying build/lib/cardio/models/hmm/init.py -> build/bdist.linux-x86_64/egg/cardio/models/hmm
copying build/lib/cardio/models/metrics.py -> build/bdist.linux-x86_64/egg/cardio/models
copying build/lib/cardio/models/init.py -> build/bdist.linux-x86_64/egg/cardio/models
copying build/lib/cardio/init.py -> build/bdist.linux-x86_64/egg/cardio
byte-compiling build/bdist.linux-x86_64/egg/cardio/pipelines/pipelines.py to pipelines.pyc
byte-compiling build/bdist.linux-x86_64/egg/cardio/pipelines/init.py to init.pyc
byte-compiling build/bdist.linux-x86_64/egg/cardio/core/ecg_batch_tools.py to ecg_batch_tools.pyc
byte-compiling build/bdist.linux-x86_64/egg/cardio/core/kernels.py to kernels.pyc
byte-compiling build/bdist.linux-x86_64/egg/cardio/core/ecg_dataset.py to ecg_dataset.pyc
byte-compiling build/bdist.linux-x86_64/egg/cardio/core/utils.py to utils.pyc
File "build/bdist.linux-x86_64/egg/cardio/core/utils.py", line 30
return func(self, *frozen_args, *args, **frozen_kwargs, **kwargs)
^
SyntaxError: invalid syntax

byte-compiling build/bdist.linux-x86_64/egg/cardio/core/init.py to init.pyc
byte-compiling build/bdist.linux-x86_64/egg/cardio/core/ecg_batch.py to ecg_batch.pyc
File "build/bdist.linux-x86_64/egg/cardio/core/ecg_batch.py", line 26
"idwt": (lambda x, *args, **kwargs: pywt.idwt(*x, *args, **kwargs), "pywt.idwt",
^
SyntaxError: invalid syntax

byte-compiling build/bdist.linux-x86_64/egg/cardio/models/keras_custom_objects.py to keras_custom_objects.pyc
File "build/bdist.linux-x86_64/egg/cardio/models/keras_custom_objects.py", line 202
return (*input_shape[:-1], self.base_dim + 3 * self.nb_filters)
^
SyntaxError: invalid syntax

byte-compiling build/bdist.linux-x86_64/egg/cardio/models/dirichlet_model/dirichlet_model.py to dirichlet_model.pyc
byte-compiling build/bdist.linux-x86_64/egg/cardio/models/dirichlet_model/init.py to init.pyc
byte-compiling build/bdist.linux-x86_64/egg/cardio/models/layers.py to layers.pyc
byte-compiling build/bdist.linux-x86_64/egg/cardio/models/fft_model/fft_model.py to fft_model.pyc
byte-compiling build/bdist.linux-x86_64/egg/cardio/models/fft_model/init.py to init.pyc
byte-compiling build/bdist.linux-x86_64/egg/cardio/models/hmm/hmm.py to hmm.pyc
byte-compiling build/bdist.linux-x86_64/egg/cardio/models/hmm/init.py to init.pyc
byte-compiling build/bdist.linux-x86_64/egg/cardio/models/metrics.py to metrics.pyc
byte-compiling build/bdist.linux-x86_64/egg/cardio/models/init.py to init.pyc
byte-compiling build/bdist.linux-x86_64/egg/cardio/init.py to init.pyc
creating build/bdist.linux-x86_64/egg/EGG-INFO
copying cardio.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO
copying cardio.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying cardio.egg-info/dependency_links.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying cardio.egg-info/not-zip-safe -> build/bdist.linux-x86_64/egg/EGG-INFO
copying cardio.egg-info/requires.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying cardio.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
creating 'dist/cardio-0.2.0-py2.7.egg' and adding 'build/bdist.linux-x86_64/egg' to it
removing 'build/bdist.linux-x86_64/egg' (and everything under it)
Processing cardio-0.2.0-py2.7.egg
removing '/usr/lib/python2.7/site-packages/cardio-0.2.0-py2.7.egg' (and everything under it)
creating /usr/lib/python2.7/site-packages/cardio-0.2.0-py2.7.egg
Extracting cardio-0.2.0-py2.7.egg to /usr/lib/python2.7/site-packages
File "/usr/lib/python2.7/site-packages/cardio-0.2.0-py2.7.egg/cardio/core/utils.py", line 30
return func(self, *frozen_args, *args, **frozen_kwargs, **kwargs)
^
SyntaxError: invalid syntax

File "/usr/lib/python2.7/site-packages/cardio-0.2.0-py2.7.egg/cardio/core/ecg_batch.py", line 26
"idwt": (lambda x, *args, **kwargs: pywt.idwt(*x, *args, **kwargs), "pywt.idwt",
^
SyntaxError: invalid syntax

File "/usr/lib/python2.7/site-packages/cardio-0.2.0-py2.7.egg/cardio/models/keras_custom_objects.py", line 202
return (*input_shape[:-1], self.base_dim + 3 * self.nb_filters)
^
SyntaxError: invalid syntax

cardio 0.2.0 is already the active version in easy-install.pth

Installed /usr/lib/python2.7/site-packages/cardio-0.2.0-py2.7.egg
Traceback (most recent call last):
File "setup.py", line 58, in
'Topic :: Scientific/Engineering'
File "/usr/lib64/python2.7/distutils/core.py", line 152, in setup
dist.run_commands()
File "/usr/lib64/python2.7/distutils/dist.py", line 953, in run_commands
self.run_command(cmd)
File "/usr/lib64/python2.7/distutils/dist.py", line 972, in run_command
cmd_obj.run()
File "/usr/lib/python2.7/site-packages/setuptools/command/install.py", line 73, in run
self.do_egg_install()
File "/usr/lib/python2.7/site-packages/setuptools/command/install.py", line 101, in do_egg_install
cmd.run()
File "/usr/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 380, in run
self.easy_install(spec, not self.no_deps)
File "/usr/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 604, in easy_install
return self.install_item(None, spec, tmpdir, deps, True)
File "/usr/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 655, in install_item
self.process_distribution(spec, dist, deps)
File "/usr/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 701, in process_distribution
distreq.project_name, distreq.specs, requirement.extras
TypeError: init() takes exactly 2 arguments (4 given)

Run very long on Training pipeline

I'm following this tutorial for detecting Atrial fibrillation but when run Training pipeline it take so much time.

I'm using Tesla K80 and I leave it ran all night, more than 7 hours, but now it's still running.
In this block it's running 1000 epochs:

template_train_ppl = (
    ds.Pipeline()
      .init_model("dynamic", DirichletModel, name="dirichlet", config=model_config)
      .init_variable("loss_history", init_on_each_run=list)
      .load(components=["signal", "meta"], fmt="wfdb")
      .load(components="target", fmt="csv", src=LABELS_PATH)
      .drop_labels(["~"])
      .rename_labels({"N": "NO", "O": "NO"})
      .flip_signals()
      .random_resample_signals("normal", loc=300, scale=10)
      .random_split_signals(2048, {"A": 9, "NO": 3})
      .binarize_labels()
      .train_model("dirichlet", make_data=concatenate_ecg_batch,
                   fetches="loss", save_to=V("loss_history"), mode="a")
      .run(batch_size=BATCH_SIZE, shuffle=True, drop_last=True, n_epochs=N_EPOCH, lazy=True)
)

train_ppl = (eds.train >> template_train_ppl).run()

Do you thing that we have smt not right here?
Or does the framework have something indicate that it's running, maybe print the number of current epoch it's running?
And btw when I run I see this in terminal FYI:

2018-04-30 16:19:57.630351: W tensorflow/core/common_runtime/bfc_allocator.cc:217] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.71GiB. The caller indicates that this
 is not a failure, but may mean that there could be performance gains if more memory is available.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.