... I'll put together a PR later today, but heads up: openephys.py's _read_header_ fails in python 3, because the header is encoded in latin-1 while python 3 assumes utf-8.

To fix, change the open line to:

f = open(file, 'r', encoding='latin-1')

Also... I'll switch to with open() as f:. Later.

I was previously using CPU only and it seemed to get passed whitening step. Now with GPU version installed I get error below.

Is it a matter of lowering batch/GPU ram? I only have a 1050TI video card, which usually only has about 3GB of free video memory.

cat@catlab:~/Downloads/spyking-circus-master$ spyking-circus ~/2017-04-25-0/spyking_circus/data.dat -c 16 -g 1
/usr/local/lib/python3.5/dist-packages/h5py/init.py:36: FutureWarning: Conversion of the second argument of issubdtype from float to np.floating is deprecated. In future, it will be treated as np.float64 == np.dtype(float).type.
from ._conv import register_converters as _register_converters

##################################################################

Welcome to the SpyKING CIRCUS (0.6.3)

Written by P.Yger and O.Marre

##################################################################

Steps : filtering, whitening, clustering, fitting
GPU detected : True
Number of CPU : 16/32
Number of GPU : 1
Parallel HDF5 : False
Shared memory : True
Hostfile : /home/cat/spyking-circus/circus.hosts

##################################################################

------------------------- Informations -------------------------
| Number of recorded channels : 512
| Number of analyzed channels : 512
| File format : RAW_BINARY
| Data type : int16
| Sampling rate : 20 kHz
| Duration of the recording : 60 min 20 s 0 ms
| Width of the templates : 3 ms
| Spatial radius considered : 70 um
| Threshold crossing : negative
| Waveform alignment : True
| Matched filters : False
| Template Extraction : median-raw
| Overwrite : True
| Collect all spikes : False
| Smart Search : False
| Smart Selection : False

------------------------- Informations -------------------------
| Smart Search could be activated for long recordings

------------------------- Informations -------------------------
| Large number of electrodes, reducing chunk sizes to 10s

------------------------- Informations -------------------------
| Filtering has already been done

Analyzing data to get whitening matrices and thresholds...
Found 0.01165s without spikes for whitening matrices...
------------------------- Informations -------------------------
| Disabling spatial whitening because of NaNs found

Because of whitening, need to recompute the thresholds...
Traceback (most recent call last):
File "/usr/local/bin/spyking-circus-subtask", line 11, in
sys.exit(main())
File "/usr/local/lib/python3.5/dist-packages/circus/scripts/subtask.py", line 43, in main
circus.launch(task, filename, nb_cpu, nb_gpu, use_gpu)
File "/usr/local/lib/python3.5/dist-packages/circus/init.py", line 21, in launch
module.main(params, nb_cpu, nb_gpu, use_gpu)
File "/usr/local/lib/python3.5/dist-packages/circus/whitening.py", line 225, in main
local_chunk = cmt.CUDAMatrix(local_chunk, copy_on_host=False)
File "/usr/local/lib/python3.5/dist-packages/cudamat/cudamat.py", line 268, in init
raise generate_exception(err_code)
cudamat.cudamat.CUDAMatException: CUBLAS error.

Dear all,

I am a new user of Spyking Circus (and Python as well). I have therefore a very little experience.

I have problems in identifying the correct padding for my data. My data files are raw binary, uint16. I can normally open them in Matlab using uint16 with the ieee-le option.

In Spyking Circus if I use the auto parameter for the data type offset I get all my signals flat, so they are not correctly imported. Could you please help me to understand how to compute the correct data type offset for my data set?

Thank you in advance.

Again, in order to ease the packaging and the install of the software, we could provide a docker image

We should make an independent python package for the file format handling, such that phy could use it and open various file formats, not only pure binary

I don't want to do any whitening on my raw data, so I've set both temporal and spatial whitening to False in my .params file. After sorting in multi-file mode and then trying to convert the results to .npy for using in phy, I get this error:

-------------------------  Informations  -------------------------
| Exporting data for the phy GUI with 1 CPUs...
------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/bin/spyking-circus-subtask", line 11, in <module>
    load_entry_point('spyking-circus', 'console_scripts', 'spyking-circus-subtask')()
  File "/home/mspacek/src/spyking-circus/circus/scripts/subtask.py", line 33, in main
    circus.launch(task, filename, nb_cpu, nb_gpu, use_gpu, extension=extension)
  File "/home/mspacek/src/spyking-circus/circus/__init__.py", line 13, in launch
    module.main(filename, params, nb_cpu, nb_gpu, use_gpu, extension)
  File "/home/mspacek/src/spyking-circus/circus/converting.py", line 206, in main
    numpy.save(os.path.join(output_path, 'whitening_mat'), load_data(params, 'spatial_whitening').astype(numpy.double))
  File "/home/mspacek/src/spyking-circus/circus/shared/files.py", line 1237, in load_data
    spatial = numpy.ascontiguousarray(myfile.get('spatial')[:])
TypeError: 'NoneType' object has no attribute '__getitem__'

Then when I try to run phy using circus-gui-python:

IOError: No such file or directory: 'whitening_mat.npy'

I guess spyking-circus cannot currently export this matrix when whitening is disabled. However, perhaps it should export something like the identity matrix for compatibility with phy? I'll see if I can get this to work...

Dear Pierre,

Filenames of probe and exclude files in the params file cannot contain spaces. The line that causes this issue is line 106 in circus/shared/parser.py (version 0.6.0):

self.parser.set(section, key, value.split('#')[0].replace(' ', '').replace('\t', ''))

This line strips all whitespaces from the value of any parameter. Could you perhaps add a conditional rule for not doing so for probe and exclude files?

Thank you very much.
Best,
Paolo

Asking for the results view on an openephys dataset, I get the following error:

Traceback (most recent call last):
  File "/Users/benacland/anaconda/envs/spykpi/bin/spyking-circus", line 11, in <module>
    load_entry_point('spyking-circus==0.5.7', 'console_scripts', 'spyking-circus')()
  File "/Users/benacland/anaconda/envs/spykpi/lib/python3.5/site-packages/circus/scripts/launch.py", line 384, in main
    mygui = gui.PreviewGUI(params, show_fit=True)
  File "/Users/benacland/anaconda/envs/spykpi/lib/python3.5/site-packages/circus/shared/gui.py", line 1101, in __init__
    self.get_data()
  File "/Users/benacland/anaconda/envs/spykpi/lib/python3.5/site-packages/circus/shared/gui.py", line 1179, in get_data
    self.data       = self.data_file.get_snippet(t_start, self.chunk_size, nodes=self.nodes)
  File "/Users/benacland/anaconda/envs/spykpi/lib/python3.5/site-packages/circus/files/datafile.py", line 449, in get_snippet
    return self.get_data(0, chunk_size=length, padding=(local_time, local_time), nodes=nodes)[0]
  File "/Users/benacland/anaconda/envs/spykpi/lib/python3.5/site-packages/circus/files/datafile.py", line 459, in get_data
    return self.read_chunk(idx, chunk_size, padding, nodes), self.t_start + idx*chunk_size      
  File "/Users/benacland/anaconda/envs/spykpi/lib/python3.5/site-packages/circus/files/openephys.py", line 113, in read_chunk
    local_chunk[:, count] = self.data[i][data_slice]
  File "/Users/benacland/anaconda/envs/spykpi/lib/python3.5/site-packages/numpy/core/memmap.py", line 335, in __getitem__
    res = super(memmap, self).__getitem__(index)
IndexError: only integers, slices (`:`), ellipsis (`...`), numpy.newaxis (`None`) and integer or boolean arrays are valid indices

... submitting PR shortly.

Hi,

While clustering, I sometimes encounter an error due to Windows not being able to delete temporary files created by SpyKING CIRCUS. Here my .params file and below the error message.

Unfortunately, it doesn't happen too often for me to debug the problem, and it doesn't happen consistently on the same data file to suspect a problem with a particular dataset. Did you ever encounter the same problem, or have any suggestions on how to fix it?

Thank you for your time.
Kind regards,
Paolo

The error message:

Traceback (most recent call last):
File "C:\ProgramData\Anaconda\envs\dev\Scripts\spyking-circus-subtask-script.py", line 11, in
load_entry_point('spyking-circus==0.6.0', 'console_scripts', 'spyking-circus-subtask')()
File "C:\ProgramData\Anaconda\envs\dev\lib\site-packages\circus\scripts\subtask.py", line 43, in main
circus.launch(task, filename, nb_cpu, nb_gpu, use_gpu)
File "C:\ProgramData\Anaconda\envs\dev\lib\site-packages\circus_init_.py", line 21, in launch
module.main(params, nb_cpu, nb_gpu, use_gpu)
File "C:\ProgramData\Anaconda\envs\dev\lib\site-packages\circus\clustering.py", line 996, in main
merged1 = algo.merging_cc(params, nb_cpu=nb_cpu, nb_gpu=nb_gpu, use_gpu=use_gpu)
File "C:\ProgramData\Anaconda\envs\dev\lib\site-packages\circus\shared\algorithms.py", line 429, in merging_cc
os.remove(filename)
WindowsError: [Error 32] The process cannot access the file because it is being used by another process: 'C:\SC\data.overlap-merging.hdf5'

My system config:
Windows 7 x64, 32GB RAM, Intel Xeon E5
python 2.7.13
SpyKING CIRCUS 0.6.0 launched to run on 24 CPUs
The raw-binary .dat file is 3.2GB

Thank you so much for the option to exclude "dead" periods from the recording in version 0.6.0.

However, I found that clustered spikes (those ending up in result.hdf5/spiketimes/temp_) do respect these boundaries, whereas the unclustered spikes (that is, "garbage") do not (those in gspikes/gelec_).

For instance, I see that in fitting.main(), the last time if ignore_dead_times is checked is on line 277 and applied to get good local_peaktimes. But unclustered spikes don't get equally checked before being written to disk (for example, in the if collect_all: block on line 483).
Is that correct?

I saw from the release notes that BRW files are now supported so tried spyking-circus on a recording. I got the error below. Any ideas what is going wrong? This file is 170GB but I can get a smaller recording to share if helpful.

Thanks!

$ spyking-circus R1_E1_AMES_30min_acuity_flashes.brw  -c 12

##################################################################
#####           Welcome to the SpyKING CIRCUS (0.6.2)        #####
#####                                                        #####
#####              Written by P.Yger and O.Marre             #####
##################################################################


Steps         : filtering, whitening, clustering, fitting
GPU detected  : False
Number of CPU : 12/12
Parallel HDF5 : False
Shared memory : False   [please consider upgrading MPI]
Hostfile      : /home/tyler/spyking-circus/circus.hosts

##################################################################


-------------------------  Informations  -------------------------
| Number of recorded channels : 4096
| Number of analyzed channels : 4096
| File format                 : BRW
| Data type                   : uint16
| Sampling rate               : 17 kHz
| Duration of the recording   : 19 min 39 s 397 ms
| Width of the templates      : 5 ms
| Spatial radius considered   : 250 um
| Threshold crossing          : negative
| Waveform alignment          : True
| Matched filters             : False
| Template Extraction         : median-raw
| Overwrite                   : False
| Collect all spikes          : False
| Smart Search                : True
| Smart Selection             : False
------------------------------------------------------------------
-------------------------  Informations  -------------------------
| Large number of electrodes, reducing chunk sizes to 10s
------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/tyler/lib/anaconda3/bin/spyking-circus-subtask", line 11, in <module>
    sys.exit(main())
  File "/home/tyler/lib/anaconda3/lib/python3.6/site-packages/circus/scripts/subtask.py", line 43, in main
    circus.launch(task, filename, nb_cpu, nb_gpu, use_gpu)
  File "/home/tyler/lib/anaconda3/lib/python3.6/site-packages/circus/__init__.py", line 21, in launch
    module.main(params, nb_cpu, nb_gpu, use_gpu)
  File "/home/tyler/lib/anaconda3/lib/python3.6/site-packages/circus/filtering.py", line 280, in main
    data_file_out.allocate(shape=data_file_in.shape)
  File "/home/tyler/lib/anaconda3/lib/python3.6/site-packages/circus/files/raw_binary.py", line 31, in allocate
    self.data = numpy.memmap(self.file_name, offset=self.data_offset, dtype=data_dtype, mode='w+', shape=shape)
  File "/home/tyler/lib/anaconda3/lib/python3.6/site-packages/numpy/core/memmap.py", line 267, in __new__
    offset=array_offset, order=order)
TypeError: 'numpy.float64' object cannot be interpreted as an integer

In 0.5, users can write wrappers to add custom file format. However, there should be a way to quickly test all I/O operations

The bug is hard to reproduce, but sometimes, users are reporting failures in the fitting step when launched with lots of cores. In fact, the code is running fine, BUT some MPI nodes are not working and do not report it. So a problem happens when gathering the spikes at the end

After updating spyking-circus I ran into a failed DLL load on two windows machines with previously working configurations. The call was line 2 of circus' mpi.py,

from mpi4py import MPI

The import also failed directly in python (Anaconda 2.7, windows 7, msmpi 6.0 installed/tested independently/appropriately pathed).

Oddly, the problem was fixed by attempting to downgrade to mpi4py with

pip install -Iv mpi4py==2.0.0

which gave a weird result - appears to download and install a 2.0.0 copy, but the installation reports as being 3.0.0

` Downloading mpi4py-2.0.0-cp27-none-win_amd64.whl (385kB)
Downloading from URL https://pypi.python.org/packages/15/51/6bf7a00ba213768ce9447e1f27fd80786d5f752cf15019d838390983f236/mpi4py-2.0.0-cp27-none-win_amd64.whl#md5=bc03f2cb1b270634a19f767d0a77514b (fr
om https://pypi.python.org/simple/mpi4py/)
98% |############################### | 378kB 1.7MB/s eta 0:00:01 Updating cache with response from "https://pypi.python.org/packages/15/51/6bf7a00ba213768ce9447e1f27fd80786d5f752cf15019d838390983
f236/mpi4py-2.0.0-cp27-none-win_amd64.whl"
Caching due to etag
100% |################################| 389kB 853kB/s
Installing collected packages: mpi4py

Successfully installed mpi4py-3.0.0`

and now

(C:\Users\mouse\Anaconda2) C:\Users\mouse>pip show mpi4py Name: mpi4py Version: 3.0.0

Regardless, the problem and solution were both reproducible locally on the two somewhat distinct rigs. The issue appears to be resolved for me locally, but only with a manual downgrade attempt.

Cyrille Rossant just made some changes to phy that deal with channel order and labelling:

cortex-lab/phy#719

Now, when I try viewing spyking-circus results in phy using circus-gui-python, I get AttributeError: 'TemplateController' object has no attribute 'channel_order'

As suggested by users, we could set a parameter bad_sites allowing users to remove bad channels from the analysis without changing the probe file

It could be nice to have the option to launching the converting script from phy format to Matlab (hdf5) format

Help... I don't know why, but builds are not working anymore on Appveyor. Sould we change the conda install?

For very large data structures (templates/overlaps), we must give indices not as int32, but either uint32 or int64, otherwise this can lead to crashes, sometimes, with thousands of electrodes/templates

Hi,

I am a new user and after installing Miniconda and Spyking Circus, I am trying to run it on a datafile. I didn't have a parameter file yet, so I told the script to make one, but it is failing to do so.

Here is the log:

Do you want SpyKING CIRCUS to create a parameter file? [Y/n] Y
Creating /Users/lawelie/lawelie/Desktop/data/botheyes/170816_145415.params
Fill it properly before launching the code! (see documentation)
------------------------- Informations -------------------------
| Keep in mind that filtering is performed on site, so please
| be sure to keep a copy of your data elsewhere

Traceback (most recent call last):
File "/Users/lawelie/miniconda3/bin/spyking-circus", line 11, in
load_entry_point('spyking-circus==0.6.0', 'console_scripts', 'spyking-circus')()
File "/Users/lawelie/miniconda3/lib/python3.6/site-packages/circus/scripts/launch.py", line 147, in main
shutil.copyfile(config_file, file_params)
File "/Users/lawelie/miniconda3/lib/python3.6/shutil.py", line 121, in copyfile
with open(dst, 'wb') as fdst:
FileNotFoundError: [Errno 2] No such file or directory: '/Users/lawelie/lawelie/Desktop/data/botheyes/170816_145415.params'

Any suggestion as to how I can proceed?
Even if I put a parameter file in the folder myself (that I found on this repository), it doesn't run.

Best regards,
Ingrid

Dear Pierre,

I found a memory leak in the functions that use dead_times. The problem is due to appending lists in a for-loop while the function creates the indices of the dead_times. For example, I ran the whitening step on my data (ca. 60 minutes, 3 channels, 11 dead intervals, shared memory = True) on my machine (24 logical CPUs, 32 GB memory, 64bit Windows 7). The code that causes the memory leak in whitening.py is on lines 280-282 (in the latest version).

I tried:

• commenting lines 281-282, which leaves the list empty: peak memory usage was 9.23 GB
  

• your implementation using lists (peak memory usage is > 31.8 GB):

all_dead_times = []
for i in xrange(len(dead_times)):
    all_dead_times += range(dead_times[i, 0], dead_times[i, 1])

• replacing lines 280-282 with a similar numpy implementation (peak memory 18.0 GB):

n_samples = dead_times.max()
mask = np.zeros((n_samples + 1, ), dtype=bool)
for i in xrange(len(dead_times)):
    mask[dead_times[i, 0]:dead_times[i, 1]] = 1
all_dead_times = np.where(mask)[0]

• First part of this SO answer (peak memory usage 15.9 GB):

start = dead_times[:, 0]
end = dead_times[:, 1]
all_dead_times = np.r_[tuple(slice(s, e) for s, e in zip(start, end))]

• Second part of the same SO answer (peak memory usage 11.7 GB):

def indices(start, end):
    lens = end - start
    np.cumsum(lens, out=lens)
    i = np.ones(lens[-1], dtype=int)
    i[0] = start[0]
    i[lens[:-1]] += start[1:]
    i[lens[:-1]] -= end[:-1]
    np.cumsum(i, out=i)
    return i
start = dead_times[:, 0]
end = dead_times[:, 1]
all_dead_times = indices(start, end)

From these tests it seems like the last implementation might be the faster and more memory efficient among those I tried. Would it be possible to replace the original implementation in both whitening and the other functions that use dead_times?

Thanks a lot.

Currently, since we made the choice of sparse matrices everywhere, the GPU usage is not really efficient and do not bring much compared to several CPUs. However, by re-writing custom CUDA kernels, there is a large room for improvements

In 0.5, by allowing users to load custom file formats, we have to face the problem that sometimes data are recorded in several chunks or blocks. Ideally, we would like a nice and neat syntax to process all those blocks as a whole, so data must be streamed transparently

If I run open ephys data that I know contains clear spikes through spyking circus, then look at the results in either gui, the spike templates look too "sharp." Instead of the familiar waveform, I see very narrow negative-going spikes. I've seen something similar out of other sorting packages (klustakwik) when I've forgotten that the samples are big endian. It looks like endianness is handled really well within openephys.py (and its superclass), but something is going on. I'm posting this here in case it jogs someone's memory, but will do some more digging on my own today.

Tests for last commits are failing on AppVeyor, with a weird error message. Should we worry or is it a problem of the platform itself, or our integration script? Note that not all python versions are crashing..

I'd like to propose that we add a feature to allow the artifact times to be expressed in samples instead of msec. I think this is probably going to be more precise and, in my case, just as convenient. We're extracting our stimulation times from TTLs on our physiology acquisition system. It is super easy to just use these numbers in samples. I'm a bit concerned that rounding errors might actually introduce problems when converting to msec.

Let me know, I can do a pull request if you agree!

Hey,

I am facing a problem when trying to visualize data using the phy gui. I processed the data and ran the conversion step on a cluster and then transferred all the data onto my pc. When i run the circus-gui-python path/file.extension command, this is the result:

-------------------------  Informations  -------------------------
| Launching the phy GUI...
------------------------------------------------------------------
19:52:20 [I] messages:49          Launching the phy GUI...
19:52:20 [E] cli:39               An error has occurred (ValueError): shapes (493,61,1) and (96,96) not aligned: 1 (dim 2) != 96 (dim 0)

My guess is, 96 corresponds to the number of channels in my dataset, but I don't know what the (493, 61, 1) could correspond to.

I then inserted certain print commands into the python_gui.py file and discovered that the error occurs at line 132, where the TemplateController class is called from phycontrib. Before I go forth debugging phycontrib, I wanted to ask you if there was a way to fix this error in spyking-circus itself. Or, if there was a way to get a more helpful error message.

Thanks!

Sometimes, for quick analysis, we don't need to perform spike sorting. Therefore, it would be good to have the option to simply export MUA, per electrode.

After an intensive debugging section with Windows users discovering SpyKING CIRCUS, I realized that the .params file is not very intuitive for many users that do not have a proper command-line approach.

Most of them are using text editors such as word, not realizing that the extension is often modified (and windows hides it). So my request is the following: can we make, in the small launching GUI, a third tab or a view displaying the content of the parameter file (when data are selected), such that the user can view/edit it from the GUI. It will decrease the amount of possible mistakes....

For very large arrays (> 500) or very large number of templates (> 2000) we could think about using the internal compression mecanism provided by HDF5. This would save some disk space, especially for overlaps. Moreover, on clusters, we could use blosc to try to reduce the amount of data exchanged between nodes

Dear Dr. Yger,

I've got an error after update, 0.7.0 version. I couldn't figure out what causes the error. I tried running with many different parameters, but the result was the same. so.. I am just copying and pasting the message below.. The error comes out in the middle of "Searching random spikes to sample amplitudes ....".

The thing is, when using previous version (0.6.7), it run properly without any issues (with exactly same data and same parameters).

In addition, the error happens for some of my data; interestingly, it runs properly on some other data (parameters & probe are all the same). The data I am referring are from the same experimental design.

I don't wanna go back to previous version because I like the new version fixing some bugs that I was struggling with.

Below is the error message I got.

Traceback (most recent call last):
File "c:\programdata\anaconda3\lib\runpy.py", line 193, in run_module_as_main
"main", mod_spec)
File "c:\programdata\anaconda3\lib\runpy.py", line 85, in run_code
exec(code, run_globals)
File "C:\ProgramData\Anaconda3\Scripts\spyking-circus-subtask.exe_main.py", line 9, in
File "c:\programdata\anaconda3\lib\site-packages\circus\scripts\subtask.py", line 43, in main
circus.launch(task, filename, nb_cpu, nb_gpu, use_gpu)
File "c:\programdata\anaconda3\lib\site-packages\circus_init.py", line 21, in launch
module.main(params, nb_cpu, nb_gpu, use_gpu)
File "c:\programdata\anaconda3\lib\site-packages\circus\clustering.py", line 415, in main
f = scipy.interpolate.RectBivariateSpline(xdata, ydata, zdata, s=zdata.size*mads[elec]**2, kx=3, ky=1)
File "c:\programdata\anaconda3\lib\site-packages\scipy\interpolate\fitpack2.py", line 1180, in init
raise ValueError(msg)
ValueError:
the maximal number of iterations maxit (set to 20 by the program)
allowed for finding a smoothing spline with fp=s has been reached:
s too small.
Weighted sum of squared residuals does not satisfy abs(fp-s)/s < tol.

I see that Travis and AppVeyor test out Python 3 and the FAQ claims Python 3 support, but it seems that circus.shared is not really tested as it contains many bits of Python 2 specific code.

For example, circus/shared/__init__.py uses:

import files
import parser
import algorithms
import plot
import utils

These relative imports don't work any more and must be made absolute.

There's also imports of cPickle in many places.

cc @SaraMahallati

When I try to run spyking-circus on my openephys data files, I get the following error:

$ spyking-circus Continuous_Data.openephys -c 8 -H ~/Documents/Code/spyking-circus/circus.host
----------------------------  Error  -----------------------------
| There is an error in the _read_from_header method of the wrapper
'encoding' is an invalid keyword argument for this function
------------------------------------------------------------------
----------------------------  Error  -----------------------------
| Shape of the data is not defined. Are you sure of the wrapper?
------------------------------------------------------------------

There is no information in the log file about this error.

The top of my params file is:

[data]
file_format    = openephys             # Can be raw_binary, openephys, hdf5, ... See >> spyking-circus help -i for more info
stream_mode    = multi-files                  # None by default. Can be multi-files, or anything depending to the file format

Is there an error in how I'm calling spyking-circus?

I just added blosc as a dependency (both in setup.py and in the config files for appveyor and travis). However, the builds are partially failing (see blosc branch), and I'm not sure why... Help appreciated

Hello,

I am trying to launch circus on file /N/dc2/scratch/mmyroshn/res/maze/raw/hp26.dat

IOError: Unable to open file (Unable to open file: name = '/n/dc2/scratch/mmyroshn/res/maze/raw/hp26/hp26.basis.hdf5', errno = 2, error message = 'no such file or directory', flags = 0, o_flags = 0)

Hello,

I am running spyking-circus 0.6.7 in a conda environment loaded on a cluster running CentOS/RHEL at my institute. I am analyzing a raw_binary dataset that I converted from a blackrock ns6 file. The cluster uses slurm to distribute tasks over multiple nodes and I have the miniconda version of mpi4py (3.0.0) and mpich (3.2.1) installed alongside spyking-circus in my conda environment. mpiexec --version yields the following:

HYDRA build details:
Version: 3.2.1
Release Date: Fri Nov 10 20:21:01 CST 2017
CC: gcc -I/home/s.sridhar/miniconda2/envs/spiking/include -m64 -Wl,-rpath,/home/s.sridhar/miniconda2/envs/spiking/lib -L/home/s.sridhar/miniconda2/envs/spiking/lib
CXX: g++ -I/home/s.sridhar/miniconda2/envs/spiking/include -DBOOST_MATH_DISABLE_FLOAT128 -m64 -Wl,-rpath,/home/s.sridhar/miniconda2/envs/spiking/lib -L/home/s.sridhar/miniconda2/envs/spiking/lib
F77: gfortran -Wl,-rpath,/home/s.sridhar/miniconda2/envs/spiking/lib -L/home/s.sridhar/miniconda2/envs/spiking/lib
F90: gfortran -Wl,-rpath,/home/s.sridhar/miniconda2/envs/spiking/lib -L/home/s.sridhar/miniconda2/envs/spiking/lib
Configure options: '--disable-option-checking' '--prefix=/home/s.sridhar/miniconda2/envs/spiking' '--disable-dependency-tracking' '--enable-cxx' '--enable-fortran' 'CC=gcc' 'CFLAGS= -m64 -O2' 'LDFLAGS= -Wl,-rpath,/home/s.sridhar/miniconda2/envs/spiking/lib -L/home/s.sridhar/miniconda2/envs/spiking/lib' 'CPPFLAGS= -I/home/s.sridhar/miniconda2/envs/spiking/include -I/feedstock_root/build_artefacts/mpich_1520073382972/work/mpich-3.2.1/src/mpl/include -I/feedstock_root/build_artefacts/mpich_1520073382972/work/mpich-3.2.1/src/mpl/include -I/feedstock_root/build_artefacts/mpich_1520073382972/work/mpich-3.2.1/src/openpa/src -I/feedstock_root/build_artefacts/mpich_1520073382972/work/mpich-3.2.1/src/openpa/src -D_REENTRANT -I/feedstock_root/build_artefacts/mpich_1520073382972/work/mpich-3.2.1/src/mpi/romio/include' 'CXX=g++' 'CXXFLAGS= -DBOOST_MATH_DISABLE_FLOAT128 -m64 -O2' '--cache-file=/dev/null' '--srcdir=.' 'LIBS=-lpthread ' 'MPLLIBNAME=mpl'
Process Manager: pmi
Launchers available: ssh rsh fork slurm ll lsf sge manual persist
Topology libraries available: hwloc
Resource management kernels available: user slurm ll lsf sge pbs cobalt
Checkpointing libraries available:
Demux engines available: poll select

I encounter an error during the fitting phase of the analysis. A small part of the error is given here:

Traceback (most recent call last):
File "/home/s.sridhar/miniconda2/envs/spiking/bin/spyking-circus-subtask", line 11, in
load_entry_point('spyking-circus==0.6.7', 'console_scripts', 'spyking-circus-subtask')()
File "/home/s.sridhar/miniconda2/envs/spiking/lib/python3.6/site-packages/circus/scripts/subtask.py", line 43, in main
circus.launch(task, filename, nb_cpu, nb_gpu, use_gpu)
File "/home/s.sridhar/miniconda2/envs/spiking/lib/python3.6/site-packages/circus/init.py", line 21, in launch
module.main(params, nb_cpu, nb_gpu, use_gpu)
File "/home/s.sridhar/miniconda2/envs/spiking/lib/python3.6/site-packages/circus/fitting.py", line 134, in main
c_overs = io.load_data_memshared(params, 'overlaps')
File "/home/s.sridhar/miniconda2/envs/spiking/lib/python3.6/site-packages/circus/shared/files.py", line 501, in load_data_memshared
win_indices = MPI.Win.Allocate_shared(indices_bytes, intsize, comm=sub_comm)
File "mpi4py/MPI/Win.pyx", line 101, in mpi4py.MPI.Win.Allocate_shared
mpi4py.MPI.Exception: Other MPI error, error stack:
PMPI_Win_allocate_shared(162).......: MPI_Win_allocate_shared(size=0, MPI_INFO_NULL, comm=0x84000002, baseptr=0x7fff9a10b810, win=0x7f0b8bc45ad8) failed
MPID_Win_allocate_shared(208).......:
win_init(276).......................:
MPIR_Comm_dup_impl(57)..............:
MPIR_Comm_copy(739).................:
MPIR_Get_contextid_sparse_group(614): Too many communicators (0/2048 free on this process; ignore_id=0)
Traceback (most recent call last):
File "/home/s.sridhar/miniconda2/envs/spiking/bin/spyking-circus", line 11, in
load_entry_point('spyking-circus==0.6.7', 'console_scripts', 'spyking-circus')()
File "/home/s.sridhar/miniconda2/envs/spiking/lib/python3.6/site-packages/circus/scripts/launch.py", line 364, in main
subprocess.check_call(mpi_args)
File "/home/s.sridhar/miniconda2/envs/spiking/lib/python3.6/subprocess.py", line 291, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['mpiexec', '-np', '24', 'spyking-circus-subtask', 'fitting', '/tmp/i140703-001.dat', '24', '0', 'False']' returned non-zero exit status 1.

The entire error output was much longer, but was basically the above error repeated several times. I have attached the spyking-circus output along with the error output as text files, if you need them. i1407003-001-err.txt
i1407003-001-out.txt It looks like the error is in MPI, with the message "Too many communicators ...".

Any idea what is going on and how to fix it?

https://ci.appveyor.com/project/spyking-circus/spyking-circus/build/1.0.496
Should we do something, or just wait to let it pass?

Whenever I try to run on Openephys I get the following error:
---------------------------- Error -----------------------------
| The file format openephys does not support stream mode multi-file

But in the documentation it says it should be supported? If I change it to None I can run it on a single instance, but am unable to run it over multiple CPUs.

Using phycontrib 1.0.15, circus 0.5.7, and python 2.7 or 3.5, I get the following after calling circus-gui-python Continuous_Data.openephys:

-------------------------  Informations  -------------------------
| File format openephys is not supported by phy. TraceView disabled
------------------------------------------------------------------
06:58:21 [I] messages:49          File format openephys is not supported by phy. TraceView disabled
-------------------------  Informations  -------------------------
| Gain of 0.195 is not supported by phy. Expecting a scaling mismatch
------------------------------------------------------------------
06:58:21 [I] messages:49          Gain of 0.195 is not supported by phy. Expecting a scaling mismatch
-------------------------  Informations  -------------------------
| Launching the phy GUI...
------------------------------------------------------------------
06:58:21 [I] messages:49          Launching the phy GUI...
06:58:22 [W] gui:264              Error while loading data: File giverandomname.dat not found.
06:58:38 [E] cli:39               An error has occurred (RuntimeError): context could not be created

Am I missing some obvious step, or is this a known issue... or can you give any beta that might speed up debugging?

For people that do not want to duplicate large datasets, maybe the filtering could be performed in memory rather than on-site?

• Pros: saving disk space
• Cons: slowing down slightly the code, and we need to be sure that phy is performing exactly the same filtering when displaying the data

For very long recordings, we must use uint32 to be more efficient

Because for now, GPU kernels are not optimized, and because this is not always simple to install cudamat on Windows, cudamat should not be activated by default during the install

We should refactor the code, when neo.rawio will be out, rely on neo as a dependence and have a common way to read/write proprietary file formats. This would make everthing easier, and phy could then also import neo to visualize the raw data

Actually, I don't know if this is already the case, and if not, if this is complicated to add such a support. As Qt5 will become soon the default, maybe it is worth implementing that

I'm not sure if I should ask this here or in phy-contrib, but my guess is something is missing in the 'converting' phase.

phy contrib documentation says there should be a templates_ind.npy to be able to track the channels on which each template is defined and I don't have that when I run the Spyking Circus python GUI.

I need to attribute the templates of my 'Good' clusters to the channels since the templates.npy gives me all the templates on all the channels.

I know that this is not as easy as it sounds, because of the way the code is structured, but can we setup an automatic code coverage workflow for 0.5? I guess with coveralls.io (I created an account). In the future, we should write more unit tests, as the code may start to be more complicated with the RT extension.

Hi,
I get an error when converting files after the main processing steps. The issue seems to be with opening the result files:
File "h5py\h5f.pyx", line 78, in h5py.h5f.open
IOError: Unable to open file (file is already open for write
Here's the full error output:

`Traceback (most recent call last):
File "C:\ProgramData\Anaconda2\envs\spkc27\Scripts\spyking-circus-subtask-script.py", line 11, in
load_entry_point('spyking-circus==0.6.2', 'console_scripts', 'spyking-circus-subtask')()
File "c:\programdata\anaconda2\envs\spkc27\lib\site-packages\spyking_circus-0.6.2-py2.7.egg\circus\scripts\subtask.py", line 41, in main
circus.launch(task, filename, nb_cpu, nb_gpu, use_gpu, extension=extension)
File "c:\programdata\anaconda2\envs\spkc27\lib\site-packages\spyking_circus-0.6.2-py2.7.egg\circus_init_.py", line 19, in launch
module.main(params, nb_cpu, nb_gpu, use_gpu, extension)
File "c:\programdata\anaconda2\envs\spkc27\lib\site-packages\spyking_circus-0.6.2-py2.7.egg\circus\converting.py", line 263, in main
N_tm = write_templates(output_path, params, extension)
File "c:\programdata\anaconda2\envs\spkc27\lib\site-packages\spyking_circus-0.6.2-py2.7.egg\circus\converting.py", line 86, in write_templates
templates = io.load_data(params, 'templates', extension)
File "c:\programdata\anaconda2\envs\spkc27\lib\site-packages\spyking_circus-0.6.2-py2.7.egg\circus\shared\files.py", line 619, in load_data
'r', libver='latest').get('temp_x')[:].ravel()
File "c:\programdata\anaconda2\envs\spkc27\lib\site-packages\h5py_hl\files.py", line 269, in init
fid = make_fid(name, mode, userblock_size, fapl, swmr=swmr)
File "c:\programdata\anaconda2\envs\spkc27\lib\site-packages\h5py_hl\files.py", line 99, in make_fid
fid = h5f.open(name, flags, fapl=fapl)
File "h5py_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
File "h5py_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
File "h5py\h5f.pyx", line 78, in h5py.h5f.open
IOError: Unable to open file (file is already open for write (may use to clear file consistency flags))
Traceback (most recent call last):
File "C:\ProgramData\Anaconda2\envs\spkc27\Scripts\spyking-circus-script.py", line 11, in
load_entry_point('spyking-circus==0.6.2', 'console_scripts', 'spyking-circus')()
File "c:\programdata\anaconda2\envs\spkc27\lib\site-packages\spyking_circus-0.6.2-py2.7.egg\circus\scripts\launch.py", line 354, in main
subprocess.check_call(mpi_args)
File "C:\ProgramData\Anaconda2\envs\spkc27\lib\subprocess.py", line 186, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['mpiexec', '-np', '1', 'spyking-circus-subtask', 'converting', 'D:\Data\Ephys\031S_1103\031S_1103_WR_MN_32Ch_nopp.dat', '1', '0', 'False', 'None']' returned non-zero exit status 1

Process exited with exit code 1`

Hi,

I've ran into this error with two different data types (nwb,npy):

...circus/shared/files.py", line 731, in load_data
basis_proj = numpy.ascontiguousarray(myfile.get('proj')[:])
TypeError: 'NoneType' object is not subscriptable

The params file wasn't changed except for the necessary changes on [data], the other thing is that I'm not doing whitening, as I'm working with tetrode data. When I included whitening, I got a different error:

...circus/shared/utils.py", line 679, in _fit
    % (n_components, n_features))
ValueError: n_components=5 invalid for n_features=3

python version 3.6, running on a mac