kwikteam / phy-contrib Goto Github PK

Might be helpful to be able to view more of the CCG at once by having the time axis be on a log scale. This way you can see the details of the refractory period, but also see the structure out to a second or more. Would definitely need axis labels in that case.

in /plugins/kwik_gui/

If you merge two clusters, these seem to stop appearing in the list of similar clusters for others. E.g.

Then I merge 441 and 1336, and when I select the new cluster (1770) I correctly see:

But if I select 185:

1770 should be in the list!! With a score of 0.931 (the max of its constituent templates against 185).

Was trying to move a cluster back to "unsorted". Put "3" in the box and pressed ok, but got:

1. A list of all snippets and a description of how to use them (Cyrille, I
   went back to an old email where you had directed me how to find them, but
   the links there are broken now)
2. A list of any parameters that can be set in a user_settings.py (is there
   such a thing anymore?) and/or phy_config?

For next release of SpyKING CIRCUS, we are planning to allow users to keep their data as they are, reading/writing directly to some classical file formats (openephys, multi-channels system, NWB, ...). This is because for large scale recordings, data files can be very large, and we would like to avoid an export to raw binary, as this can take time and disk space. The problem is that phy is currently only able to deal with raw binary files, am I right? Or I guess because of kwik format, any HDF5-like structure could easily be supported? How can we design wrappers to tell phy how to properly read chunks of data, then?

When I have automatic zoom on, if I set the waveform view to look like this:

(which looks good to me) then go to the next unit I now see this:

(in this case I actually went to the next cluster but then back to the first one, just so it's most clear what the difference is). I would prefer it keep the proportions that I had decided were best for viewing spikes and instead only shifted the "x,y camera position" to be above the new cluster.

Could it be possible to have an option such that in the Waveform view, individuals waveforms would be multiplied by the corresponding amplitudes in order to be properly scaled? This could be useful for visualization sometimes

If the unit color were in cluster view, it would facilitate matching up something in traceview with its cluster, e.g. when the list is sorted by depth.

An even better solution would be a hotkey+click on traceview to select the unit of the nearest spike! Would that be possible?

Just to give it a try, I wanted to launch the GUI on larger datasets, and it throws an error. Again, no rush, I'm just reporting in case it can help

15:20:33 [E] cli:37 An error has occurred (ValueError): insecure string pickle

If we're closing in on a stable version of the template gui (not sure whether that is true, but if it is - ), could we think about adding a PSTH view? This can be very simple:

• If there is a event_times.npy then this view is created. event_times is [nEvents,].
• It shows a PSTH and rasters within a window around the events, like this:
  
• There is some kind of snippet and/or hotkey to control the two parameters: window size (in the screenshot, it is -0.5 to 2.5 seconds around each event) and smoothing width (the PSTH is generated by making a histogram with a very small bin size (e.g. 1ms) then convolving it with a gaussian of some width).

We could go from there to thinking about a sensible system for the two extensions you might want: 1) support for multiple different events that you could flip between or show side-by-side; 2) support for "event labels" which tells which type of event each event_times was, i.e. if there are 200 event_times maybe 50 were for stimulus1, 50 for stimulus2, etc, and you want to show 4 traces and color the raster points accordingly. But we should have a discussion about how to implement all that sensibly if we get there.

I.e. I sort by spike count. I take the next one in the list, merge it with some other units, now it is higher in the list. But if I assign to good, I don't want to now see the next one from its current position in the list - I want to see the next one I haven't yet considered, which is the next one from its original position in the list.

Just to be sure, because on some datasets I have a TraceView that do not make any sense. Can it be that the offset in the data file is not well taken into account?

Best

Pierre

• install phy
• make sure the plugins/ dir is discoverable
• run the plugin tests

Un-whitened templates are subtracted from the raw data for visualization, but the templates were not only whitened but also high-pass filtered. I don't think there can be any way to un-filter them, so this leads to the subtraction being imperfect.

I guess the only options to make it accurate are: 1) Change the requirements of the template-matching software (kilosort, spyking circus) such that they go ahead and return templates that are from the raw data, not the whitened or filtered data; 2) Have the option to show the raw data whitened and filtered instead of raw. Then subtract the original (whitened) templates from it. Option 2 seems distinctly preferable, both because you might want to see the whitened/filtered data anyway, and because having the templates generated from un-filtered data would potentially introduce a lot of noise into them. But it has the problem that the filter must be specified somehow so phy can do it correctly.

Low priority...

I already sent you some suggestions by emails, but let's now switch to git to keep track of them:

• toggle on/off the electrodes numbers
• I agree that the color of the selected cluster should be highlighted in the TraceView.
• The GUI should work without the datafile (thus no TraceView)
• In the TraceView, have a command to be able to jump to a particular time
• There is a bug when switching the Waveform view to Mean view (with w). The zoom level is not kept in my case
• The number of waveform to select from each cluster should be configured (see #12)
• or alternatively display them only in the local area of the template. Because for high channel count > 252, the GUI take ages to open. We already spoke about that, but we could plot only Traces from the channels involved in a given templates, or in a selected regions drawn on the probe design by the user (see #11)
• And as we spoke, the possibility to click on a spike in the TraceView to get the appropriate cluster would be great (see #8)

How are the spike colorcoded in the TraceView. How do you decide that the color is applied or not? If there is a spike, you do not color all channels belonging to the template, but only a subset. How is this subset chosen?

Otherwise, this is great ! Happy to make the switch from MATLAB soon !!

According to the labels in the feature view, left click assigns feature 1 (indexed as 0) of the selected channel to slots 1 & 4, and feature 2 (indexed as 1) to spot 3. However, the data for slot 4 don't change. Right click seems to add feature 1 of the selected channel to slot 2, and feature 2 to slot 4. However in this case, only the data are updated, whereas the label does not. I think this is a bug and the label for slot 4 should change on right click (along with the data), but not left click.

In the feature view of phy kwik-gui, the scatter plots of selected clusters seems to have a different scale than the background noise. If I zoom out a lot (ctrl -) I can see clouds of point out of the noise, but even if I select all my clusters, I cannot find any unit corresponding to these point. All the selected features appear to have a different scale. A picture is probably better than any explanation, here is a screenshot.

If I zoom in a lot, it seems that the distribution of points is similar to that seen in the background, but at a different scale (screenshot).

Hi again, just some thoughts on the GUI. The job achieved with phy is impressive, but I'm just writing down here ideas that may be useful for exploring template-matching data. As I'm soon done with the main code of SpyKING CIRCUS 0.3, I may start to help also with phy-contrib (once this is stable enough, but it seems to almost be the case!)

• We could click on a given point in the AmplitudeView such that it should interact with the TraceView and center it around that particular spike time.
• I still think that an axis (or at least, in a more elegant manner, a scale bar) should be here for the AmplitudeView plot
• We have our "meta-merging" GUI, currently in Qt, and I'm wondering how complicated it will be to use phy instead. The best may be to directly see that with Cyrille
• We could imagine some color shaded regions in the AmplitudeView trace, corresponding to different part of the recording session/stimulation, such that it can help people to quickly see if that cells is firing at time they expect it to fire. For example, the user could provide an extra file stimulation_times.npy as a 3d array with t_start on the first column, t_stop on the second, and a digit for a label/color. Then, based on that, regions in the AmplitudeView will either be colorshaded as function of the labels, or more simpler, we can just draw a color bold line on top of the cloud of dots. This will allow user to quickly see how responses are impacted by the stimulation, if cell is firing during spontaneous activity, ...
• I think that people should be able to assign arbitrary labels to neurons, such as for example 'reponding to light', 'responding to movements', ... They can be useful for further analysis
• We could have an option to quickly highlight all templates that have less than X spikes? In fact, there should be a way to delete several templates at once.

Hi there,
not sure what I'm missing in the installation procedure but something's not going right for me.
Easy install with > pip install phycontrib returns:
Collecting phycontrib
Could not find a version that satisfies the requirement phycontrib (from versions: ) No matching distribution found for phycontrib

Advanced install: there's no plugins directory in phy. I create one, then clone the source code. I end up with a directory structure that looks like this: \phy\plugins\phy-contrib\phycontrib
I tested import IPlugin from phy, and it works. But then calling phycontrib results in this error:
from phycontrib.template import create_template_gui
File "C:\Anaconda\envs\spykc\lib\site-packages\phycontrib-1.0-py2.7.egg\phycontrib__init__.py", line 16, in
from . import kwik_gui # noqa
ImportError: cannot import name kwik_gui

Yet the kwik_gui directory is there and should be accessed.
I tried variations on directory structures, and running phy-contrib install.

Any tip on how to get this running?

:filter rate>10 where the expression after filter is a Python expression to be evaluated on every cluster, the fields corresponding to the fields registered in the cluster view

Amplitude view makes it so that the first spike of each cluster is at time=left edge of plot and the last is at time=right edge of plot. Instead the time axis should go from the beginning to the end of recording. E.g. here is amplitude view:

And here is the same two clusters in feature view:

You can see they're misaligned relative to each other (the main group of red points comes before the blue in feature view but not in amplitude view) - it's feature view that's correct, and amplitude view is mis-scaled.

Otherwise, switching shank raises an error (one has to delete the cache before switching).

It appears to be somewhere below and to the left of the center of the plot, where "center of the plot" means the location of the crosshairs.

Hi all

I have a question following a post that we had on our software https://groups.google.com/forum/#!category-topic/spyking-circus-users/P9Dlp4SCFU0

My question is as follow: would it be possible, when merging templates, to take into account the fact that there might be a temporal lag between two templates? This phenomena appears in our case when we allowed the code to detect positive peaks, as a request from several users. Imagine a biphasic spike, crossing both positive and negative threshold. We might end up with two templates, slightly time shifted, centered on the positive and on the negative peaks. We will think on our side about ways to avoid that, but I was still raising the issue here as it may be relevant for someone else. When merging time-delayed version of templates, with a lag (that could easily be detected as the maximum of the delayed CC), that lag should impact the spike times of the merged cluster... Hope I am clear enough

Best

Big picture idea: often there are several merges to make where the ACGs and CCG of the two clusters are all very similar. If we make a second similarity score that's about the ACG/CCG between a pair of templates (whether these three look the same), then we should just go ahead and select all units that score high on this, so you can merge them all at once.

E.g. if there are three or more units like this, just show them all at once right away:

cc @kdharris101

This is a feature enhancement request:
As I understand it, splitting clusters only works based on a selection made in Feature view. It would be nice (and effectively quicker sometimes) to be able to split clusters in WaveformView as well.

If 0, disable filtering in the waveform view

Although you can add custom fields in a plugin, we could add more fields by default.
There shouldn't be too much of a performance hit, as we already compute and cache the mean features/waveforms/templates etc. In addition to that, all cluster statistics are cached.

@nsteinme  @yger please complete the following list:

• firing rate
• ...

Feel free to post some bits of Python code too if you want.

It would be good if there were an option - something like ctrl+click on a certain feature plot - that would paint all the points for any spikes that have a PC feature on the channel(s) in that feature plot. All these points, if not among the currently selected clusters, would be grey. Perhaps it could paint them all up to some limit N (like 50 or 100 thousand). This would be similar to the old klustaviewa style in which you'd see all the features on that channel, making it easy to identify if you have missed a cluster that is similar to the present one.

Hi,
I installed klusta, phy and phy contrib in a conda environment on Windows 7. When I run "phy kwik-gui hybrid_10sec.kwik", it returns:
Error: No such command "kwik-gui"

I've forced an upgrade on phycontrib, just to check - no difference.

The output from >phy kwik-gui --debug is:

23:55:56 [D] config:45            Load config file `C:\Users\Vincent\.phy\phy_config.py`.
23:55:56 [D] plugin:35            Register plugin `__builtin__.MyPlugin`.
23:55:56 [D] cli:87               Attach plugin `__builtin__.MyPlugin` to CLI.
Usage: phy [OPTIONS] COMMAND [ARGS]...

Error: No such command "kwik-gui".

I'm not sure if that's the issue, but phy_config.py is in C:\Users\Vincent.phy\ and the kwik_gui folder is found in C:\Anaconda\envs\klusta\Lib\site-packages\phycontrib. However, adding the relevant directories to PATH does not solve the issue.

Would that be difficult? If so, we can forget it.

to put in /phycontrib/kwik/:

• KwikModel and tests
• KwikCreator

Because the bug is still here in fact, and even if I mentioned it before, I create a separate issue to keep track of it. It can already be seen with 252 channels

The problem seems to be that when changing the ranges with Ctrl+ arrows, the numbers are not aligned anymore with the channels. This is annoying because after, when Zoom on channel is activated, the GUI zoom on the number and not on the channel when switching clusters...

I'm not immensely experienced with Ubuntu and remote access, so this may be a naive or silly question...

I am unable to run phy template-gui params.py --debug from a remote display. See screenshot of error message below. The identical command works fine when run from the physical machine.

I use MobaXterm 9.0 in Windows XPS 13 to run the Matlab GUI on a labmate's Ubuntu (14.04.4 LTS x86_64) machine in another room. We have tried VNC Viewer and NoMachine but were unable to simultaneously use separate accounts on the machine: displays were either duplicated or just an empty gray screen with a cursor.

Given that both the Matlab and Firefox GUIs work fine from MobaXterm, I thought perhaps the issue is something specific to Phy or, more likely, how I'm using it.

Any thoughts? Thanks!

I have verified that new_cluster_id.json is in fact missing from that location (and all subdirectories), however that doesn't seem to be a problem when running from the physical Ubuntu machine.

When pressing "w", the templates does not have the same scaling as the raw data. However, in our case, it should, and the mean over several whitened waveforms should be the same as the template (not normalized). Just be sure that there is not something fancy on under the carpet. I'll also double check our phy export

The environment.yml file in the repository differs from that in the installation instructions. Why is there this duplication?

The GUI is working fine, however, in the shell, I have the following error

/home/pierre/github/phy/phy/plot/transform.py:184: RuntimeWarning: invalid value encountered in divide
return t0 + (t1 - t0) * (arr - f0) / (f1 - f0)

I'm just reporting it here to keep it in mind, but I don't think we should worry about for now

It would be great if there was an option to replace a given cluster by the original clusters (that appear in spike_templates.npy). This would allow Kilosort to make putative (but not final) merges, write those into spike_clusters.npy and then allow the user to undo Kilosort's merge.

Could they turn off whenever they overlap one of the waveform traces, or possibly it would be just as good if they were simply plotted behind the waveform traces? I.e. this is great:

But this is not so good:

Can I have one with waveforms not overlaid, and another at the same time that shows the templates overlaid? I switch back and forth between these a lot, it would be better just to have them both all the time.