melizalab / jill Goto Github PK

pthread needed in library list

Provides a mechanism for programs to influence the behavior of running clients.

This metadata is extremely useful when looking at event times, because it tells you the endpoint of the entry. Otherwise you have to hope there's a sampled dataset in the entry and calculate it from there.

The jill prerequisites specify zmq22, but a more recent version is needed for installation (zmq @4.3.2_0 worked): sudo port install scons boost libsamplerate libsndfile zmq22

Also, it would be nice to add the information about the no-arf install like the info in the Debian section: scons -Q --no-arf modules

For some experiments, each stimulus is its own trial (e.g. calcium imaging where we want to space out the times when the excitation light is on), and we need to be able to start acquisition some amount of time before the stimulus starts. This should be pretty easy to implement, because the realtime thread already calculates how many samples until it starts the next stimulus. We need to distinguish this event from the ordinary stim_on events, because the pre-stimulus trigger will typically need to be emitted on a different channel from the ordinary sync messages.

If it's not too much trouble we should have a post-stimulus trigger event as well. With the proper downstream circuitry (some kind of latch) this could be used to generate a pulse that extends from the start of acquisition to the end.

This program would provide a convenient interface for turning on and off hardware monitoring on specific channels.

This is related to #19. When two triggered entries occur in succession, and there is not enough data to fill the prebuffer for the second entry, the second entry should begin on the sample immediately after the first entry ended. This is to allow downstream applications to stitch entries together during analysis.

Currently user can enable/disable clicks for onset and offset events but more precise control may be needed, for example, if jstim is generating is a pre-stimulus trigger and stim-on/stim-off events.

As noted in #24, a post-stimulus trigger event may be needed if the data acquisition device needs the TTL signal to be high for the duration of the recording. This is indeed the case for the inscopix box, so this will need to be added.

I have two 32 channel single ended headstages plugged into the RHD2000 eval board. One is plugged into port A, one into port B on the board.

Starting jackd with:
jackd -d rhd2000 -A 0xffffffff,1,15000,1,0

should only activate the 32 channels on port A1, however, 32 more channels show up in A2. See output below. I get the same result if I start jackd with

jackd -d rhd2000 -A 0xffffffff,1,15000,1,0 -B 0xffffffff,1,15000,1,0

output:

jackd 0.122.0
Copyright 2001-2009 Paul Davis, Stephane Letz, Jack O'Quinn, Torben Hohn and others.
jackd comes with ABSOLUTELY NO WARRANTY
This is free software, and you are welcome to redistribute it
under certain conditions; see the file COPYING for details

JACK compiled with System V SHM support.
loading driver ..
RHD2K: initializing driver
opal kelly driver: /home/dmeliza/src/jack_rhd/driver/libokFrontPanel.so
FPGA bitfile: /home/dmeliza/src/jack_rhd/driver/rhythm_130302.bit
RHD2K: scanning SPI ports
RHD2000 Controller:
Opal Kelly Front Panel version: Feb 1 2012 17:18:14
Opal Kelly device ID: Opal Kelly XEM6010
Opal Kelly device serial number: 000000050D2
Opal Kelly device firmware version: 3.1
FPGA frequency: 100 MHz
Rhythm version: 1
Sampling rate: 30000 Hz
FIFO data: 0/67108864 words (0% full)
Analog inputs enabled: 72
MISO lines:
A1: no amplifier connected
A2: no amplifier connected (off)
B1: no amplifier connected
B2: no amplifier connected (off)
C1: no amplifier connected (off)
C2: no amplifier connected (off)
D1: no amplifier connected (off)
D2: no amplifier connected (off)
period = 1024 frames (34.133 ms)
FIFO buffering = 0 frames (0 ms)
RHD2K: attaching driver
RHD2K: scratch buffer size=180224 bytes
RHD2K: registering capture port A1_0 (offset = 24)
RHD2K: registering capture port A1_1 (offset = 28)
RHD2K: registering capture port A1_2 (offset = 32)
RHD2K: registering capture port A1_3 (offset = 36)
RHD2K: registering capture port A1_4 (offset = 40)
RHD2K: registering capture port A1_5 (offset = 44)
RHD2K: registering capture port A1_6 (offset = 48)
RHD2K: registering capture port A1_7 (offset = 52)
RHD2K: registering capture port A1_8 (offset = 56)
RHD2K: registering capture port A1_9 (offset = 60)
RHD2K: registering capture port A1_10 (offset = 64)
RHD2K: registering capture port A1_11 (offset = 68)
RHD2K: registering capture port A1_12 (offset = 72)
RHD2K: registering capture port A1_13 (offset = 76)
RHD2K: registering capture port A1_14 (offset = 80)
RHD2K: registering capture port A1_15 (offset = 84)
RHD2K: registering capture port A1_16 (offset = 88)
RHD2K: registering capture port A1_17 (offset = 92)
RHD2K: registering capture port A1_18 (offset = 96)
RHD2K: registering capture port A1_19 (offset = 100)
RHD2K: registering capture port A1_20 (offset = 104)
RHD2K: registering capture port A1_21 (offset = 108)
RHD2K: registering capture port A1_22 (offset = 112)
RHD2K: registering capture port A1_23 (offset = 116)
RHD2K: registering capture port A1_24 (offset = 120)
RHD2K: registering capture port A1_25 (offset = 124)
RHD2K: registering capture port A1_26 (offset = 128)
RHD2K: registering capture port A1_27 (offset = 132)
RHD2K: registering capture port A1_28 (offset = 136)
RHD2K: registering capture port A1_29 (offset = 140)
RHD2K: registering capture port A1_30 (offset = 144)
RHD2K: registering capture port A1_31 (offset = 148)
RHD2K: registering capture port A2_0 (offset = 26)
RHD2K: registering capture port A2_1 (offset = 30)
RHD2K: registering capture port A2_2 (offset = 34)
RHD2K: registering capture port A2_3 (offset = 38)
RHD2K: registering capture port A2_4 (offset = 42)
RHD2K: registering capture port A2_5 (offset = 46)
RHD2K: registering capture port A2_6 (offset = 50)
RHD2K: registering capture port A2_7 (offset = 54)
RHD2K: registering capture port A2_8 (offset = 58)
RHD2K: registering capture port A2_9 (offset = 62)
RHD2K: registering capture port A2_10 (offset = 66)
RHD2K: registering capture port A2_11 (offset = 70)
RHD2K: registering capture port A2_12 (offset = 74)
RHD2K: registering capture port A2_13 (offset = 78)
RHD2K: registering capture port A2_14 (offset = 82)
RHD2K: registering capture port A2_15 (offset = 86)
RHD2K: registering capture port A2_16 (offset = 90)
RHD2K: registering capture port A2_17 (offset = 94)
RHD2K: registering capture port A2_18 (offset = 98)
RHD2K: registering capture port A2_19 (offset = 102)
RHD2K: registering capture port A2_20 (offset = 106)
RHD2K: registering capture port A2_21 (offset = 110)
RHD2K: registering capture port A2_22 (offset = 114)
RHD2K: registering capture port A2_23 (offset = 118)
RHD2K: registering capture port A2_24 (offset = 122)
RHD2K: registering capture port A2_25 (offset = 126)
RHD2K: registering capture port A2_26 (offset = 130)
RHD2K: registering capture port A2_27 (offset = 134)
RHD2K: registering capture port A2_28 (offset = 138)
RHD2K: registering capture port A2_29 (offset = 142)
RHD2K: registering capture port A2_30 (offset = 146)
RHD2K: registering capture port A2_31 (offset = 150)
RHD2K: registering capture port EV_0 (offset = 156)
RHD2K: registering capture port EV_1 (offset = 158)
RHD2K: registering capture port EV_2 (offset = 160)
RHD2K: registering capture port EV_3 (offset = 162)
RHD2K: registering capture port EV_4 (offset = 164)
RHD2K: registering capture port EV_5 (offset = 166)
RHD2K: registering capture port EV_6 (offset = 168)
RHD2K: registering capture port EV_7 (offset = 170)
RHD2K: latency callback: 1024--1024 frames
RHD2K: starting acquisition

This bug came up in analyzing #4. When jstim is run a second time, jrecord reports an event on the trig_in port with the previous stimulus's message and a status of stim_off. For example:

20131221T110444.963646 [jrecord] D: trigger on event: time=208355328
20131221T110444.978528 [jrecord] created entry: /jrecord_0002 (frame=208307328)
20131221T110444.978673 [jrecord] I: writing pretrigger data from 208307328--208355328
20131221T110444.978723 [jrecord] D: prebuf frame: t=208307200, on=128, id=stim, dtype=0
20131221T110444.978913 [jrecord] created dataset: /jrecord_0002/stim
20131221T110444.979764 [jrecord] created dataset: /jrecord_0002/trig_in
20131221T110444.979826 [jrecord] D: event: t=208347460 id=trig_in status=16 message=F9698B0C2CA2A2BDF417E8F4891EE5F0
20131221T110444.980446 [jrecord] D: event: t=208355328 id=trig_in status=0 message=07E250DFF0DEF4C41369A5506E2A1C85

This extra event may or may not get stored in the dataset for that entry. Interestingly, this problem can persist across multiple runs of jrecord, suggesting that the event is hanging around in the jack port buffer.

We should support negative-going clicks as this would allow us to have a latching output (goes high on stim on, then low at stim off) with the proper digital circuitry or microcontroller on the output. It would also be useful to be able to generate biphasic clicks, as this would help our Schmitt triggers reset (sometimes they get latched high).

This would require more flexibility in how the program maps different event types to different kinds of clicks. See #22 and #24.

Running an audio neurophysiology experiment with jill requires several modules to be started and connected properly. It's not reasonable to expect the user to do this on their own, so we need a master control script that will do the following:

1. Make sure the user provided some key attributes - experimenter name, animal id, list of stimuli, stimulus port and sync port.
2. Start jrecord, set output file name based on the current date and time, create a single event port.
3. Start jclicker, connect to sync audio port.
4. If a pre-stim trigger is needed, a second jclicker instance should be started.
5. Start jstim with the arguments provided to the script, wiring event out to jrecord and jclicker and audio out to stim audio port.

The script needs to kill jstim, jrecord, and jclicker when the user hits ctrl-c or or jstim ends.

Assuming that jrecord sessions are started for each new experimental configuration (i.e. recording site), assigning a uuid to each channel would provide a unique identifier for raw data sources.

This would mostly be for the convenience of downstream applications like aplot that try to load the entire dataset into memory. As far as the HDF5 file is concerned, datasets have 64-bit indices so the upper limit on size is quite large.

The stored MIDI messages in jrecord have the wrong status byte and empty or garbage messages. Can't replicate on Snow Leopard with MacPorts JACK1 or on Debian (also JACK1). Possibly related to how JACK OS X integrates with coreaudio?

The click is currently a single sample, which may not be long enough to detect downstream. Letting it be more than one sample long introduces some complexity, though, as the click may span periods.

The bug occurs when using jrecord with recording triggered by jstim or jdetect. If the client first connected to the jrecord trigger port shuts down and a new client is connect, the first recording triggered by the second client ends prematurely. Here are the sequence of commands I used to reproduce the bug:

jackd -d alsa&
jrecord -t -I stim -f test6.arf &
jstim -r 5 song.wav -o jrecord:stim -e jrecord:trig_in
jstim -r 5 song.wav -o jrecord:stim -e jrecord:trig_in

In this case the sixth recording is cut off in the middle of the stimulus playback.

Commenting out "arf_thread->reset()" in the jack_portcon function of
jrecord.cc prevents the bug, as does commenting out "stop_recording(data->time + nframes)" in triggered_data_writer::write in the "if (_recording)" block. Commenting out "_writer->close_entry()" in buffered_data_writer::write doesn't prevent the bug, however (I might have mistakenly said so earlier).

Even after these fixes, the first recording triggered by the second client still doesn't contain the full pre-trigger time (about .7s as opposed to 1s).

Code already in file::stimfile but macroed out. Needs testing.

The advantage of using a string-encoded mapping is that it makes the message a bit more flexible. The disadvantage is that it's bulky and requires parsing.

zmq-related code needs shims so it can compile against 2.2 and 3.x versions of ZMQ library

Is is possible to set dataset level attributes? Some code in our lab requires datatypes to be set to specific values, eg pcm_000 might be audio (datatype 1). It seems like jrecord is the correct level to add this information, otherwise the recorded file has "lost" information readily available to the experimenter. @pmalonis

This error occurs when doing triggered recording, which generates an ARF file with a bunch of entries, one for each time the recording was triggered. Sometimes the new recording gets triggered right after the end of a previous recording, before the pretrigger buffer has had a chance to fill completely. When this happens, the second entry ought to start on the sample right after the first entry's end. It's pretty obvious when these happens when inspecting the recordings, and downstream code may want to splice these entries together for analysis.

However, the jack_frame or timestamp attributes are compared to each other for these cases, the difference is less than the duration of the first entry. What I think is happening is that the attributes are not reflecting the actual amount of data used from the pretrigger buffer. This would also mean that the trigger times are not being correctly offset.

It would be useful to limit recordings to specific times of day. Say you're doing a continuous recording of all the sounds in a nestbox: you probably don't need (or want) data from times when the birds are asleep.

The simplest way to implement this would be a client (jsplit? jtrig?) that triggers recording to start and stop over some interval.

This client could also handle splitting up the interval into smaller chunks, which would address #16.