alleninstitute / mies Goto Github PK

MCC app has Zap/Pulse/Buzz functions, but these are not exposed by the MCC XOP (DLL has them) and thus are not usable from within MIES

Add zap button(s) to clamp mode tabs.

Cross links

#22
#179 (TODO list)
AllenInstitute/IPNWB#5
AllenInstitute/IPNWB#16

Support matrix (read/write):

(IP7 is not supported anymore)

Tests:

• Copy UTF_NwbExportV1.ipf, adapt for V2 and test both exports in all current cases
• Add validation tests using pynwb for V2 files as well
• Update UTF_NwbExportV2.ipf with changes to UTF_NwbExportV1.ipf since last sync

We currently don't use the ADC for pressure control. The task is to record the applied user pressure.

We probably need to set the limit a bit higher than 100mus as we want to have a valid wave even after decimation. Maximum decimation is a factor of 160 (16 from sampling multiplier and 10 from DAQ).

• Recompile ITC XOP for IP8
• Recompile MCC XOP for IP8
• Recompile ZeroMQ XOP for IP8
• Add support for multiple return value syntax to ZeroMQ XOP (AllenInstitute/ZeroMQ-XOP#5)
• Add multiple return value syntax and new input parameter syntax to doxygen conversion script (byte-physics/doxygen-filter-ipf#1)

For numerical entries the settings history graph updates automatically on newly acquired sweeps. But this is not the case for textual entries as these are added/hacked into the graph as tags.

This is usually a bug in the setup or the stimset.

This situation can be created via different onset delays and/or different stimsets.

The basic idea is that the user can define epochs (start time and end time) in the data waves for which they can then pick an analysis type(ex. simple type: max, min etc. Complex:..spike detection, epsp detection). These analysis would be applied to all the data sweeps, or a subset of sweeps as defined by the user. The results would be stored in analysis results waves (in a analysis folder) and could be plotted in a graph in the data browser.

• Allow to put GUI control change events into a queue when a user changes something during DAQ
• On the next sweep start we process all these changes, call the respective GUI control procedures and continue
• The GUI state wave changes only after the events from the queue have been processed.

For this we need to convert all GUI control users to always use the GUI state wave.

• Add a function to pressure regulation that monitors changes in the steady state resistance to automatically switch from Approach to Seal
• Need to get data from Tim to asses when the switch is happening

That breaks the indexing logic as we rely on menuExp being in sync with the popup menu entries.

• Must support RA/indexing/etc
• Fix currently broken things like outerset randomization
• Add support for innerset randomization
• Visualize data which will be sent to the hardware. Use case: Future sets not yet acquired
• would enable per headstage sweep skipping
• would reduce compute between sweeps

Related to #33.

This first needs to determine the wished-for tab order.

• Needs history rewriting to remove offending parts.
• Need to connect bamboo with this repository
  • add machine user to the MIES repo and generate an oauth token for it
  • PR testing: https://blog.buildo.io/github-bamboo-ci-6f681b00d310 and https://gitlab.com/gitlab-org/gitlab-ee/issues/3836#note_56949346
  • uploading of new releases: https://blogs.infosupport.com/accessing-githubs-rest-api-with-curl/ and https://gist.github.com/stefanbuck/ce788fee19ab6eb0b4447a85fc99f447
  • upload of latest master version
  • upload documentation to github for latest release
  • Try out clean/smudge filter to compress pxps, see https://git-scm.com/docs/git-config and https://git-scm.com/docs/gitattributes and https://stackoverflow.com/a/34780618
  • Update and re-enable bamboo jobs for uploading the installer and the release package.

For testing without hardware it would be nice to have a simulation mode. Decide if it should be hardware independent or not.

• move into own file
• Streamline interface to be more consistent
• Reduce number of functions
• add a way to get the numericalValues from textualValues and vice versa
• add entrySourceType as input parameter to GetLastSweepWithSetting and friends

Will not be API compatible.

Functions like GetLastSettingEachSCI should also allow to only return the last valid entry of a particular headstage.

The logic behind MSQ_GetLBNEntryForHeadstageSCI should then also be included in that rewrite.

Scope: Something like: right click on a plot with mouse movement zooms in/out depending on the direction of mouse movement

Where type is steady state resistance or instantaneous resistance. Later versions should also support: baseline average, capacitance and baseline noise quantification.

The function will evaluate all epochs (stock and user) with a shortname matching ^(U_)?TP[[:digit:]]*$, the baselines and pulse is expected to match ^(U_)?TP[[:digit:]]*_(P|B0|B1)$. If multiple testpulses are present the raw data is averaged before continuing.

The code in CalculateTPLikePropsFromSweep already does part of the evaluation.

Output: one value per headstage (a sweep can have multiple headstages active) -> 3D wave like data with dimensions
ROW: value
COL: sweeps
LAYER: channels

DAQ: Allow 100,50,25,10 kHz for all channels
TP: As fast as possible
Will be in addition to the sampling interval multiplier, but not combined.

It would be nice if MIES ensured that the ITI was respected between stimsets when manually initialized. MIES could use the start time of the last sweep recorded in the lab notebook. It wouldn't need to be super precise, the interval would just need to be longer than the specified ITI of the previous stim set.

We would need to lookup the ITI of the last sweep from the labnotebook add that onto the high precision timestamp of the last sweep and wait with sweep starting until the ITI is over. And of course introduce a setting for that as we don't want to have it always.

Prevent DAQ if ITI is not yet reached and tell the user the leftover time.

• Options for tuning axis scaling: One for continuous axes updates (default igor option),
  one for a fixed scaling (no updates) or updating as now.
  Having these modes are helpful for trouble shooting electrical noise on the rig.
• TP update frequency set by the user

It should be possible to select all sweeps from the same repeated acquisition cycle/stimset cycle in the overlays sweeps panel.

It depends on the order of how the these operations are applied if it works or not.

• Add documentation for each entry in the LBN key waves
  • Numerical
  • Textual
• Document if it is headstage dependent or independent data.
• Implement for #228 the settings history panel

The new mode is called "TP during DAQ" for the headstages which have that activated. Similiar to the name "TP during ITI" (inter trial interval).

• A special stimset named "Testpulse" can be selected
• This stimset will be generated on the fly from the testpulse settings using the existing TP MD code for ITC hardware
• DAQ itself is done as usual
• Display is different for channels with "TestPulse" stimset. On these channels we have a display like TP.
  Think about adding two different groups in the oscilloscope window. One for TP data with its own x-axis and one for DAQ data.
• TP_Delta will be called on every one of these pulses from the "Testpulse" stimsets. LBN entries only for the last one as we do for TP itself.
• PressureControl should still work with "TP during DAQ", (calling P_PressureControl should be enough)
• Ignore "TP during DAQ" headstages for dDAQ/oodDAQ. (dDAQ = "distributed DAQ", oodDAQ = "optimized overlapped distributed DAQ")

One option would be to use a website with a form whichs sends an email.
And then call that website using HTTP PUT.

A user should be able to navigate the controls with TAB and SHIFT-TAB in a natural order from entry to delta, delta dme, etc.

See also , see http://www.igorexchange.com/node/8380#comment-16845.

• Add option to use average for Y-axes scaling
• Add option to use scale bars instead of axes
  • if Equal Y, use only 1 vertical scale bar
  • use only one horizontal scale bar if bottom axes is the same for all plots
  • non overlapping placement of scale bars
• support two formatting options: presentation and publication
• add a pulse range in addition to a pulse count option
• add option to zero exported traces
• extend to support SF plot (12/5/2022)

• Add popupmenu for categories of lbn entries (add "all" category as well)
• Merge numerical and textual entries,
• add search setvariable and listbox for lbn entries. Use checkboxes to visualize entry.

Autobias and Send to all Amp does not play nice together. E.g. the holding potential of one headstage is applied to the other headstages as well, but it should actually be calculated by the autobias function.

• sphinx based getting started manual
• introductory screencast videos

• fallback length will become override length with checkbox
• Instead of a user specified length/duration, instead have a max length. This will avoid the situation where second pulses would be plotted when the user input is longer than the interpulse interval.
• Y-scale to max option on all channels without spikes (non diagonal elements).
• time align spikes and corresponding responses
• Use the pulse length of each pulse and not the average one for mixed frequency

• Properly implement the generic threadsafe framework prototyped in branch postponed/feature/threadsafe-NWB-writing with documentation and tests
• Finalize documentation what we are calculating here, CalculateTPLikePropsFromSweep does something very similiar to TP_Delta. For testing if it works we want both the old and new code results be outputted/comared.
• Rewrite, remove the activeAD columns approach, make it threadsafe so that it can run in a preemptive thread (so it must not access any DFREFs)
• Execute TP_Delta with that threadsafe framework in a separate thread

• Clarify scope and explain better

This also tests the installer and is therefore better.

• Add script install-from-installer.sh
• Add installer CLI option to select "install without hardware"
• Add option to autorun-test.sh to choose the igor version
• Add option to install-from-installer.sh to "install without hardware"
• reorganize scripts
  • remove bit-rotted batch files
  • unify compilation-testing and unit-testing to testing
• Adapt bamboo tests
  Tests to perform:
  • Compilation testing with full repository (mint box)
  • Build installer (with userMode flag) and add installer and release package as artefact, reuse artefacts on all later stages (mint box)
  • Compilation testing with release package (mint box)
  • Build documentation, add documentation as artefact (mint box)
  • Generic tests, use installation package without hardware XOPs (Windows)
    • IP7/8 32/64bit
  • Testing with hardware, use installation package with hardware XOPs (Windows)
    • IP7 64bit (NI)
    • IP8 64bit (ITC)

• Don't ignore the time DC_ConfigureDataForITC takes in RA:
  A totally crazy idea: Why do we not call DC_ConfigureDataForITC(panelTitle,
  DATA_ACQUISITION_MODE) before we start waiting for the ITT elapse? This would
  not require any guessing about the config time from our part we just would have
  to keep separate ITCDataWaves and ITCConfigWaves for DAQ and TP. As we need to
  configure DAQ before TP.
  • GetITCDataWave and friends would get a dataAcqOrTP parameter like
    DC_UpdateGlobals and create the wave either in
    MIES:ITCDevices:ITC18USB_DEV or MIES:ITCDevices:ITC18USB_DEV:TestPulse
  • Most of the code in RA_CounterMD and RA_Counter has to be moved and done
    in RA_HandleITI before we can fill the ITCDataWave for DAQ and before we
    can start TP.
  • In ITC_StartDAQMultiDeviceLowLevel we need to pull out the DC_ConfigureDataForITC calls.
    See also the discussion in PR 459 (AI stash)