When we complete Helveg/cerebellum-scaffold#369 we rename the old nest to nest2 and the class to Nest2Adapter

For some connections, divergence values are not used for connectivity (e.g. for IO to PC connections), but it should be checked and a warning should be thrown.

• For IO to PC values see #104.

• For PC-DCN: "Because 30–50 Purkinje cells converge onto each CbN cell (Person & Raman, 2012a), the number of inhibitory inputs was held constant at 40 in all experiments." [https://physoc.onlinelibrary.wiley.com/doi/pdf/10.1113/JP274321]

Currently the morphology_repository attribute is problematic: It looks for the specified file relative to the cwd and not to where the configuration was found or some other sane behavior, so when you compile from another folder it won't find the morphology repository anymore

as_cache should return an empty MorphologyCache based on the MR. MorphologyCaches should produce singleton morphologies that are immutable by default.

The report module requires a slight refactoring so that report listeners can be added. Reporting to screen should then be added as a default report listener. This should improve the design for a variety of future logging functions (better than the messy report file which is mostly obsolete when #495 is implemented)

In order to print output during an MPI process stdout needs to operate in unbuffered mode. The reporting module can overwrite stdout if MPI is present.

Describe the work done

• The setup now produces a wheel named bsb
• The console_script entry point is renamed to bsb (so the command is bsb compile instead of scaffold compile)
• The package source directory has been renamed to bsb
• Imports should import the bsb module instead of the scaffold module.
• Removed the obfuscation module. Since the package will go public we don't need to encrypt our public releases anymore.

Migration

Users (which we don't have) need to uninstall the dbbs-scaffold package and pip install bsb instead.

Developers need to run pip install -e . from the git repo root to register the new bsb command and change all from scaffold/ import scaffold statements to from bsb/import bsb

List which issues this resolves:

Closes #482

Tasks

• Added tests
• Updated documentation

For some connections, divergence values are not used for connectivity (e.g. for IO to PC connections), but it should be checked and a warning should be thrown.

• For IO to PC values see Helveg/cerebellum-scaffold#104.

• For PC-DCN: "Because 30–50 Purkinje cells converge onto each CbN cell (Person & Raman, 2012a), the number of inhibitory inputs was held constant at 40 in all experiments." [https://physoc.onlinelibrary.wiley.com/doi/pdf/10.1113/JP274321]

Fixes dbbs-lab/models-old#25

Describe the work done

More plotting functions now return their produced figs, and fixed some other small bugs. PSTHs can now be stacked.

Usecase:

The user wants to compile an hdf5 exploiting the morphologies, but it does not want to simulate it in NEURON and does not have NEURON installed.

Actual behaviour:

The user has to remove from the json the"NEURON" simulation part.

Desired behaviour:

The compilation should in principle work even if NEURON is not installed.

Currently the global CLI arguments are position specific and there's no unified way access global scaffold settings. It would be better that there's an options module where the values of singleton settings can be read from and stored in from 3 sources:

• environment variables
• cli arguments
• set by script bsb.options.verbosity = 2?

class Options(types.ModuleType):
  verbosity = IntegerOption(cli=("verbosity", "v"), env="BSB_VERBOSITY", default=1)
  log = StringOption(cli=("log", "l"), env="BSB_LOG")

sys.modules[__name__] = Options()

They're a mess, should become similar to the PlacementSet

Tests are being skipped to allow #494 to merge. We should:

A) Check if we can do weekly builds and tests of full scale networks that we can then cache on Travis, or upload as build artifacts to possibly Amazon AWS or our own cluster?

B) Re-enable push and branch testing by using the old mouse_cerebellum.json as basis for these smaller per-commit tests

@AliceGem is this a correct explanation of num_neurons? (git blame says you added it 😋) Could you also look into #471 for this PR?

SC and BC (not labelled) should be connected to PC (labelled) implicitly respecting the microzone division.
In the current neuron.json scaffold the segregation is ok, but we should check that this is consistent with morphologies orientations:

They're a mess, should become similar to the PlacementSet

Right now, hours of radio-silence pass when connecting GrCs to PCs. It would be useful to track the progress %.

Tests are being skipped to allow #494 to merge. We should:

A) Check if we can do weekly builds and tests of full scale networks that we can then cache on Travis, or upload as build artifacts to possibly Amazon AWS or our own cluster?

B) Re-enable push and branch testing by using the old mouse_cerebellum.json as basis for these smaller per-commit tests

In order to print output during an MPI process stdout needs to operate in unbuffered mode. The reporting module can overwrite stdout if MPI is present.

Fixes dbbs-lab/models-old#25

as_cache should return an empty MorphologyCache based on the MR. MorphologyCaches should produce singleton morphologies that are immutable by default.

And wrap them in more information such as cell model, current context, ...

We should check that NEST-specific variables are properly set when running NEST simulations with the scaffold (e.g. local_num_threads or RNG variables).

Not optimized number of possible searches..
probably n^(2)*log(n) searches , useless..

File "/home/alberto/workspace/cerebellum-scaffold/scaffold/postprocessing.py", line 263, in after_connectivity
    compartment_matrix = self.scaffold.connection_compartments[tag]
KeyError: 'golgi_to_glomerulus'

Even if using "detailed" for golgi_to_glomerulus:

"golgi_to_glomerulus": {
      "class": "scaffold.connectivity.ConnectomeGolgiGlomerulus",
      "from_cell_types": [{"type": "golgi_cell", "compartments": ["axon"]}],
      "to_cell_types": [{"type": "glomerulus", "compartments": ["soma"]}],
      "divergence": 40,
      "detailed": true
    },

In VoxelIntersection it's contacts and in TouchDetection it's synapses

Time to split foolks.

File "/home/alberto/workspace/cerebellum-scaffold/scaffold/postprocessing.py", line 263, in after_connectivity
    compartment_matrix = self.scaffold.connection_compartments[tag]
KeyError: 'golgi_to_glomerulus'

Even if using "detailed" for golgi_to_glomerulus:

"golgi_to_glomerulus": {
      "class": "scaffold.connectivity.ConnectomeGolgiGlomerulus",
      "from_cell_types": [{"type": "golgi_cell", "compartments": ["axon"]}],
      "to_cell_types": [{"type": "glomerulus", "compartments": ["soma"]}],
      "divergence": 40,
      "detailed": true
    },

Right now, hours of radio-silence pass when connecting GrCs to PCs. It would be useful to track the progress %.

Previous (less) or future (more)?

When we #369 we rename the old nest to nest2 and the class to Nest2Adapter

Not optimized number of possible searches..
probably n^(2)*log(n) searches , useless..

Since point neuron models ignore any compartments given, there are many duplicate entries between the same neuron pairs in connectivity sets created with detailed connectivity strategies. To optimize, a reduced dataset with unique pairs and a parallel dataset with the amount of times they occur could be provided.

Impact on ConnectivitySet

The ConnectivitySets get_intersection method should be adapted to incorporate a get_intersection_iterator.

Impact on NEST adapter

Multi-contact connections are created only once. Unitary weights are tuned to get the right population firing rates, and then multiplied by the synaptic contact multiplicity to get the single connection weight (population weight will be an array)

Impact on NEURON adapter

None, the get_intersections function should produce the same result. To optimize, we should switch to the get_intersection_iterator to reduce memory footprint.

Time to split foolks.

Usecase:

The user wants to compile an hdf5 exploiting the morphologies, but it does not want to simulate it in NEURON and does not have NEURON installed.

Actual behaviour:

The user has to remove from the json the"NEURON" simulation part.

Desired behaviour:

The compilation should in principle work even if NEURON is not installed.

We should check that NEST-specific variables are properly set when running NEST simulations with the scaffold (e.g. local_num_threads or RNG variables).

Previous (less) or future (more)?

Transform the Configuration and its current state back into a configuration tree (ignore breakpoints and multifiles for now)

Transform the Configuration and its current state back into a configuration tree (ignore breakpoints and multifiles for now)

@AliceGem is this a correct explanation of num_neurons? (git blame says you added it 😋) Could you also look into #471 for this PR?

Currently the morphology_repository attribute is problematic: It looks for the specified file relative to the cwd and not to where the configuration was found or some other sane behavior, so when you compile from another folder it won't find the morphology repository anymore

Since point neuron models ignore any compartments given, there are many duplicate entries between the same neuron pairs in connectivity sets created with detailed connectivity strategies. To optimize, a reduced dataset with unique pairs and a parallel dataset with the amount of times they occur could be provided.

Impact on ConnectivitySet

The ConnectivitySets get_intersection method should be adapted to incorporate a get_intersection_iterator.

Impact on NEST adapter

Multi-contact connections are created only once. Unitary weights are tuned to get the right population firing rates, and then multiplied by the synaptic contact multiplicity to get the single connection weight (population weight will be an array)

Impact on NEURON adapter

None, the get_intersections function should produce the same result. To optimize, we should switch to the get_intersection_iterator to reduce memory footprint.

Describe the work done

List which issues this resolves:

(e.g. "closes #1, #2, #3")

Tasks

• Added tests
• Updated documentation