Comments (7)
This would be a fundamental SciUnit change that I cannot easily make while other libraries depend on it. However, you can always access an element of a list by its attributes, e.g. [test for test in suite.tests if test.some_attribute == some_value]
, and then access the first value of that new list. Or if it has zero (no such test found) or >=2 elements (no such unique test) you could raise an Exception.
There are several candidates for that attribute. If you name the tests in your suite (with the name
keyword on Test
instantiation) then you can just use Test.name
. Or there are other identifiers to choose from, depending on whether you want a unique object in memory (Test._id
) or a hash of its attributes (Test.hash
), or a serialization of it (Test.json
), or some state defined at the level of the test class (Test.state
). You can see many of these in methods here the base class for Models, Tests, etc. Specific implementation changes can be made at the level of the Test
class, especially for something like state
. But name
might be the easiest since you are shuffling and regenerating tests in memory, and there could be attributes lost or added on pickling that those methods are not catching, as they were written with serial applications in mind.
from neuronunit.
Actually, I can make this even easier. I'll just change the __getitem__
of sciunit.TestSuite so that if you try to access a string, e.g. MySuite['smith']
then it will look for the test in that suite with the name
"smith" and return it. Then from the user perspective it is as if the TestSuite
was just a dictionary. Then the only user obligation is to name the tests, which is no different than giving them keys if you are using an actual dictionary. I'll report here when I've finished this.
from neuronunit.
OK, that was easy. In sciunit dev branch (implemented in scidash/sciunit@deb589d), you can now do this:
from sciunit import Test, TestSuite
empty_observation = {}
test1 = Test(empty_observation, name='smith')
test2 = Test(empty_observation, name='jones')
suite = TestSuite([test1, test2])
x = suite['smith']
x == test1, x == test2
Which will print True, False
since x
refers to test1
(with name 'smith') and not to test2
. So this is basically equivalent to the TestSuite
being a dictionary, as long as the keys you want to use are set as the test names.
from neuronunit.
Okay, can we keep this issue open until I establish if this syntax works?
You proposed that optimize should be a method of a sciunit test suite. I am trying to achieve this through inheritance.
Now that you have changed suites internal code, maybe method binding would be a better solution.
When TSD inherited from dictionary self.update(tests) would appropriately package tests as key item pairs.
from sciunit import TestSuite
class TSD(TestSuite):
def __init__(self,tests={},use_rheobase_score=False):
self.DO = None
super(TSD,self).__init__(tests)
self.update(tests)
if 'name' in self.keys():
self.cell_name = tests['name']
self.pop('name',None)
else:
self.cell_name = 'simulated data'
self.use_rheobase_score = use_rheobase_score
self.elaborate_plots = elaborate_plots
self.backend = None
def optimize(self,param_edges,backend=None,protocol={'allen': False, 'elephant': True},\
from neuronunit.
Sure, I've reopened it. But based on the changes I made above in scidash/sciunit@deb589d, you won't need to call self.update(tests)
, because you automatically now get key-based access to tests in anything that inherits from TestSuite
. And you would access names like you do in a TestSuite
itself, by the name
attribute (which means the signature of the constructor should probably match that of TestSuite
up until the additional kwargs like use_rheobase_score
. So it would be something like:
class TSD(TestSuite):
def __init__(self, *args, **kwargs, use_rheobase_score=False, elaborate_plots=False):
super(TSD, self).__init__(*args, **kwargs)
self.DO = None
self.use_rheobase_score = use_rheobase_score
self.elaborate_plots = elaborate_plots
def optimize(self, param_edges, protocol={'allen': False, 'elephant': True}):
...
where I took out backend
because I think that should be an attribute of the model being tested, not the test or test suite.
from neuronunit.
Hey Rick, to properly fit my use case, I may need a bit more developmental support.
My use case is cell 11.
I may need you to help design a derived class TSS. That inherits from TestSuite, but has an optimize method.
You can see my stumbling attempt to do that here.
What I want to do is take a collection of sciunit tests, that are already dictionaries and to construct a test suite out of it (something that behaves like a dictionary). I think the constructor should be able to accept both lists and dictionaries. It should sense if it receives a dictionary type and just converts dict to list if appropriate.
I think what exists now, is a suite, that has key,value behavior (like a dictionary), but its constructed only from lists.
The other thing that would help is if upon construction if I could pass in the user-defined optimization method to the constructor as a constructor method?
That would help to circumvent messy post hoc method binding, that you can see in the notebooks here.
tests= test_frame['Neocortex pyramidal cell layer 5-6']
tests['name'] = 'Neocortex pyramidal cell layer 5-6'
tt = list(tests.values())[0:-1]
tt = TestSuite(tt)
TD = TSD(tests)
tt.optimize = MethodType(TD.optimize,tt)
return tt
def optimize_selection(backend,cell_id= None):
#code;
NGEN = 8
MU = 4
"""
tests= self.test_frame['Hippocampus CA1 pyramidal cell']
tests['name'] = 'Hippocampus CA1 pyramidal cell'
a = TestSuite(tests)
a.optimize = MethodType(optimize,a)
tests = TSD(tests = tests,use_rheobase_score=True)
"""
tt = process_tests()backend,protocol={'allen': False, 'elephant': True
param_edges = model_parameters.MODEL_PARAMS[backend]
out = tt.optimize(param_edges,MU=5,NGEN=5,free_params=param_edges.keys())
return out
#return optimized_models, passive_
from neuronunit.
Addressed in russelljjarvis/neuronunit_opt@3ee0f2db709252dc02e34c8518a74f845238e90e and subsequent commits.
from neuronunit.
Related Issues (20)
- newer test class for passive elephant tests breaks parallelism in dask, in the case of brian2 backend. HOT 4
- Documenting all Score edge cases that break optimization gradients HOT 3
- A document Notice to debug set PARALLEL_CONFIDENT=False
- Use standardized units throughout
- Sciunit judge no longer works with static Models. How can I opt of using the ProtocolToFeaturesTest? HOT 8
- Druckmann Tests are not proper tests
- Confirm that NeuronUnits jNeuromLB backend really can be regarded as a ground truth. HOT 7
- NU test.protocol should be flat or more predictable to traverse.
- Major NU interface redesigns could come with interface diagrams.
- documentation chapters of NeuronUnit should be upgraded to Unit Tests via pytest-ipynb
- 2.5 different code locations that determine sim_length need to be collapsed into one.
- Duplicated content injected current test protocol/parameter
- Installation Fails on Windows HOT 1
- create branch optimization based on dev, that I can merge pull requests into. HOT 2
- Migrate away from travis HOT 2
- Merge error in backend __init__.py HOT 2
- Missing closing ' HOT 1
- Relax constraints on install requirements HOT 1
- unresolved merge conflict in requirements.txt
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from neuronunit.