Test script not working on my machine (maybe not supposed to?) about talos HOT 7 CLOSED

I've never seen the "segmentation fault" before.

Given the description, I wonder if it has to do with the number of permutations? For example if you remove the first test, will it run ok for the second?

I've seen similar issues in the past with looping in general (like looping through Keras models). How about if you restart the vm / machine you are running from?

When it comes to...

Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA

...you are right, this is typical behavior when installing with pip install tensorflow (or conda) This should never cause errors, but just lose performance.

from talos.

Yeah I just tested this on my own fork and Travis has no problem with it. Interesting.

I guess this is just a curiosity at this point. Why does the test script build fine on Travis, while failing on my computer when Talos seems to work seamlessly on its own...?

Edit: I will try your suggestions real quick.

from talos.

This is so bizarre...

Ok so I removed the entire first test and added a helper print out in Scan. In the _run(self) function specifically:

...

round_params(self)
# _model() function should return both the result from training
# and the model itself
try:
     _hr_out, self.keras_model = self._model()
except TypeError:
    print('The model needs to have Return in format '
             ' "return history, model"')

print("Model successfully returned.")  # new helper code code

self.epoch_entropy.append(epoch_entropy((_hr_out)))
_hr_out = run_round_results(self, _hr_out)

...

and end up with this for the second Iris test:

Model successfully returned.
7 scans will take roughly 0 seconds
Model successfully returned.
Model successfully returned.
Model successfully returned.
[1]    19800 segmentation fault  python3 test_script.py

Still seg faults after 4 executions. Weird.

from talos.

Ok that's "good" ... I was guessing that might happen. If you restart your machine, and then create a new virtualenv, do you still get the seg fault?

from talos.

I'll try this soon. Humor me though, I'm still not so comfortable with this stuff just yet. The process I should take is:

1. Create new anaconda VI
2. pip install -r requirements.txt
3. python setup.py install
4. python test_script.py

Can you confirm? Thanks very much for the help! 👍

from talos.

I did exactly that and the script ran with no problems. In summary, my problem not Talos'.

from talos.

@x94carbone Unfortunately TensorFlow comes with issues with resource management, both memory and GPU. Given what it does, I think this is expectable though.

Usually things are resolved with re-installing both Keras and Tensorflow directly from the respective repos, unless it's resolved with a simple vm reset. I think that at some point it's worthwhile to create a "debugging" script for both conda and regular python that reinstalls, verify version, and a bunch of other stuff to make it super easy for the user to troubleshoot upstream issues.

from talos.