update code requirements, fix train.py about example-versioning HOT 17 CLOSED

OK great, please self-assign when you have the time if this is still open. 🙂

from example-versioning.

@casperdcl true. in this case I decided to specify it anyway since it's a simple example and we use virtualenv to completely isolate it Just for the sake of simplicity.

from example-versioning.

@shcheklein said (in iterative/dvc.org#762 (comment)):

I think it's bette to review it and update either using PyTorch or the latest release TensorFlow + Keras.

from example-versioning.

Can do in the next couple of weeks if nobody else does

from example-versioning.

@casperdcl that would be great! Please feel free to assign it to yourself when you start reviewing it.

from example-versioning.

p.s. sorry I was mistaken actually its tensorflow==1.13.1 now (1.11.0 is no longer available with pip) and also test.py is broken with:

Traceback (most recent call last):
  File "train.py", line 123, in <module>
    train_top_model()
  File "train.py", line 118, in train_top_model
    json.dump(history.history, open("metrics.json", 'w'))
...
  File "/.../lib/python3.7/json/encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type float32 is not JSON serializable

from example-versioning.

Still interested given this ^ @casperdcl?

from example-versioning.

yes easy fix just don't have much time over the next week or so

from example-versioning.

Actually this is kind of a high priority I would say don't you think @shcheklein? Since basically our versioning tutorial is broken because of this.

Can someone from @iterative/engineering take a look? I was able to run train.py successfully by accident at some point but I'm not sure which versions of what I was using.

p.s. Preferably someone with a GPU because testing this takes a while on regular computers.

from example-versioning.

ah fine... there are a few other minor issues too. Will PR soon.

from example-versioning.

ok see #5... btw I used tensorflow==2 and it takes less than 10 secs on my computer. We'd have to write tensorflow-gpu==2 in requirements.txt for a GPU version.

from example-versioning.

Thanks @casperdcl didn't mean to turn on the pressure! Just need to get this resolved soon by our team. Your contribution is greatly appreciated! Checking your PR... (will take some time to run on my MacBook air...)

from example-versioning.

@jorgeorpinel @casperdcl I would avoid GPU - it can complicate installation. It should be quick enough when you run it even on CPU since model is pre-trained as far as I remember.

from example-versioning.

yes AFAIK best practice is (seemingly as with dvc in this case) is to leave tensorflow out of requirements.txt completely and write in the docs that people need to install it themselves (so they can choose their own flavour).

from example-versioning.

I would avoid GPU...

@shcheklein I just meant that train.py is not very fast. It's not super slow either but it takes around 10 min to run on my MacBook Air (when/if it works).

from example-versioning.

10 min? On my 12 core (48 with hyperthreading) machine with many background tasks running I still get under 10 secs. My guess is I/O rather than CPU is your main bottleneck?

from example-versioning.

I'm on a 2015 MacBook Air so maybe that explains it. But yeah maybe I/O. Who knows these things haha

from example-versioning.