tensorflowcv pads conv2d different than kerascv, tf2cv & gluoncv2.

On the keras/tf2 implementation of conv2d, the padding is as follows:

padding: Int, or tuple of 2 ints, or tuple of 2 tuples of 2 ints.
If int: the same symmetric padding is applied to height and width.
If tuple of 2 ints: interpreted as two different symmetric padding values for height and width: (symmetric_height_pad, symmetric_width_pad).
If tuple of 2 tuples of 2 ints: interpreted as ((top_pad, bottom_pad), (left_pad, right_pad))

(source)
When a tuple of 2 ints is passed, this is consistent with mxnet version.

Currently conv2d (for tf1) uses the following padding:
paddings_tf = [[0, 0], list(padding), list(padding), [0, 0]]
where padding is a pair of ints.
This is only consistent when padding[0] == padding[1].
Most models keep H == W throughout, so this is okay.
But in networks such as squeezenext, this causes the tf1 version to behave differently.

First off, thank you for compiling all these feature extractors implemented in different frameworks into a single repo. It's super useful while testing the performance of a new detection model.
I came here looking for a PyTorch implementation of SqueezeNext - following the link on here.
I can't find anything on your lists though, not a single implementation of SqueezeNext. I found two repos being maintained independently(Tf and PyTorch) and an old issue on your page.
I was wondering why you took it down, is there any particular reason why you've chosen not to include SqueezeNext?

Is there any plan to make models configurable for batch norm and group norm?

Hello, I've tried to export efficientnet_b0b. However, it failed.

The following is my code segment:

net = glcv2_get_model("efficientnet_b0b", pretrained=True)
net.hybridize()

x = np.zeros([1,3,224,244])
x = mx.nd.array(x)

net.forward(x)
net.export('efficientnet_b0b')

And the error message is as following:

File "/usr/local/lib/python2.7/dist-packages/mxnet/gluon/block.py", line 915, in forward
return self._call_cached_op(x, *args)
File "/usr/local/lib/python2.7/dist-packages/mxnet/gluon/block.py", line 805, in _call_cached_op
self._build_cache(*args)
File "/usr/local/lib/python2.7/dist-packages/mxnet/gluon/block.py", line 757, in _build_cache
data, out = self._get_graph(*args)
File "/usr/local/lib/python2.7/dist-packages/mxnet/gluon/block.py", line 749, in _get_graph
out = self.hybrid_forward(symbol, *grouped_inputs, **params) # pylint: disable=no-value-for-parameter
File "/usr/local/lib/python2.7/dist-packages/gluoncv2/models/efficientnet.py", line 393, in hybrid_forward
x = self.features(x)
File "/usr/local/lib/python2.7/dist-packages/mxnet/gluon/block.py", line 548, in call
out = self.forward(*args)
File "/usr/local/lib/python2.7/dist-packages/mxnet/gluon/block.py", line 932, in forward
return self.hybrid_forward(symbol, x, *args, **params)
File "/usr/local/lib/python2.7/dist-packages/mxnet/gluon/nn/basic_layers.py", line 117, in hybrid_forward
x = block(x)
File "/usr/local/lib/python2.7/dist-packages/mxnet/gluon/block.py", line 548, in call
out = self.forward(*args)
File "/usr/local/lib/python2.7/dist-packages/mxnet/gluon/block.py", line 932, in forward
return self.hybrid_forward(symbol, x, *args, **params)
File "/usr/local/lib/python2.7/dist-packages/gluoncv2/models/efficientnet.py", line 274, in hybrid_forward
x = F.pad(x, mode="constant", pad_width=calc_tf_padding(x, kernel_size=3, strides=2), constant_value=0)
File "/usr/local/lib/python2.7/dist-packages/gluoncv2/models/efficientnet.py", line 43, in calc_tf_padding
height, width = x.shape[2:]
AttributeError: 'Symbol' object has no attribute 'shape'

Do you know how to modify it? Thanks

The PResNet18 model seems incompatible with PyTorch 0.4.0.

Traceback (most recent call last):
  File "convert.py", line 135, in <module>
    net = ptcv_get_model("PreResNet18", pretrained=True)
  File "/home/nero/Files/Workspace/env/lib/python3.6/site-packages/pytorchcv/model_provider.py", line 190, in get_model
    net = _models[name](**kwargs)
  File "/home/nero/Files/Workspace/env/lib/python3.6/site-packages/pytorchcv/models/preresnet.py", line 582, in preresnet18
    return get_preresnet(blocks=18, model_name="preresnet18", **kwargs)
  File "/home/nero/Files/Workspace/env/lib/python3.6/site-packages/pytorchcv/models/preresnet.py", line 461, in get_preresnet
    local_model_store_dir_path=root)))
  File "/home/nero/Files/Workspace/env/lib/python3.6/site-packages/torch/nn/modules/module.py", line 721, in load_state_dict
    self.__class__.__name__, "\n\t".join(error_msgs)))
RuntimeError: Error(s) in loading state_dict for PreResNet:
	Unexpected key(s) in state_dict: "features.init_block.bn.num_batches_tracked", "features.stage1.unit1.body.conv1.bn.num_batches_tracked", "features.stage1.unit1.body.conv2.bn.num_batches_tracked", "features.stage1.unit2.body.conv1.bn.num_batches_tracked", "features.stage1.unit2.body.conv2.bn.num_batches_tracked", "features.stage2.unit1.body.conv1.bn.num_batches_tracked", "features.stage2.unit1.body.conv2.bn.num_batches_tracked", "features.stage2.unit2.body.conv1.bn.num_batches_tracked", "features.stage2.unit2.body.conv2.bn.num_batches_tracked", "features.stage3.unit1.body.conv1.bn.num_batches_tracked", "features.stage3.unit1.body.conv2.bn.num_batches_tracked", "features.stage3.unit2.body.conv1.bn.num_batches_tracked", "features.stage3.unit2.body.conv2.bn.num_batches_tracked", "features.stage4.unit1.body.conv1.bn.num_batches_tracked", "features.stage4.unit1.body.conv2.bn.num_batches_tracked", "features.stage4.unit2.body.conv1.bn.num_batches_tracked", "features.stage4.unit2.body.conv2.bn.num_batches_tracked", "features.post_activ.bn.num_batches_tracked"

Hello! I download the resnet-18 model weights(resnet18-0952-0817d058.h5) but it doesn't contain the model structure. So I download the common.py and the resnet.py. The code is shown as follows:
my enviornment: python 3.6, tensorflow version: 1.10.0, keras: 2.2.5

`from keras.models import load_model
path = "C://Users//ME//.keras//models//resnet18-0952-0817d058.h5"

model = resnet18()
model.load_weights(path)
x = np.ones((1,224, 224,3), np.float32)
y = model.predict(x)`

The problem occurs as the "ValueError: Dimension 2 in both shapes must be equal, but are 3 and 64. Shapes are [7,7,3,64] and [7,7,64,3]. for 'Assign' (op: 'Assign') with input shapes: [7,7,3,64], [7,7,64,3]."

Hello, I tried to run inception-v4 on our local machine, found that the output size of average pool after the fully connected layer is not correct, I checked the source code, found that there lost a parameter padding=1 in nn.AvgPool2D() function. Is it right?

Hi
I've check the original paper and this two repos (caffe, pytorch) of mobilenet-v2.
I found that the mobilenet-v2 implementation in pytorchcv seems to have an additional conv-bn-relu block at the top of model comparing to normal mobilenet-v2.
Was that intended to be?

thank you for your nice work. i want to use your classifiers for my thesis. but I need convolutional outputs. but as I try there isn't any way to get this layers output in tf2cv because you have used sequential model. do you know another way to use it?

I am finding that the names of the models are difficult to match to the reported rows in the HTML table

For example, on the HTML tables we have

ResNet-18 x0.25 | 39.62 | 17.85 | 3,937,400 | 270.94M | Converted from GL model (log)

But in the python code we have

resnet18_wd4

Do you have a lookup table or some set of regular expressions that would allow us to match the model names to the HTML tables ?

We have been able to match 370 of the 444 models but still a lot of work to do

Hi, I noticed that weights for EfficientNet B4 and B5 have been released now, tensorflow/tpu#404 (comment)

Do you have plans on converting those weights and making available for your keras implementation of EfficientNet B4 and B5?

In any case, thanks for this amazing resource

The model is in the list, but not found in the pip package:

  File "/home/nero/Files/Workspace/env/lib/python3.6/site-packages/gluoncv2/model_provider.py", line 199, in get_model
    raise ValueError('Unsupported model: {}'.format(name))
ValueError: Unsupported model: inceptionv4

What should be the input image size for xception model?

hi, @osmr, is this project a company project or personal project?
I sent a mail to you yesterday, we are very interested at this project.

Hi, thanks for the great repo.
For SqueezeNet, the input image size may be 227x227 instead of 224x224, otherwise it will not match the output size.

Xue Wen
Shenzhen, China

Currently models obtained by calling tf2cv.model_provider can't be saved by tensorflow.
This makes it difficult to use in another context - feature extractor, served by an application, converted via tflite, etc..

For example:
net = tfcv_get_model(model_name, pretrained=True, data_format="channels_last") net.save(file_path)
results in:
NotImplementedError: Saving the model to HDF5 format requires the model to be a Functional model or a Sequential model. It does not work for subclassed models, because such models are defined via the body of a Python method, which isn't safely serializable. Consider saving to the Tensorflow SavedModel format (by setting save_format="tf") or using 'save_weights'.

I figured I can bypass this restriction by directly accessing the underlying keras model:
net.features.save(file_path)
But this results in
NotImplementedError: Layers with arguments in '__init__' must override 'get_config'

Will it be possible to support this?

Hello, is there any download address for pth.zip, one download is too slow?

The imagenet-pretrained model has 1000 classes, but when I only want to replace the last dense layer, I got the error. It looks like classes=1000 cannot be changed. I would suggest add some argument include_top=False just like in tf.keras.applications.ResNet50, which we can customize the last dense layer.

net = kecv_get_model("resnet50", pretrained=True, classes=100)

AssertionError                            Traceback (most recent call last)
<ipython-input-26-c5cb45b6fa44> in <module>
----> 1 net = kecv_get_model("resnet50", pretrained=True, classes=100)

~/miniconda3/envs/tf114/lib/python3.6/site-packages/kerascv/model_provider.py in get_model(name, **kwargs)
    246     if name not in _models:
    247         raise ValueError("Unsupported model: {}".format(name))
--> 248     net = _models[name](**kwargs)
    249     return net

~/miniconda3/envs/tf114/lib/python3.6/site-packages/kerascv/models/resnet.py in resnet50(**kwargs)
    585         Location for keeping the model parameters.
    586     """
--> 587     return get_resnet(blocks=50, model_name="resnet50", **kwargs)
    588 
    589 

~/miniconda3/envs/tf114/lib/python3.6/site-packages/kerascv/models/resnet.py in get_resnet(blocks, bottleneck, conv1_stride, width_scale, model_name, pretrained, root, **kwargs)
    376             net=net,
    377             model_name=model_name,
--> 378             local_model_store_dir_path=root)
    379 
    380     return net

~/miniconda3/envs/tf114/lib/python3.6/site-packages/kerascv/models/model_store.py in download_model(net, model_name, local_model_store_dir_path)
    511         file_path=get_model_file(
    512             model_name=model_name,
--> 513             local_model_store_dir_path=local_model_store_dir_path))

~/miniconda3/envs/tf114/lib/python3.6/site-packages/kerascv/models/model_store.py in load_model(net, file_path, skip_mismatch)
    489             _load_weights_from_hdf5_group(
    490                 f=f,
--> 491                 layers=net.layers)
    492 
    493 

~/miniconda3/envs/tf114/lib/python3.6/site-packages/kerascv/models/model_store.py in _load_weights_from_hdf5_group(f, layers)
    391         weight_values = _preprocess_weights_for_loading(
    392             layer=layer,
--> 393             weights=weight_values)
    394         if len(weight_values) != len(symbolic_weights):
    395             raise ValueError('Layer #' + str(k) +

~/miniconda3/envs/tf114/lib/python3.6/site-packages/kerascv/models/model_store.py in _preprocess_weights_for_loading(layer, weights)
    346             weights[0] = np.transpose(weights[0], (2, 3, 0, 1))
    347     for i in range(len(weights)):
--> 348         assert (K.int_shape(layer.weights[i]) == weights[i].shape)
    349     return weights
    350 

AssertionError:

I have trouble when running the train_tf2.py script. Do you have any example script for executing train_tf2.py with CocoSeg dataset?

Execution Env

• Google Colab Pro
• tensorflow==2.1.0
• tensorflow-gpu==2.1.0
• python==3.6.9

Execution Script

python train_tf2.py --model simplepose_mobile_mobilenetv3_small_w1_coco --dataset CocoSeg --batch-size 1

Error Message

Full Message

2020-03-01 02:41:24.974479: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libnvinfer.so.6'; dlerror: libnvinfer.so.6: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/lib64-nvidia
2020-03-01 02:41:24.974607: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libnvinfer_plugin.so.6'; dlerror: libnvinfer_plugin.so.6: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/lib64-nvidia
2020-03-01 02:41:24.974629: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:30] Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.
INFO:root:Script command line:
train_tf2.py --model simplepose_mobile_mobilenetv3_small_w1_coco --dataset CocoSeg --batch-size 1
INFO:root:Script arguments:
Namespace(attempt=1, batch_size=1, data_dir='../imgclsmob_data/coco', dataset='CocoSeg', image_base_size=520, image_crop_size=480, in_channels=3, log_interval=50, log_packages='tensorflow-gpu', log_pip_packages='tensorflow-gpu', logging_file_name='train.log', lr=0.1, lr_decay=0.1, lr_decay_epoch='40,60', lr_decay_period=0, lr_mode='cosine', model='simplepose_mobile_mobilenetv3_small_w1_coco', momentum=0.9, num_classes=21, num_epochs=120, num_gpus=0, num_workers=4, optimizer_name='nag', resume='', resume_state='', save_dir='', save_interval=4, seed=2949, start_epoch=1, target_lr=1e-08, use_pretrained=False, wd=0.0001, work_dir='../imgclsmob_data')
fatal: not a git repository (or any of the parent directories): .git
INFO:root:Env_stats:
{
    "tensorflow-gpu": "Name: tensorflow-gpu\nVersion: 2.1.0\nSummary: TensorFlow is an open source machine learning framework for everyone.\nHome-page: https://www.tensorflow.org/\nAuthor: Google Inc.\nAuthor-email: [email protected]\nLicense: Apache 2.0\nLocation: /usr/local/lib/python3.6/dist-packages\nRequires: wheel, tensorboard, google-pasta, gast, keras-preprocessing, scipy, termcolor, numpy, astor, absl-py, six, protobuf, grpcio, wrapt, keras-applications, opt-einsum, tensorflow-estimator\nRequired-by:",
    "python": "3.6.9",
    "pwd": "/content/common",
    "git": "unknown",
    "platform": "Linux-4.14.137+-x86_64-with-Ubuntu-18.04-bionic"
}
2020-03-01 02:41:29.288155: W tensorflow/core/common_runtime/gpu/gpu_bfc_allocator.cc:39] Overriding allow_growth setting because the TF_FORCE_GPU_ALLOW_GROWTH environment variable is set. Original config value was 0.
Traceback (most recent call last):
  File "train_tf2.py", line 291, in <module>
    main()
  File "train_tf2.py", line 253, in main
    assert (ds_metainfo.ml_type != "imgseg") or args.disable_cudnn_autotune
AttributeError: 'Namespace' object has no attribute 'disable_cudnn_autotune'

What I tried

Add disable_cudnn_autotune arg on execution command

Not working 😓

python train_tf2.py --model simplepose_mobile_mobilenetv3_small_w1_coco --dataset CocoSeg --batch-size 1 --disable_cudnn_autotune true

python train_tf2.py --model simplepose_mobile_mobilenetv3_small_w1_coco --dataset CocoSeg --batch-size 1 --disable_cudnn_autotune

python train_tf2.py --model simplepose_mobile_mobilenetv3_small_w1_coco --dataset CocoSeg --batch-size 1 --disable-cudnn-autotune

Full Source

Here is a full notebook what I work.

thanks

This is an awesome collection of classifiers, with many classifiers having multiple variations for speed/accuracy tradeoff.
Several state of the art classifiers are included such as mobilenet_v3 and efficient-net.
These networks also have Edge-Tpu specific variations:
EfficientNet-EdgeTPU (small, medium & large): https://github.com/tensorflow/tpu/tree/master/models/official/efficientnet/edgetpu
https://ai.googleblog.com/2019/08/efficientnet-edgetpu-creating.html

Mobilenet_v3 edgetpu:
https://github.com/tensorflow/models/tree/master/research/slim/nets/mobilenet#mobilenet-edgetpu-latency
https://ai.googleblog.com/2019/11/introducing-next-generation-on-device.html

Will it be possible to add them?

Hi,

I tried to use ImageNet classifier with a pretrained model SE-ResNext50. I added just a Softmax activation. It results in values with mean 0.01 and range about 0.005-0.02, which don't seem like proper softmax predictions. Actual results are incorrect.

If I change 1 line to use torchvision.models.resnet50, inference works like a charm: it detects dogs, cars, etc. Is there anything I'm missing? I'm using pytorchcv==0.0.45.

Btw, maybe you should add __version__ into the package.

Cheers!

hi,
thank you for the great repository.
The problem that I came to is that I cannot find a list of classes for which the tf2 models of imagenet1k were trained for. Also the website of imagenet is not very helpfull in that regard.

For example, how could I load an image, classify it and print the class label?
An example for how to do this would be very helpful to add.

Traceback
(most recent call last):
File "C:/Users/THINKPAD/Desktop/kaggle_code/quickdraw-doodle-recognition/baseline/kerascv_test.py", line 4, in
net=get_model("resnet10",pretrained=True)
File "D:\python3.6\lib\site-packages\kerascv\model_provider.py", line 172, in get_model
net = _modelsname
File "D:\python3.6\lib\site-packages\kerascv\models\resnet.py", line 532, in resnet10
return get_resnet(blocks=10, model_name="resnet10", kwargs)
File "D:\python3.6\lib\site-packages\kerascv\models\resnet.py", line 506, in get_resnet
kwargs)
File "D:\python3.6\lib\site-packages\kerascv\models\resnet.py", line 401, in resnet
name="features/init_block")
File "D:\python3.6\lib\site-packages\kerascv\models\resnet.py", line 354, in res_init_block
name=name + "/conv")
File "D:\python3.6\lib\site-packages\kerascv\models\resnet.py", line 64, in res_conv
x = GluonBatchNormalization(name=name + "/bn")(x)
File "D:\python3.6\lib\site-packages\keras\engine\base_layer.py", line 497, in call
arguments=user_kwargs)
File "D:\python3.6\lib\site-packages\keras\engine\base_layer.py", line 565, in _add_inbound_node
output_tensors[i]._keras_shape = output_shapes[i]
AttributeError: 'NoneType' object has no attribute '_keras_shape'

What's wrong with it ? I have changed 'channle_last' to 'channel_first'

Hi guys

I'd like to thank you for this wonderful collection! One little issue I found is, there's a lot of code like this:

self.activ = nn.ReLU(inplace=True)
...
x = self.activ(x)

which modifies both old copy of x and a new copy (https://discuss.pytorch.org/t/one-of-the-variables-needed-for-gradient-computation-has-been-modified-by-an-inplace-operation/12578/7). This results in the following error when I try to use Xception (I'm using PyTorch 1.0.0):

Traceback (most recent call last):
  File "./train_j13_xception.py", line 497, in <module>
    train(train_loader, model, criterion, optimizer, epoch, lr_scheduler)
  File "./train_j13_xception.py", line 282, in train
    loss.backward()
  File "/home/cppg/.local/lib/python3.6/site-packages/torch/tensor.py", line 102, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph)
  File "/home/cppg/.local/lib/python3.6/site-packages/torch/autograd/__init__.py", line 90, in backward
    allow_unreachable=True)  # allow_unreachable flag
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation
Command exited with non-zero status 1

Removing of inplace=True fixes this. It's an easy search-and-replace fix, but I saw this in many files, so I didn't want to push a giant PR.

Cheers

Is there a good way to go about modifying the pre-trained models? I want to tweak the forward() function to return activations at a few layers.
I'm going to be comparing between several CIFAR models so training them all myself isn't really viable.

Thanks!

Hi, thanks for the great repo.
I was wondering if you could provide some examples of train/eval scripts in order to train models that are currently missing from the repo, with sensible default params, so that we can contribute back any trained models currently missing? Maybe adding those train/eval script examples in the wiki would be valuable?

I am trying to use pspnet_resnetd101b_voc as follows:

sample_batch_size = 1
channel = 3
height, width = 224.224
dummy_input = torch.randn(sample_batch_size, channel, height, width)
out = pspnet_resnetd101b_voc(dummy_input)

But I get following error: Expected more than 1 value per channel when training, got input size torch.Size([1, 512, 1, 1])

What are the right data format for this model or similar as pspnet_resnetd101b_coco?.

Note: With Imagenet pretrained model resnetd101b works fine.

In pytorch/pytorchcv/model_provider.py , under _models on line 108 , squeezenext is missing although it has been imported in line 47

Traceback (most recent call last):
File "C:/Users/THINKPAD/Desktop/kaggle_code/quickdraw-doodle-recognition/baseline/kerascv_test.py", line 4, in
net=get_model("resnext101_32x4d",pretrained=True)
File "D:\python3.6\lib\site-packages\kerascv\model_provider.py", line 172, in get_model
net = _modelsname
File "D:\python3.6\lib\site-packages\kerascv\models\resnext.py", line 487, in resnext101_32x4d
return get_resnext(blocks=101, cardinality=32, bottleneck_width=4, model_name="resnext101_32x4d", **kwargs)
File "D:\python3.6\lib\site-packages\kerascv\models\resnext.py", line 446, in get_resnext
**kwargs)
File "D:\python3.6\lib\site-packages\kerascv\models\resnext.py", line 380, in resnext
name="features/stage{}/unit{}".format(i + 1, j + 1))
File "D:\python3.6\lib\site-packages\kerascv\models\resnext.py", line 273, in resnext_unit
name=name + "/body")
File "D:\python3.6\lib\site-packages\kerascv\models\resnext.py", line 208, in resnext_bottleneck
name=name + "/conv2")
File "D:\python3.6\lib\site-packages\kerascv\models\resnext.py", line 155, in resnext_conv3x3
name=name)
File "D:\python3.6\lib\site-packages\kerascv\models\resnext.py", line 66, in resnext_conv
name=name + "/conv")
File "D:\python3.6\lib\site-packages\kerascv\models\common.py", line 109, in conv2d
name=name + "/convgroup{}".format(gi + 1))(xi)
File "D:\python3.6\lib\site-packages\keras\engine\base_layer.py", line 414, in call
self.assert_input_compatibility(inputs)
File "D:\python3.6\lib\site-packages\keras\engine\base_layer.py", line 311, in assert_input_compatibility
str(K.ndim(x)))
ValueError: Input 0 is incompatible with layer features/stage1/unit1/body/conv2/conv/convgroup1: expected ndim=4, found ndim=0

there are some problems when i try to use 'ResNeXt' . What's wrong with it?

in this example:
net = tfcv_get_model("resnet18", pretrained=True, data_format="channels_first") x = tf.placeholder(dtype=tf.float32, shape=(None, 3, 224, 224), name="xx") y_net = net(x) with tf.Session() as sess: tfcv_init_variables_from_state_dict(sess=sess, state_dict=net.state_dict) x_value = np.zeros((1, 3, 224, 224), np.float32) y = sess.run(y_net, feed_dict={x: x_value})
I replace the x with TFRecord by tf.parse_single_example. But it seems stuck?

Hello @osmr.
I see you have a lot different models and so on. Have you looked at the A^2-nets approach? I haven't found any implementation on github, and maybe you plan to implement this one?
the paper

I use this ropo keras model,but save model show this error.

Traceback (most recent call last):
  File "/code/train_4.py", line 209, in <module>
    verbose=1, callbacks=cbks)
  File "/opt/conda/lib/python3.6/site-packages/comet_ml/monkey_patching.py", line 246, in wrapper
    return_value = original(*args, **kwargs)
  File "/opt/conda/lib/python3.6/site-packages/keras/legacy/interfaces.py", line 91, in wrapper
    return func(*args, **kwargs)
  File "/opt/conda/lib/python3.6/site-packages/keras/engine/training.py", line 1418, in fit_generator
    initial_epoch=initial_epoch)
  File "/opt/conda/lib/python3.6/site-packages/keras/engine/training_generator.py", line 251, in fit_generator
    callbacks.on_epoch_end(epoch, epoch_logs)
  File "/opt/conda/lib/python3.6/site-packages/keras/callbacks.py", line 79, in on_epoch_end
    callback.on_epoch_end(epoch, logs)
  File "/opt/conda/lib/python3.6/site-packages/keras/callbacks.py", line 446, in on_epoch_end
    self.model.save(filepath, overwrite=True)
  File "/opt/conda/lib/python3.6/site-packages/keras/engine/network.py", line 1090, in save
    save_model(self, filepath, overwrite, include_optimizer)
  File "/opt/conda/lib/python3.6/site-packages/keras/engine/saving.py", line 382, in save_model
    _serialize_model(model, f, include_optimizer)
  File "/opt/conda/lib/python3.6/site-packages/keras/engine/saving.py", line 83, in _serialize_model
    model_config['config'] = model.get_config()
  File "/opt/conda/lib/python3.6/site-packages/keras/engine/network.py", line 931, in get_config
    return copy.deepcopy(config)
  File "/opt/conda/lib/python3.6/copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "/opt/conda/lib/python3.6/copy.py", line 240, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/opt/conda/lib/python3.6/copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "/opt/conda/lib/python3.6/copy.py", line 215, in _deepcopy_list
    append(deepcopy(a, memo))
  File "/opt/conda/lib/python3.6/copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "/opt/conda/lib/python3.6/copy.py", line 240, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/opt/conda/lib/python3.6/copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "/opt/conda/lib/python3.6/copy.py", line 240, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/opt/conda/lib/python3.6/copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "/opt/conda/lib/python3.6/copy.py", line 220, in _deepcopy_tuple
    y = [deepcopy(a, memo) for a in x]
  File "/opt/conda/lib/python3.6/copy.py", line 220, in <listcomp>
    y = [deepcopy(a, memo) for a in x]
  File "/opt/conda/lib/python3.6/copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "/opt/conda/lib/python3.6/copy.py", line 220, in _deepcopy_tuple
    y = [deepcopy(a, memo) for a in x]
  File "/opt/conda/lib/python3.6/copy.py", line 220, in <listcomp>
    y = [deepcopy(a, memo) for a in x]
  File "/opt/conda/lib/python3.6/copy.py", line 169, in deepcopy
    rv = reductor(4)
TypeError: can't pickle module objects

checkpoint = ModelCheckpoint('./gdrive/My Drive/MLAI_files/project-images/Full_Dataset/squeezenext_weights.{epoch:02d}-{val_loss:.2f}.hdf5',
monitor='val_loss', save_best_only=True, mode='min')
callbacks_list = [checkpoint]
hist = new_model.fit_generator(generator = my_gen(train_generator),
steps_per_epoch = STEP_SIZE_TRAIN,
validation_data = my_gen(valid_generator),
validation_steps = STEP_SIZE_VALID,
epochs = 3,
callbacks = callbacks_list)

Epoch 1/3
38/377 [==>...........................] - ETA: 3:54 - loss: 2.0859 - acc: 0.2188/usr/local/lib/python3.6/dist-packages/PIL/Image.py:914: UserWarning: Palette images with Transparency expressed in bytes should be converted to RGBA images
'to RGBA images')
117/377 [========>.....................] - ETA: 2:35 - loss: 2.0482 - acc: 0.2356/usr/local/lib/python3.6/dist-packages/PIL/TiffImagePlugin.py:742: UserWarning: Corrupt EXIF data. Expecting to read 4 bytes but only got 0.
warnings.warn(str(msg))
377/377 [==============================] - 221s 586ms/step - loss: 2.0704 - acc: 0.2301 - val_loss: 5.2917 - val_acc: 0.2820

TypeError Traceback (most recent call last)
in ()
4 validation_steps = STEP_SIZE_VALID,
5 epochs = 3,
----> 6 callbacks = callbacks_list)

12 frames
/usr/local/lib/python3.6/dist-packages/keras/engine/saving.py in get_json_type(obj)
89 return obj.name
90
---> 91 raise TypeError('Not JSON Serializable: %s' % (obj,))
92
93 from .. import version as keras_version

TypeError: Not JSON Serializable: <module 'tensorflow' from '/usr/local/lib/python3.6/dist-packages/tensorflow/init.py'>

m = ptcv_get_model("mobilenetv2", pretrained=True) returns
ValueError: Unsupported model: mobilenetv2

If i use the mobilenetv2.py file and do:

get_mobilenetv2(width_scale=1.0, model_name="mobilenetv2_w1") it works fine, but if i use pretrained = True i get:

ModuleNotFoundError: No module named 'main.model_store'; 'main' is not a package

Is there any way i can fix this?

Hi, this might be an off-topic question.

I was wondering if you'd be able to help me.
I'm trying to import resnet-10 as a feature extractor to TensorFlow's Object Detection API.
I posted a question at stackoverflow, and I wanted to know if my way of importing only the top-most layers (those that don't convert the model into a classifier) was right.

Any other help solving the issue is welcome!
Thanks

Hi, do you have any plan to release this paper implementation?

MixConv: Mixed Depthwise Convolutional Kernels https://arxiv.org/abs/1907.09595

Pip installing pytorchcv then importing get_model results in an error:

/usr/local/lib/python3.6/dist-packages/pytorchcv/model_provider.py in <module>()
    111 from .models.sinet import *
    112 
--> 113 from .models.others.oth_sinet import *
    114 
    115 from .models.superpointnet import *

ModuleNotFoundError: No module named 'pytorchcv.models.others'

THIS IS such a COOL project!
Would you please share the imagenet-pretrained shufflenet v2 x1.0 model?Thanks very much @osmr

(PyTorch 0.4.1, sqnxt23v5_w2)
Backward pass in PyTorch with squeezenext returns an error:

  File ".env/local/lib/python3.6/site-packages/torch/tensor.py", line 93, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph)
  File ".env/local/lib/python3.6/site-packages/torch/autograd/__init__.py",line 90, in backward
    allow_unreachable=True)  # allow_unreachable flag
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation

Changing ReLU in line 115 in to

self.activ = nn.ReLU(inplace=False)

solves the problem, but perhaps there are better solutions. For some reason activation in line 122 can't be inplace, so a change there to

identity = torch.nn.functional.relu(identity, inplace=False)

would be enough

Hi, unfortunately I fall in another problem.
i want to use your weights for input size of 600*600 and then get 3 outputs eg. [(20,20,-1),(40,40,-1),(80,80,-1)] from different stages. could you plz help me with this....
all of this with tf2
thx

I could not get some of the models that are included your keys. An example would be seresnet101b.
I used ptcv_get_model('seresnet101b', pretrained=True)

I did some digging and the model seem to be missing in the _model_sha1... as there are 691 models returned from pytorchcv.model_provider._models.keys() but only 444 models exist in _model_sha1 in model_store.py.

could you please add the missing models or fix the keys?

Thanks.

Hi @osmr ,

Nice work!
How did you get the IGCV3 performance on imagenet, did you train the model by yourself or cite acc data from original repo? Since i found the acc of IGCV3(IGCV3 x1.0) differs from the original repo(72.2).

Hey,

I tried to use the ZFNet pretrained model to see how this one perform. I add softmax function and here is the weird result I got:

I really wonder where does the problem come from. My code doesn't seem to have any error. Link to the repo : https://github.com/3outeille/Research-Paper-Summary/blob/master/src/architecture/zfnet/pytorch/zfnet_pytorch.ipynb

imagenet_mean = np.array([104., 117., 124.], dtype=np.float32)
fig2 = plt.figure(figsize=(30,10))

for i, image in enumerate(imgs):
    img = cv2.resize(image.astype(np.float32), (224, 224))
    img -= imagenet_mean
    img = img.reshape((1, 3, 224, 224))
    img = torch.from_numpy(img)
    
    probs = torch.nn.Softmax(dim=-1)(net(img))
    probs = probs.detach().numpy()

Does it mean that there was a bug during ZFNet training ?

I'm using SimplePose models and train them with my custom dataset generator with this snippet:

print(f'Tensror Flow version: {tf.__version__}')
tf.keras.backend.clear_session()

BATCH_SIZE=64
NUM_KEYPOINTS=14
IMAGE_RES=128
HEATMAP_RES=32

net = tf2cv_get_model("simplepose_mobile_mobilenetv3_large_w1_coco", 
                      pretrained_backbone=True,
                      keypoints=NUM_KEYPOINTS, 
                      return_heatmap=True)

net.build(input_shape=(BATCH_SIZE, IMAGE_RES, IMAGE_RES, 3))
net.heatmap_max_det.build((BATCH_SIZE, HEATMAP_RES, HEATMAP_RES, NUM_KEYPOINTS))
net.summary()
net.compile(optimizer=tf.keras.optimizers.Adam(lr=5e-4), 
            loss=tf.keras.losses.mean_squared_error)

history = net.fit_generator(
  generator=train_data,
  validation_data=valid_data,
  epochs=15
)

And that works or not depending on the model type I choose. If I use any non mobile model ( simplepose_resnet18_coco for example) then everything works, the network trains and predicts accurate results. Whereas if use any mobile model like simplepose_mobile_mobilenetv3_large_w1_coco or simplepose_mobile_resnet18_coco, the above code will break with the following error:

Tensror Flow version: 2.1.0
Downloading /root/.tensorflow/models/mobilenetv3_large_w1-0769-f66596ae.tf2.h5.zip from https://github.com/osmr/imgclsmob/releases/download/v0.0.422/mobilenetv3_large_w1-0769-f66596ae.tf2.h5.zip...
Model: "simple_pose_mobile"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
backbone (Sequential)        multiple                  2996352   
_________________________________________________________________
decoder (Sequential)         multiple                  1798080   
_________________________________________________________________
heatmap_max_det (HeatmapMaxD multiple                  0         
=================================================================
Total params: 4,794,432
Trainable params: 4,768,240
Non-trainable params: 26,192
_________________________________________________________________

TypeError: in converted code:

    /usr/local/lib/python3.6/dist-packages/tf2cv/models/simpleposemobile_coco.py:94 call  *
        heatmap = self.decoder(x, training=training)
    /tensorflow-2.1.0/python3.6/tensorflow_core/python/keras/engine/base_layer.py:773 __call__
        outputs = call_fn(cast_inputs, *args, **kwargs)
    /usr/local/lib/python3.6/dist-packages/tf2cv/models/common.py:2016 call  *
        x = self.pix_shuffle(x)
    /tensorflow-2.1.0/python3.6/tensorflow_core/python/keras/engine/base_layer.py:773 __call__
        outputs = call_fn(cast_inputs, *args, **kwargs)
# .... 

    TypeError: Failed to convert object of type <class 'tuple'> to Tensor. Contents: (None, 4, 4, 128, 4). Consider casting elements to a supported type.

It looks to me like the net output is in some unexpected format.
See this gist for the full output: https://gist.github.com/grin/d1a9836aca5ca462dbb03527246ba941

What could be causing this error? Am I missing some crucial configuration for mobile models? I would expect the two APIs work similarly.

Thank you

I tried running the model and I ran into this error. Did this link change?

https://github.com/osmr/imgclsmob/releases/download/v0.0.242/shufflenetv2b_w2-0810-2149df38.h5.zip

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-31-55f09ca47a7e> in <module>()
----> 1 model = create_model(input_shape=(img_size_in, img_size_in, 3), n_out=1)

<ipython-input-30-49a6d27ad1db> in create_model(input_shape, n_out)
      1 def create_model(input_shape, n_out):
      2     inp_mask = Input(shape=input_shape)
----> 3     net = kecv_get_model("shufflenetv2b_w2", pretrained=True)#"shufflenetv2_w3d2"
      4     net.name = 'shufflenet'
      5 

/opt/conda/lib/python3.6/site-packages/kerascv/model_provider.py in get_model(name, **kwargs)
    209     if name not in _models:
    210         raise ValueError('Unsupported model: {}'.format(name))
--> 211     net = _models[name](**kwargs)
    212     return net

/opt/conda/lib/python3.6/site-packages/kerascv/models/shufflenetv2b.py in shufflenetv2b_w2(**kwargs)
    375         width_scale=(61.0 / 29.0),
    376         model_name="shufflenetv2b_w2",
--> 377         **kwargs)
    378 
    379 

/opt/conda/lib/python3.6/site-packages/kerascv/models/shufflenetv2b.py in get_shufflenetv2b(width_scale, model_name, pretrained, root, **kwargs)
    281             net=net,
    282             model_name=model_name,
--> 283             local_model_store_dir_path=root)
    284 
    285     return net

/opt/conda/lib/python3.6/site-packages/kerascv/models/model_store.py in download_model(net, model_name, local_model_store_dir_path)
    489         file_path=get_model_file(
    490             model_name=model_name,
--> 491             local_model_store_dir_path=local_model_store_dir_path))

/opt/conda/lib/python3.6/site-packages/kerascv/models/model_store.py in get_model_file(model_name, local_model_store_dir_path)
    181             file_name=file_name),
    182         path=zip_file_path,
--> 183         overwrite=True)
    184     with zipfile.ZipFile(zip_file_path) as zf:
    185         zf.extractall(local_model_store_dir_path)

/opt/conda/lib/python3.6/site-packages/kerascv/models/model_store.py in _download(url, path, overwrite, sha1_hash, retries, verify_ssl)
    266                 retries -= 1
    267                 if retries <= 0:
--> 268                     raise e
    269                 else:
    270                     print("download failed, retrying, {} attempt{} left"

/opt/conda/lib/python3.6/site-packages/kerascv/models/model_store.py in _download(url, path, overwrite, sha1_hash, retries, verify_ssl)
    252                 r = requests.get(url, stream=True, verify=verify_ssl)
    253                 if r.status_code != 200:
--> 254                     raise RuntimeError("Failed downloading url {}".format(url))
    255                 with open(fname, "wb") as f:
    256                     for chunk in r.iter_content(chunk_size=1024):

RuntimeError: Failed downloading url https://github.com/osmr/imgclsmob/releases/download/v0.0.242/shufflenetv2b_w2-0810-2149df38.h5.zip

I got this error when I had tried to import preresnet18 model.

Code:

from pytorchcv.model_provider import get_model as ptcv_get_model
net = ptcv_get_model("PreResNet18", pretrained=True)

Error log:

Downloading /home/nero/.torch/models/preresnet18-1018-98958fd2.pth.zip from https://github.com/osmr/imgclsmob/releases/download/v0.0.39/preresnet18-1018-98958fd2.pth.zip...
download failed, retrying, 4 attempts left
Downloading /home/nero/.torch/models/preresnet18-1018-98958fd2.pth.zip from https://github.com/osmr/imgclsmob/releases/download/v0.0.39/preresnet18-1018-98958fd2.pth.zip...
download failed, retrying, 3 attempts left
Downloading /home/nero/.torch/models/preresnet18-1018-98958fd2.pth.zip from https://github.com/osmr/imgclsmob/releases/download/v0.0.39/preresnet18-1018-98958fd2.pth.zip...
download failed, retrying, 2 attempts left
Downloading /home/nero/.torch/models/preresnet18-1018-98958fd2.pth.zip from https://github.com/osmr/imgclsmob/releases/download/v0.0.39/preresnet18-1018-98958fd2.pth.zip...
download failed, retrying, 1 attempt left
Downloading /home/nero/.torch/models/preresnet18-1018-98958fd2.pth.zip from https://github.com/osmr/imgclsmob/releases/download/v0.0.39/preresnet18-1018-98958fd2.pth.zip...
Traceback (most recent call last):
  File "convert.py", line 135, in <module>
    net = ptcv_get_model("PreResNet18", pretrained=True)
  File "/home/nero/Files/Workspace/env/lib/python3.6/site-packages/pytorchcv/model_provider.py", line 190, in get_model
    net = _models[name](**kwargs)
  File "/home/nero/Files/Workspace/env/lib/python3.6/site-packages/pytorchcv/models/preresnet.py", line 582, in preresnet18
    return get_preresnet(blocks=18, model_name="preresnet18", **kwargs)
  File "/home/nero/Files/Workspace/env/lib/python3.6/site-packages/pytorchcv/models/preresnet.py", line 461, in get_preresnet
    local_model_store_dir_path=root)))
  File "/home/nero/Files/Workspace/env/lib/python3.6/site-packages/pytorchcv/models/model_store.py", line 133, in get_model_file
    overwrite=True)
  File "/home/nero/Files/Workspace/env/lib/python3.6/site-packages/pytorchcv/models/model_store.py", line 220, in _download
    raise e
  File "/home/nero/Files/Workspace/env/lib/python3.6/site-packages/pytorchcv/models/model_store.py", line 204, in _download
    r = requests.get(url, stream=True, verify=verify_ssl)
AttributeError: type object 'requests_failed_to_import' has no attribute 'get'

Environment:

• Python 3.6.6
• PyTorch 0.4.0

For the Keras models, does the input parameter in_size actually do something? I notice that, for all models, the input image dimensions are hard-coded to 224 x 224. Shouldn't it be

input_shape = (in_channels, *in_size) if is_channels_first() else (*in_size, in_channels)

for all models?