@thibo73800 During training with different images sizes i.e. 128x128 when it calls and runs init_session() in the init() and self.sess.run(tf.global_variables_initializer()) in init_session() system memory reaches up to 32Gb and get crash. Why is like that? ( Perhaps I only change the caps_2_vec_len to 128 in json file as well tplaceholder() input argument of (32,32) to (128,128) in _build_inputs() and resize_nearest_neighbor() input argument from (32,32) to (128,128) in _build_decoder() )

Along with accuracy, how one can calculate precision and recall on this dataset?

Hi, Would you like to discuss the data set? Did you resize all images to 32x32 or 28x28? Please reply.

Hi,

May I know what is the data format you had used on the train, test, valid pickle files?

How can we use ur code in other RGB dataset?
Suppose the structure of dataset is like that. it contains some sub-folder. Each sub-folder represents one class.

Class A:
0001.jpg 1
0002.jpg 1
Class B:
0001.jpg 2
0002.jpg 2

I test the model trained by your capsule net code followed your guide as:
$> python test.py outputs/checkpoints/ckpt_name dataset/
but the terminal point out the error:
Traceback (most recent call last):
File "test.py", line 100, in
test(arguments[""], arguments[""])
File "test.py", line 81, in test
model.load(ckpt)
File "/home/cs/capsnet_traffic_sign_classifier/model_base.py", line 317, in load
tensors = json.loads(tensors)
File "/usr/lib/python3.5/json/init.py", line 312, in loads
s.class.name))
TypeError: the JSON object must be str, not 'bytes'
Could you tell me how to test the model?

I trained the model in my PC with the GTX-1060, I interrupt the training use Ctrl+c
I get 4 files in checkpoints, they are:
1.checkpoint
2.c2n_64_c1s_5_c1vl_16_c1s_9_c2d_0.7_c1nf_16_lr_0.0001_rs_1_c2vl_32_c1n_256_c2s_6--TrafficSign--1510475262.826812.data-00000-of-00001
3.c2n_64_c1s_5_c1vl_16_c1s_9_c2d_0.7_c1nf_16_lr_0.0001_rs_1_c2vl_32_c1n_256_c2s_6--TrafficSign--1510475262.826812.index
4.c2n_64_c1s_5_c1vl_16_c1s_9_c2d_0.7_c1nf_16_lr_0.0001_rs_1_c2vl_32_c1n_256_c2s_6--TrafficSign--1510475262.826812.meta
The content of the file in checkpoint is：
model_checkpoint_path: "/home/chenshuo/capsnet_traffic_sign_classifier/outputs/checkpoints/c2n_64_c1s_5_c1vl_16_c1s_9_c2d_0.7_c1nf_16_lr_0.0001_rs_1_c2vl_32_c1n_256_c2s_6--TrafficSign--1510475262.826812"
all_model_checkpoint_paths: "/home/chenshuo/capsnet_traffic_sign_classifier/outputs/checkpoints/c2n_64_c1s_5_c1vl_16_c1s_9_c2d_0.7_c1nf_16_lr_0.0001_rs_1_c2vl_32_c1n_256_c2s_6--TrafficSign--1510475262.826812"
I test the model use:
python3 test.py outputs/checkpoints/c2n_64_c1s_5_c1vl_16_c1s_9_c2d_0.7_c1nf_16_lr_0.0001_rs_1_c2vl_32_c1n_256_c2s_6--TrafficSign--1510475262.826812 dataset/
The error is:
ModelBase::Loading ckpt ...
2017-11-13 10:41:21.279347: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
2017-11-13 10:41:21.376162: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:892] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2017-11-13 10:41:21.376427: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1030] Found device 0 with properties:
name: GeForce GTX 1060 6GB major: 6 minor: 1 memoryClockRate(GHz): 1.7715
pciBusID: 0000:01:00.0
totalMemory: 5.93GiB freeMemory: 5.56GiB
2017-11-13 10:41:21.376442: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1120] Creating TensorFlow device (/device:GPU:0) -> (device: 0, name: GeForce GTX 1060 6GB, pci bus id: 0000:01:00.0, compute capability: 6.1)
INFO:tensorflow:Restoring parameters from outputs/checkpoints/c2n_64_c1s_5_c1vl_16_c1s_9_c2d_0.7_c1nf_16_lr_0.0001_rs_1_c2vl_32_c1n_256_c2s_6--TrafficSign--1510475262.826812
Restoring parameters from outputs/checkpoints/c2n_64_c1s_5_c1vl_16_c1s_9_c2d_0.7_c1nf_16_lr_0.0001_rs_1_c2vl_32_c1n_256_c2s_6--TrafficSign--1510475262.826812
Traceback (most recent call last):
File "test.py", line 100, in
test(arguments[""], arguments[""])
File "test.py", line 81, in test
model.load(ckpt)
File "/home/chenshuo/capsnet_traffic_sign_classifier/model_base.py", line 317, in load
tensors = json.loads(tensors)
File "/usr/lib/python3.5/json/init.py", line 312, in loads
s.class.name))
TypeError: the JSON object must be str, not 'bytes'

Could you tell me why I can not test the model and what should I do?
Thank you.

I wonder, did you also get any reconstruction results while training/testing? If so, could you please share them here with us?

Thanks a lot sir,
Could you apply this to the diabetic retinopathy dataset from kaggle competition ?

I'm trying to convert the checkpoints to saved model for tensorflow serving.Can you help me?

The path of ckpt and tensorboard which are saved every 1000 batches is a little bit wrong with the
command linepython train.py path/to/dataset outputs/checkpoints/c1s_9_c1n_256_c2s_6_c2n_64_c2d_0.7_c1vl_16_c1s_5_c1nf_16_c2vl_32_lr_0.0001_rs_1--TrafficSign--1516951763.9006186

When I test the model using a trained model I get the error below. My TensorFlow Version: 1.2.1
this error changes when I rerun the kernel into ValueError: At least two variables have the same name: weight and into ValueError: At least two variables have the same name: fully_connected/biases

`ModelBase::Loading ckpt ...
INFO:tensorflow:Restoring parameters from outputs/checkpoints/c1nf_16_c2d_0.7_lr_0.0001_c1s_5_rs_1_c1vl_16_c1n_256_c2s_6_c2n_64_c2vl_32_c1s_9--TrafficSign--1513264580.046234
Restoring parameters from outputs/checkpoints/c1nf_16_c2d_0.7_lr_0.0001_c1s_5_rs_1_c1vl_16_c1n_256_c2s_6_c2n_64_c2vl_32_c1s_9--TrafficSign--1513264580.046234
ModelBase::Ckpt ready
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-10-e5003bd16d6a> in <module>()
     89 skpt = "outputs/checkpoints/c1nf_16_c2d_0.7_lr_0.0001_c1s_5_rs_1_c1vl_16_c1n_256_c2s_6_c2n_64_c2vl_32_c1s_9--TrafficSign--1513264580.046234"
     90 testing_file = "dataset/"
---> 91 test(testing_file, skpt)

<ipython-input-10-e5003bd16d6a> in test(dataset, ckpt)
     66     model = ModelTrafficSign("TrafficSign", output_folder=None)
     67     # Load the model
---> 68     model.load(ckpt)
     69 
     70     # Evaluate all the dataset

/home/a/WA/CapsNet/capsnet-traffic-sign-classifier/model_base.py in load(self, ckpt)
    345 
    346         self.model_name = ckpt.split("/")[-1]
--> 347         self.saver = tf.train.Saver()
    348 
    349 

/home/a/anaconda3/envs/keras/lib/python3.5/site-packages/tensorflow/python/training/saver.py in __init__(self, var_list, reshape, sharded, max_to_keep, keep_checkpoint_every_n_hours, name, restore_sequentially, saver_def, builder, defer_build, allow_empty, write_version, pad_step_number, save_relative_paths)
   1137     self._pad_step_number = pad_step_number
   1138     if not defer_build:
-> 1139       self.build()
   1140     if self.saver_def:
   1141       self._check_saver_def()

/home/a/anaconda3/envs/keras/lib/python3.5/site-packages/tensorflow/python/training/saver.py in build(self)
   1168           keep_checkpoint_every_n_hours=self._keep_checkpoint_every_n_hours,
   1169           name=self._name,
-> 1170           restore_sequentially=self._restore_sequentially)
   1171     elif self.saver_def and self._name:
   1172       # Since self._name is used as a name_scope by builder(), we are

/home/a/anaconda3/envs/keras/lib/python3.5/site-packages/tensorflow/python/training/saver.py in build(self, names_to_saveables, reshape, sharded, max_to_keep, keep_checkpoint_every_n_hours, name, restore_sequentially, filename)
    671         unique.
    672     """
--> 673     saveables = self._ValidateAndSliceInputs(names_to_saveables)
    674     if max_to_keep is None:
    675       max_to_keep = 0

/home/a/anaconda3/envs/keras/lib/python3.5/site-packages/tensorflow/python/training/saver.py in _ValidateAndSliceInputs(self, names_to_saveables)
    555     """
    556     if not isinstance(names_to_saveables, dict):
--> 557       names_to_saveables = BaseSaverBuilder.OpListToDict(names_to_saveables)
    558 
    559     saveables = []

/home/a/anaconda3/envs/keras/lib/python3.5/site-packages/tensorflow/python/training/saver.py in OpListToDict(op_list)
    533         if name in names_to_saveables:
    534           raise ValueError("At least two variables have the same name: %s" %
--> 535                            name)
    536         names_to_saveables[name] = var
    537       # pylint: enable=protected-access

ValueError: At least two variables have the same name: conv2d_1/kernel/Adam`

How do you use tensorboard to draw the loss and accuracy diagrams in the end?

Also how do you set an specific number of epochs for the algorithm to stop?

i am getting model_base not found while running the code,what should i do?

Hi, I was trying to run train.py dataset on AWS cloud with ubuntu 14.04 but got this error, I am pasting all the errors, please help me help me on how to fix this ?

ValueErrorTraceback (most recent call last)
/home/ubuntu/capsnet-traffic-sign-classifier/train.py in ()
123 if name == 'main':
124 arguments = docopt(doc)
--> 125 train(arguments[""], arguments["--ckpt"], arguments[""])

/home/ubuntu/capsnet-traffic-sign-classifier/train.py in train(dataset, ckpt, output)
53 return np.array(img) / 255
54
---> 55 X_train, y_train, X_valid, y_valid, X_test, y_test = get_data(dataset)
56
57 X_train = X_train / 255

/home/ubuntu/capsnet-traffic-sign-classifier/data_handler.pyc in get_data(folder)
21
22 with open(training_file, mode='rb') as f:
---> 23 train = pickle.load(f)
24 with open(validation_file, mode='rb') as f:
25 valid = pickle.load(f)

/usr/lib/python2.7/pickle.pyc in load(file)
1376
1377 def load(file):
-> 1378 return Unpickler(file).load()
1379
1380 def loads(str):

/usr/lib/python2.7/pickle.pyc in load(self)
856 while 1:
857 key = read(1)
--> 858 dispatchkey
859 except _Stop, stopinst:
860 return stopinst.value

/usr/lib/python2.7/pickle.pyc in load_proto(self)
884 proto = ord(self.read(1))
885 if not 0 <= proto <= 2:
--> 886 raise ValueError, "unsupported pickle protocol: %d" % proto
887 dispatch[PROTO] = load_proto
888

ValueError: unsupported pickle protocol: 4

Hi, I am getting following error. I have changed the paths and tried to change the name of the folder, but still getting this error.

NotFoundError: Failed to create a directory: C:/python_projects/Capsule_Network/capsnet-traffic-sign-classifier-master/capsnet-traffic-sign-classifier-master/result\tensorboard\TrafficSign\c1s_9_c1s_5_c1n_256_lr_0.0001_c2n_64_rs_1_c1vl_16_c2s_6_c1nf_16_c2d_0.7_c2vl_32-train--TrafficSign--1524983554.079131; No such file or directory
Any help?

[Train] Batch ID = 0, loss = 4.22318, acc = 0.0
[Validation] Batch ID = 0, loss = 1.2876, acc = 0.02
Evaluate full validation dataset ...
Current loss: 1.4351 Best loss: None
[TOTAL Validation] Batch ID = 0, loss = 1.4351, acc = 0.037641723356

Thank you for sharing this repository, When I run the Jupiter notebook file I run into error which says Cannot import name Utils.

I ran my dataset on a capsule network. My image size is 224 * 224. I modified caps_ 2_ vec_ Len and upsample3= tf.image.resize_ nearest_ Neighbor (conv2, (224, 224)) and data_ handler, I'm a novice. I don't understand the code very well. Is there anything else in the _build_decoder decoder that needs to be modified?
`fc1 = tf.contrib.layers.fully_connected(capsule_vector, num_outputs=400)
fc1 = tf.reshape(fc1, shape=(batch_size, 5, 5, 16))
upsample1 = tf.image.resize_nearest_neighbor(fc1, (8, 8))
conv1 = tf.layers.conv2d(upsample1, 4, (3,3), padding='same', activation=tf.nn.relu)

    upsample2 = tf.image.resize_nearest_neighbor(conv1, (16, 16))
    conv2 = tf.layers.conv2d(upsample2, 8, (3,3), padding='same', activation=tf.nn.relu)

    upsample3 = tf.image.resize_nearest_neighbor(conv2, (224, 224))
    conv6 = tf.layers.conv2d(upsample3, 16, (3,3), padding='same', activation=tf.nn.relu)

    # 3 channel for RGG
    logits = tf.layers.conv2d(conv6, 3, (3,3), padding='same', activation=None)
    decoded = tf.nn.sigmoid(logits, name='decoded')
    tf.summary.image('reconstruction_img', decoded)

`

File "train.py", line 125, in
train(arguments[""], arguments["--ckpt"], arguments[""])
File "train.py", line 55, in train
X_train, y_train, X_valid, y_valid, X_test, y_test = get_data(dataset)
File "/home/unist/Documents/CapsNet/capsnet-traffic-sign-classifier/data_handler.py", line 23, in get_data
train = pickle.load(f)
File "/usr/lib/python3.4/codecs.py", line 319, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 0: invalid start byte

I have this problem. I am using python3.
Please help me to solve this problem or any comments about this one.
Thank you a lot

Hi I trained the model on colab. How can I restore the last saved checkpoints?

Anyone knows how to make own dataset (.p pickle) for this project? thanks.