tensorfreitas / temporal-ensembling-for-semi-supervised-learning Goto Github PK
View Code? Open in Web Editor NEWImplementation of Temporal Ensembling for Semi-Supervised Learning by Laine et al. with tensorflow eager execution
Implementation of Temporal Ensembling for Semi-Supervised Learning by Laine et al. with tensorflow eager execution
I changed the code to make it run on my data. My images are also 32x32x3 patches. There are 18000+ labelled samples. I am using a GPU with 8GB memory. I set the batch size to 128. When training starts, everything looks good, the GPU utilization is about 50%, but as the training going on, the GPU utilization decreases slowly. After 100+ epoches, the training becomes very slow and the GPU utilization is about 1% to 5%. Does anyone have this problem? Thank you!
Hi,
First of all, I'd like to thank you tremendously for sharing this clean TensorFlow implementation - it saved me a lot of time!
Second, I am wondering if by any chance you have compared your pi-model results with the original paper. I tried training the pi-model on SVHN using the same hyperparameters reported with 1000 labels, but for some reason the model overfits around the 50th-60th epoch, and the final test accuracy of the best ckpt is only 85%, versus almost 95% in the paper.
then i find that the training time of this code
current_outputs, loss_val, grads = temporal_ensembling_gradients(X_labeled_train, y_labeled_train, X_unlabeled_train,model, unsupervised_weight, current_ensemble_targets)
become more and more long after every epoch
anyone know how to deal with this problem. thanks
How much effort would it be to convert this to handle row vectors instead of images?
thanks
hi,
when l use the real ratio: labeled_batch_fraction = num_labeled_samples / num_train_unlabeled_sample
(it should be num_labeled_samples / (num_train_unlabeled_sample+num_labeled_samples?)), after training about more than 50 epochs, l find that the both train and val acc is always between 9%-10%. it means that the model doesn't work at all. do you have the same question? besides, train many times your code, the best test acc is only 89% far away from the paper. is it same with you?
thank you.
Whenever I try to execute train_pi_model.py or train_temporal_model.py I get this same error. Can someone help me diagnose the problem?
(tf-env) C:\Users\crm0376\Documents\ONR\Temporal-Ensembling-for-Semi-Supervised-Learning-master>C:/Users/crm0376/.local/conda_root/envs/tf-env/python.exe c:/Users/crm0376/Documents/ONR/Temporal-Ensembling-for-Semi-Supervised-Learning-master/train_temporal_ensembling_model.py 2018-11-08 17:23:21.615533: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 Traceback (most recent call last): File "c:/Users/crm0376/Documents/ONR/Temporal-Ensembling-for-Semi-Supervised-Learning-master/train_temporal_ensembling_model.py", line 199, in <module> main() File "c:/Users/crm0376/Documents/ONR/Temporal-Ensembling-for-Semi-Supervised-Learning-master/train_temporal_ensembling_model.py", line 111, in main model, unsupervised_weight, current_ensemble_targets) File "c:\Users\crm0376\Documents\ONR\Temporal-Ensembling-for-Semi-Supervised-Learning-master\pi_model.py", line 54, in temporal_ensembling_gradients model, unsupervised_weight, ensembling_targets) File "c:\Users\crm0376\Documents\ONR\Temporal-Ensembling-for-Semi-Supervised-Learning-master\pi_model.py", line 25, in temporal_ensembling_loss z_labeled = model(X_train_labeled) File "C:\Users\crm0376\.local\conda_root\envs\tf-env\lib\site-packages\tensorflow\python\keras\engine\base_layer.py", line 757, in __call__ outputs = self.call(inputs, *args, **kwargs) File "c:\Users\crm0376\Documents\ONR\Temporal-Ensembling-for-Semi-Supervised-Learning-master\pi_model.py", line 264, in call h = self.__apply_image_augmentation(h) File "c:\Users\crm0376\Documents\ONR\Temporal-Ensembling-for-Semi-Supervised-Learning-master\pi_model.py", line 248, in __apply_image_augmentation image, random_translation.numpy, 'NEAREST') File "C:\Users\crm0376\.local\conda_root\envs\tf-env\lib\site-packages\tensorflow\contrib\image\python\ops\image_ops.py", line 125, in translate translations_to_projective_transforms(translations), File "C:\Users\crm0376\.local\conda_root\envs\tf-env\lib\site-packages\tensorflow\contrib\image\python\ops\image_ops.py", line 191, in translations_to_projective_transforms translations, name="translations", dtype=dtypes.float32) File "C:\Users\crm0376\.local\conda_root\envs\tf-env\lib\site-packages\tensorflow\python\framework\ops.py", line 1050, in convert_to_tensor as_ref=False) File "C:\Users\crm0376\.local\conda_root\envs\tf-env\lib\site-packages\tensorflow\python\framework\ops.py", line 1146, in internal_convert_to_tensor ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref) File "C:\Users\crm0376\.local\conda_root\envs\tf-env\lib\site-packages\tensorflow\python\framework\constant_op.py", line 229, in _constant_tensor_conversion_function return constant(v, dtype=dtype, name=name) File "C:\Users\crm0376\.local\conda_root\envs\tf-env\lib\site-packages\tensorflow\python\framework\constant_op.py", line 179, in constant t = convert_to_eager_tensor(value, ctx, dtype) File "C:\Users\crm0376\.local\conda_root\envs\tf-env\lib\site-packages\tensorflow\python\framework\constant_op.py", line 114, in convert_to_eager_tensor return ops.EagerTensor(value, context=handle, device=device, dtype=dtype) ValueError: Attempt to convert a value (<bound method _EagerTensorBase.numpy of <tf.Tensor: id=233, shape=(1, 2), dtype=float32, numpy=array([[-0.76205397, 0.42147446]], dtype=float32)>>) with an unsupported type (<class 'method'>) to a Tensor.
tf.losses.softmax_cross_entropy is softmax(output) and then cross_entropy, while the model's last layer--dense also has the softmax activation. It means that the final loss will softmax twice. is it wrong?
Hi,
Thanks for your code.
In your implement, the ensemble targets are network output before softmax. However, I think the target is the ensemble of output after softmax. Since in table 5 in the paper, output is the softmax.
I'm sorry,..., but your codes seem to have several bugs, and I think it's why it can't reach the accuracy that the paper Temporal Ensembling... reported.
Args: translations: A 2-element list representing [dx, dy] or a matrix of 2-element lists representing [dx, dy] to translate for each image (for a batch of images). The rank must be statically known (the shape is not TensorShape(None). name: The name of the op.
Much appreciated for your succinct implementation. I'm trying to reproduce the result using your code. However, when I try to produce the baseline result, the supervised version with 1000 labels, using this code, it gives rather poor result, ~30%. Did you validate this by yourself? Or am I doing something stupid?
Thank you again.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.