anujshah1003 / own_data_cnn_implementation_keras Goto Github PK
View Code? Open in Web Editor NEWA complete tutorial on using own dataset to train a CNN from scratch in Keras (TF & Theano Backend)-
A complete tutorial on using own dataset to train a CNN from scratch in Keras (TF & Theano Backend)-
score = model.evaluate(X_test, y_test, show_accuracy=True, verbose=0)
Traceback (most recent call last):
File "", line 1, in
score = model.evaluate(X_test, y_test, show_accuracy=True, verbose=0)
TypeError: evaluate() got an unexpected keyword argument 'show_accuracy'
While executing the code i am getting the above error in the following code.:
model.fit_generator(
train_generator,
steps_per_epoch=nb_train_samples // batch_size,
epochs=epochs,
validation_data=validation_generator,
validation_steps=nb_validation_samples // batch_size)
error as:
ValueError Traceback (most recent call last)
in ()
27 epochs=epochs,
28 validation_data=validation_generator,
---> 29 validation_steps=nb_validation_samples // batch_size)
30
31 model.save_weights('first_try.h5')
~\AppData\Roaming\Python\Python36\site-packages\keras\legacy\interfaces.py in wrapper(*args, **kwargs)
86 warnings.warn('Update your ' + object_name + 87 '
call to the Keras 2 API: ' + signature, stacklevel=2)
---> 88 return func(*args, **kwargs)
89 wrapper._legacy_support_signature = inspect.getargspec(func)
90 return wrapper
~\AppData\Roaming\Python\Python36\site-packages\keras\models.py in fit_generator(self, generator, steps_per_epoch, epochs, verbose, callbacks, validation_data, validation_steps, class_weight, max_q_size, workers, pickle_safe, initial_epoch)
1122 workers=workers,
1123 pickle_safe=pickle_safe,
-> 1124 initial_epoch=initial_epoch)
1125
1126 @interfaces.legacy_generator_methods_support
~\AppData\Roaming\Python\Python36\site-packages\keras\legacy\interfaces.py in wrapper(*args, **kwargs)
86 warnings.warn('Update your ' + object_name + 87 '
call to the Keras 2 API: ' + signature, stacklevel=2)
---> 88 return func(*args, **kwargs)
89 wrapper._legacy_support_signature = inspect.getargspec(func)
90 return wrapper
~\AppData\Roaming\Python\Python36\site-packages\keras\engine\training.py in fit_generator(self, generator, steps_per_epoch, epochs, verbose, callbacks, validation_data, validation_steps, class_weight, max_q_size, workers, pickle_safe, initial_epoch)
1922 max_q_size=max_q_size,
1923 workers=workers,
-> 1924 pickle_safe=pickle_safe)
1925 else:
1926 # No need for try/except because
~\AppData\Roaming\Python\Python36\site-packages\keras\legacy\interfaces.py in wrapper(*args, **kwargs)
86 warnings.warn('Update your ' + object_name + 87 '
call to the Keras 2 API: ' + signature, stacklevel=2)
---> 88 return func(*args, **kwargs)
89 wrapper._legacy_support_signature = inspect.getargspec(func)
90 return wrapper
~\AppData\Roaming\Python\Python36\site-packages\keras\engine\training.py in evaluate_generator(self, generator, steps, max_q_size, workers, pickle_safe)
2008 '(x, y, sample_weight) '
2009 'or (x, y). Found: ' +
-> 2010 str(generator_output))
2011 if len(generator_output) == 2:
2012 x, y = generator_output
ValueError: output of generator should be a tuple (x, y, sample_weight) or (x, y). Found: None
My train accuracy is as high as 93%, test accuracy is not even 60%
Train on 646 samples, validate on 162 samples
Epoch 1/20
646/646 [==============================] - 3s 5ms/step - loss: 1.6475 - acc: 0.2771 - val_loss: 1.3762 - val_acc: 0.4012
Epoch 2/20
646/646 [==============================] - 1s 2ms/step - loss: 1.3441 - acc: 0.3746 - val_loss: 1.3336 - val_acc: 0.3951
Epoch 3/20
646/646 [==============================] - 1s 2ms/step - loss: 1.2240 - acc: 0.4628 - val_loss: 1.2016 - val_acc: 0.4506
Epoch 4/20
646/646 [==============================] - 1s 2ms/step - loss: 1.0834 - acc: 0.5325 - val_loss: 1.1964 - val_acc: 0.4012
Epoch 5/20
646/646 [==============================] - 1s 2ms/step - loss: 1.0213 - acc: 0.5666 - val_loss: 1.1028 - val_acc: 0.4321
Epoch 6/20
646/646 [==============================] - 1s 2ms/step - loss: 0.8154 - acc: 0.6625 - val_loss: 1.0602 - val_acc: 0.5864
Epoch 7/20
646/646 [==============================] - 1s 2ms/step - loss: 0.7496 - acc: 0.7198 - val_loss: 1.0646 - val_acc: 0.5370
Epoch 8/20
646/646 [==============================] - 1s 2ms/step - loss: 0.6410 - acc: 0.7554 - val_loss: 1.0732 - val_acc: 0.5123
Epoch 9/20
646/646 [==============================] - 1s 2ms/step - loss: 0.5153 - acc: 0.8204 - val_loss: 1.2143 - val_acc: 0.5617
Epoch 10/20
646/646 [==============================] - 1s 2ms/step - loss: 0.4199 - acc: 0.8452 - val_loss: 1.2830 - val_acc: 0.5617
Epoch 11/20
646/646 [==============================] - 1s 2ms/step - loss: 0.3557 - acc: 0.8607 - val_loss: 1.3736 - val_acc: 0.5679
Epoch 12/20
646/646 [==============================] - 1s 2ms/step - loss: 0.3054 - acc: 0.8947 - val_loss: 1.3489 - val_acc: 0.5617
Epoch 13/20
646/646 [==============================] - 1s 2ms/step - loss: 0.2247 - acc: 0.9257 - val_loss: 1.6261 - val_acc: 0.5864
Epoch 14/20
646/646 [==============================] - 1s 2ms/step - loss: 0.2627 - acc: 0.9164 - val_loss: 1.3588 - val_acc: 0.5864
Epoch 15/20
646/646 [==============================] - 1s 2ms/step - loss: 0.2434 - acc: 0.9133 - val_loss: 1.5136 - val_acc: 0.5864
Epoch 16/20
646/646 [==============================] - 1s 2ms/step - loss: 0.1817 - acc: 0.9427 - val_loss: 1.8481 - val_acc: 0.6235
Epoch 17/20
646/646 [==============================] - 1s 2ms/step - loss: 0.1826 - acc: 0.9427 - val_loss: 1.6952 - val_acc: 0.6049
Epoch 18/20
646/646 [==============================] - 1s 2ms/step - loss: 0.1354 - acc: 0.9567 - val_loss: 2.2561 - val_acc: 0.5926
Epoch 19/20
646/646 [==============================] - 1s 2ms/step - loss: 0.1587 - acc: 0.9365 - val_loss: 2.0255 - val_acc: 0.5864
Epoch 20/20
646/646 [==============================] - 1s 2ms/step - loss: 0.1892 - acc: 0.9396 - val_loss: 2.1586 - val_acc: 0.5432
Now test
('Test Loss:', 2.15857571731379)
('Test accuracy:', 0.5432098765432098)
How do implement this code on a GPU using tensorflow backend
Hello,
In read me file there is dead link to part 1 of tutorial.
I guess it should be: https://www.youtube.com/watch?v=u8BW_fl6WRc
Have a nice day.
Thanks for the video tutorial.
I think the way the labels are assigned is problematic. The "os.listdir(data_path + '/' + dataset)" doesn't seems to list the directories alphabetically in ascending order, if you print the list you will see the directories are read as ['cats', 'horses', 'dogs', 'humans']. Assigning the labels in ascending order (as in names = ['cats', 'dogs' 'horses', 'humans']) and matching them to the read data in the directories (as labels[0:202] =0 for cat and [202:404] = 2 for dogs then [202:404]=1 for dogs would actually contain horses) would likely to create mismatch between the assigned labels and the actual labels.
The way around this is to sort img_list before you read the images.
Hi Friend,
Can you help me to check this issue?
Traceback (most recent call last):
File "D:/projects-python/keras-sample/cnn/custom_data_cnn.py", line 217, in
plt.plot(xc,train_loss)
File "C:\Users\ccc\AppData\Local\Programs\Python\Python36\lib\site-packages\matplotlib\pyplot.py", line 3317, in plot
ret = ax.plot(*args, **kwargs)
File "C:\Users\ccc\AppData\Local\Programs\Python\Python36\lib\site-packages\matplotlib_init_.py", line 1898, in inner
return func(ax, *args, **kwargs)
File "C:\Users\ccc\AppData\Local\Programs\Python\Python36\lib\site-packages\matplotlib\axes_axes.py", line 1406, in plot
for line in self._get_lines(*args, **kwargs):
File "C:\Users\ccc\AppData\Local\Programs\Python\Python36\lib\site-packages\matplotlib\axes_base.py", line 407, in _grab_next_args
for seg in self._plot_args(remaining, kwargs):
File "C:\Users\ccc\AppData\Local\Programs\Python\Python36\lib\site-packages\matplotlib\axes_base.py", line 385, in _plot_args
x, y = self._xy_from_xy(x, y)
File "C:\Users\ccc\AppData\Local\Programs\Python\Python36\lib\site-packages\matplotlib\axes_base.py", line 244, in _xy_from_xy
"have shapes {} and {}".format(x.shape, y.shape))
ValueError: x and y must have same first dimension, but have shapes (20,) and (2,)
Hi
Thanks for your job
I notice that ligne 12 sklearn.cross_validation
is not anymore implemented
You have to use sklearn.model_selection
Regards
Kindly explain your answer. I did this but I am not sure if it's correct.
PATH = os.getcwd()
# Define data path
data_path2 = r'C:\Users\Admin\test'
data_dir_list_t = os.listdir(data_path2)
img_rows=64
img_cols=64
num_channel=3
num_epoch= 40
# Define the number of classes
num_classes = 2
img_data_list_t=[]
for dataset in data_dir_list_t:
img_list_t=os.listdir(data_path2 +'/'+ dataset)
print ('Loaded the images of dataset-'+'{}\n'.format(dataset))
for img in img_list_t:
input_img_t=cv2.imread(data_path2 + '/'+ dataset + '/'+ img )
#input_img=cv2.cvtColor(input_img, cv2.COLOR_BGR2GRAY)
input_img_t_resize=cv2.resize(input_img,(64,64))
img_data_list_t.append(input_img_t_resize)
img_data1 = np.array(img_data_list_t)
img_data1 = img_data1.astype('float32')
img_data1 /= 255
print (img_data1.shape)
if num_channel==1:
if K.image_dim_ordering()=='th':
img_data1= np.expand_dims(img_data1, axis=1)
print (img_data1.shape)
else:
img_data1= np.expand_dims(img_data1, axis=4)
print (img_data1.shape)
else:
if K.image_dim_ordering()=='th':
img_data1=np.rollaxis(img_data1,3,1)
print (img_data1.shape)
#%%
USE_SKLEARN_PREPROCESSING=False
if USE_SKLEARN_PREPROCESSING:
# using sklearn for preprocessing
from sklearn import preprocessing
def image_to_feature_vector_t(image, size=(64,64)):
# resize the image to a fixed size, then flatten the image into
# a list of raw pixel intensities
return cv2.resize(image, size).flatten()
img_data_list_t=[]
for dataset in data_dir_list:
img_list_t=os.listdir(data_path2 +'/'+ dataset)
print ('Loaded the images of dataset-'+'{}\n'.format(dataset))
for img in img_list_t:
input_img=cv2.imread(data_path2 + '/'+ dataset + '/'+ img )
#input_img=cv2.cvtColor(input_img, cv2.COLOR_BGR2GRAY)
input_img_flatten=image_to_feature_vector_t(input_img,(64,64))
img_data_list_t.append(input_img_flatten)
img_data1 = np.array(img_data_list_t)
img_data1 = img_data1.astype('float32')
print (img_data1.shape)
img_data1_scaled = preprocessing.scale(img_data1)
print (img_data1_scaled.shape)
print (np.mean(img_data1_scaled))
print (np.std(img_data1_scaled))
print (img_data1_scaled.mean(axis=0))
print (img_data1_scaled.std(axis=0))
if K.image_dim_ordering()=='th':
img_data1_scaled=img_data1_scaled.reshape(img_data1.shape[0],num_channel,img_rows,img_cols)
print (img_data1_scaled.shape)
else:
img_data1_scaled=img_data1_scaled.reshape(img_data1.shape[0],img_rows,img_cols,num_channel)
print (img_data1_scaled.shape)
if K.image_dim_ordering()=='th':
img_data1_scaled=img_data1_scaled.reshape(img_data1.shape[0],num_channel,img_rows,img_cols)
print (img_data1_scaled.shape)
else:
img_data1_scaled=img_data1_scaled.reshape(img_data1.shape[0],img_rows,img_cols,num_channel)
print (img_data1_scaled.shape)
if USE_SKLEARN_PREPROCESSING:
img_data1=img_data1_scaled
#%%
# Assigning Labels
# Define the number of classes
num_classes_t = 2
num_of_samples = img_data1.shape[0]
labels_t = np.ones((num_of_samples,),dtype='int64')
labels_t[0:40]=0
labels_t[40:]=1
#labels[404:606]=2
#labels[606:]=3
names = ['cats',' dogs']
# convert class labels to on-hot encoding
test_lbs = np_utils.to_categorical(labels_t, num_classes_t)
#Shuffle the dataset
x_test_t, y_test_t = shuffle(img_data1, test_lbs, random_state=2)
# Split the dataset
#X_train, X_test, y_train, y_test = train_test_split(x, y, test_size=0.2, random_state=2)
#%%
# Evaluating the model
score = model.evaluate(x_test_t, y_test_t, batch_size=10)
print('Test Loss:', score[0])
print('Test accuracy:', score[1])
test_image = X_test[0:1]
print (test_image.shape)```...
Re: custom_data_cnn.py
Anaconda2-5.1.0-MacOSX-x86_64.sh
Dear Anuj Shah:
Suspect ‘show_accuracy’ is not supported on this build, please enlighten.
Sincerely,
eltoneo
File "/Users/dutch/anaconda3/lib/python3.6/site-packages/spyder/utils/site/sitecustomize.py", line 102, in execfile
exec(compile(f.read(), filename, 'exec'), namespace)
File "/Users/dutch/anaconda3/SRC/custom_data_cnn.py", line 246, in
score = model.evaluate(X_test, y_test, show_accuracy=True, verbose=0)
TypeError: evaluate() got an unexpected keyword argument 'show_accuracy'
/Users/dutch/anaconda3/SRC/custom_data_cnn.py(246)()
244 # Evaluating the model 245 --> 246 score = model.evaluate(X_test, y_test, show_accuracy=True, verbose=0) 247 print('Test Loss:', score[0]) 248 print('Test accuracy:', score[1])
when runnig the following line (line 59 in the updated version of the code):
img_data= np.expand_dims(img_data, axis=4)
I get the following error:
AxisError: axis 4 is out of bounds for array of dimension 4
Is there somehting changed in the packages? Maybe the function works differently. It also throws the same error if I use it with the exact same data that you provide and apply your code 1:1....
Any help would be much appreciated!
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.