Is this right command:
y = model.predict_on_batch(x_train)
for feature extraction purpose?

@iwantooxxoox I would like to know about the dataset structure used to train the model nn4.small2.h5.
Can you please elaborate the label file used while training? how we will get the 128x1 vector ?

I'd like to know if the model also outputs face pose angles, or is it only detecting the area where it finds a face?

Thank you!

Thank you for publishing such a nice library!
But I am in a little bit trouble about h5 file.
I tried "nn4.small2.v1.h5", and using it I compared face images same as official demo.

As a result, the squared l2 distance between clapton1 and clapton2 is so small, 0.0360859, although two images are same persons.
That value between clapton-1 and lennon-2 is 0.0384319.
That value between clapton-2 and lennon-2 is 0.0224784.

On the other hand, the official demo values are here.

clapton-1 and clapton-2

• 0.318360479234912674

clapton-1 and lennon-2

• 1.447068150294569921

clapton-2 and lennon-2

• 1.520698983951225713

I think it may be wrong to write my code, how to calculate the distance.

Code to calculate the distance is here. Any problem?

from keras.models import load_model
from keras.utils import CustomObjectScope
import tensorflow as tf
from keras.preprocessing.image import load_img, img_to_array
import numpy as np
from matplotlib.pyplot import imshow

img_path = './images/'

#96 x 96 , already aligned ,cropped face
file1 = 'clapton-1_aligned.png'
file2 = 'clapton-2_aligned.png'
file3 = 'lennon-2_aligned.png'

with CustomObjectScope({'tf': tf}):
  model = load_model('./model/nn4.small2.v1.h5')

def face_vector(file):
    img = load_img(img_path+file, target_size=(96, 96))
    imshow(np.asarray(img))
    x = img_to_array(img)
    x = np.expand_dims(x, axis=0)

    p = model.predict(x)
    return p[0]

rep_clapton1 = face_vector(file1)
rep_clapton2 = face_vector(file2)

#clapton-1 and clapton-2
d1 = rep_clapton1 - rep_clapton2
distance1 = np.dot(d1, d1)
print distance1

All code is here
https://github.com/osmszk/Keras-OpenFace-test/blob/master/Keras-Openface-test.ipynb

How to know loss function of model above ???

I don't know it. I want to know it

Does anybody know ?
Please help me

Any idea why it takes nearly a second to generate embeddings? (1080Ti with 11GB DDR5X). Or am I doing something wrong? Let me know please. Thanks!

Hi,

First of all, thanks for going through all the hassle of converting the model to keras so I don't have to :)

I tried to do the keras to coreml conversion, but it fails with the following error:

55 : activation_14, <keras.layers.core.Activation object at 0x12fc5a350>
56 : concatenate_1, <keras.layers.merge.Concatenate object at 0x12fbee610>
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Library/Python/2.7/site-packages/coremltools/converters/keras/_keras_converter.py", line 504, in convert
    predicted_probabilities_output = predicted_probabilities_output)
  File "/Library/Python/2.7/site-packages/coremltools/converters/keras/_keras2_converter.py", line 278, in _convert
    converter_func(builder, layer, input_names, output_names, keras_layer)
  File "/Library/Python/2.7/site-packages/coremltools/converters/keras/_layers2.py", line 511, in convert_merge
    mode = _get_elementwise_name_from_keras_layer(keras_layer)
  File "/Library/Python/2.7/site-packages/coremltools/converters/keras/_layers2.py", line 79, in _get_elementwise_name_from_keras_layer
    raise ValueError('Only channel and sequence concatenation are supported.')
ValueError: Only channel and sequence concatenation are supported.

Do you have an idea what I could do to fix that? Or would it be possible to send me the converted coreml model directly?

After looking through the notebook again, it seems like you changed the keras / coremltools packge. Can you share those changes / include them in the source?

Additionally, how well does the core ml model work? Did it generate the correct embeddings for the lfw images?

Hello, could you tell me where i can find triplet_generator()?
from data import triplet_generator

triplet_generator() creates a generator that continuously returns

([a_batch, p_batch, n_batch], None) tuples where a_batch, p_batch

and n_batch are batches of anchor, positive and negative RGB images

each having a shape of (batch_size, 96, 96, 3).

generator = triplet_generator()

nn4_small2_train.compile(loss=None, optimizer='adam')
nn4_small2_train.fit_generator(generator, epochs=10, steps_per_epoch=100)

Please note that the current implementation of the generator only generates

random image data. The main goal of this code snippet is to demonstrate

the general setup for model training. In the following, we will anyway

use a pre-trained model so we don't need a generator here that operates

on real training data. I'll maybe provide a fully functional generator

later.

What is a your License?

Thank you for a nice project.

Hi

Thanks for the great work, right now your implementation uses the naive version of triplet loss ?

Have you tried the triplet semihard loss or the triplet loss with online mining?

Hi man, thanks so much for this awesome project. But can you explain or can you please show some code how was you generate the reps2.csv?

i want to build model from scratch on my colab notebook, i referred to the model on Keras-openface-convertion.ipynb but i got nan value after training, do you know what's the problem?

How can I train model with my own dataset (e.g webFace) from scratch?

Hi there,
On the OpenFace website it's said that they go through a "transform" step where they align some face features with an affine transform (+crop) before training/processing images...

So trying to use the coreml model available here, how should I align my images before processing them ?

Thanks

I am able to save the keras model (nn4.small2.v1.h5) using the command model.save('new_model.h5'). But when I tried to load the model back, I am getting error, the complete details are given below.

nn4_small2_pretrained = create_model()
nn4_small2_pretrained.load_weights('weights/nn4.small2.v1.h5')

nn4_small2_pretrained.save("new_model.h5")

del nn4_small2_pretrained

from keras.models import load_model as lm

model = lm("new_model.h5")
Traceback (most recent call last):

  File "<ipython-input-8-74cf4e3a50de>", line 1, in <module>
    model = lm("waste.h5")

  File "/data/repos/tfenv/lib/python3.6/site-packages/keras/models.py", line 270, in load_model
    model = model_from_config(model_config, custom_objects=custom_objects)

  File "/data/repos/tfenv/lib/python3.6/site-packages/keras/models.py", line 347, in model_from_config
    return layer_module.deserialize(config, custom_objects=custom_objects)

  File "/data/repos/tfenv/lib/python3.6/site-packages/keras/layers/__init__.py", line 55, in deserialize
    printable_module_name='layer')

  File "/data/repos/tfenv/lib/python3.6/site-packages/keras/utils/generic_utils.py", line 144, in deserialize_keras_object
    list(custom_objects.items())))

  File "/data/repos/tfenv/lib/python3.6/site-packages/keras/engine/topology.py", line 2535, in from_config
    process_node(layer, node_data)

  File "/data/repos/tfenv/lib/python3.6/site-packages/keras/engine/topology.py", line 2492, in process_node
    layer(input_tensors[0], **kwargs)

  File "/data/repos/tfenv/lib/python3.6/site-packages/keras/engine/topology.py", line 619, in __call__
    output = self.call(inputs, **kwargs)

  File "/data/repos/tfenv/lib/python3.6/site-packages/keras/layers/core.py", line 685, in call
    return self.function(inputs, **arguments)

  File "/data/Work/workspace/person_recognition/python/face-recognition/utils.py", line 36, in LRN2D
    return tf.nn.lrn(x, alpha=1e-4, beta=0.75)

NameError: name 'tf' is not defined

How can I load a model without error

For a personal project, I have image datasets with N exclusive categories,
and each image corresponds to one category. How do I train the neural network?
Also, do I have to resize or format the images a certain way to make this work?

Hello,

You mentioned that the pre-trained weights offer the accuracy score 0.938+-0.013. You converted the raw model from original OpenFace? Because none of the 4 shared models got this accuracy in the official site. If you trained the model from scratch you got this score on LFW data set?

Hi, Thanks for your work. I have used the model nn4.small2.lrn.h5 for extracting the features after re-running Keras-openface-convertion.ipynb.

I got worst accuracy while applied with minimum / euclidean distance classifier for one shot face authentication (one image per class) compared with dlib feature extractor.

Am I doing right ?
Does the model work for one shot learning or not ?

Thanks and regards.
Bhanuchander

I'm just attempting to load the model below like you have in README.md,
but am getting the error: " ValueError: bad marshal data (unknown type code)".

These are the version of keras and python that I'm using.
Keras: 2.1.1
Python 3.6.2

Do I need to use a different version of Keras/Python?

from keras.models import load_model
from keras.utils import CustomObjectScope
import tensorflow as tf
with CustomObjectScope({'tf': tf}):
    model = load_model('Keras-OpenFace-master/model/nn4.small2.v1.h5')

ValueError Traceback (most recent call last)
in ()
3 import tensorflow as tf
4 with CustomObjectScope({'tf': tf}):
----> 5 model = load_model('Keras-OpenFace-master/model/nn4.small2.v1.h5')

//anaconda/envs/tf/lib/python3.6/site-packages/keras/models.py in load_model(filepath, custom_objects, compile)
238 raise ValueError('No model found in config file.')
239 model_config = json.loads(model_config.decode('utf-8'))
--> 240 model = model_from_config(model_config, custom_objects=custom_objects)
241
242 # set weights

//anaconda/envs/tf/lib/python3.6/site-packages/keras/models.py in model_from_config(config, custom_objects)
312 'Maybe you meant to use '
313 'Sequential.from_config(config)?')
--> 314 return layer_module.deserialize(config, custom_objects=custom_objects)
315
316

//anaconda/envs/tf/lib/python3.6/site-packages/keras/layers/init.py in deserialize(config, custom_objects)
53 module_objects=globs,
54 custom_objects=custom_objects,
---> 55 printable_module_name='layer')

//anaconda/envs/tf/lib/python3.6/site-packages/keras/utils/generic_utils.py in deserialize_keras_object(identifier, module_objects, custom_objects, printable_module_name)
137 return cls.from_config(config['config'],
138 custom_objects=dict(list(_GLOBAL_CUSTOM_OBJECTS.items()) +
--> 139 list(custom_objects.items())))
140 with CustomObjectScope(custom_objects):
141 return cls.from_config(config['config'])

//anaconda/envs/tf/lib/python3.6/site-packages/keras/engine/topology.py in from_config(cls, config, custom_objects)
2488 # First, we create all layers and enqueue nodes to be processed
2489 for layer_data in config['layers']:
-> 2490 process_layer(layer_data)
2491 # Then we process nodes in order of layer depth.
2492 # Nodes that cannot yet be processed (if the inbound node

//anaconda/envs/tf/lib/python3.6/site-packages/keras/engine/topology.py in process_layer(layer_data)
2474
2475 layer = deserialize_layer(layer_data,
-> 2476 custom_objects=custom_objects)
2477 created_layers[layer_name] = layer
2478

//anaconda/envs/tf/lib/python3.6/site-packages/keras/layers/init.py in deserialize(config, custom_objects)
53 module_objects=globs,
54 custom_objects=custom_objects,
---> 55 printable_module_name='layer')

//anaconda/envs/tf/lib/python3.6/site-packages/keras/utils/generic_utils.py in deserialize_keras_object(identifier, module_objects, custom_objects, printable_module_name)
137 return cls.from_config(config['config'],
138 custom_objects=dict(list(_GLOBAL_CUSTOM_OBJECTS.items()) +
--> 139 list(custom_objects.items())))
140 with CustomObjectScope(custom_objects):
141 return cls.from_config(config['config'])

//anaconda/envs/tf/lib/python3.6/site-packages/keras/layers/core.py in from_config(cls, config, custom_objects)
697 elif function_type == 'lambda':
698 # Unsafe deserialization from bytecode
--> 699 function = func_load(config['function'], globs=globs)
700 else:
701 raise TypeError('Unknown function type:', function_type)

//anaconda/envs/tf/lib/python3.6/site-packages/keras/utils/generic_utils.py in func_load(code, defaults, closure, globs)
198 if isinstance(defaults, list):
199 defaults = tuple(defaults)
--> 200 code = marshal.loads(code.encode('raw_unicode_escape'))
201 if globs is None:
202 globs = globals()

ValueError: bad marshal data (unknown type code)