Corrupt image file during training about deeposlandia HOT 2 CLOSED

See this Keras issue for diagnosing the involved file(s).

from deeposlandia.

The preprocessing phase may introduce broken images.

By opening these images with small viewer like eog we get libpng error like IDAT: CRC error. By opening them in a ipython console with Pillow is fine, however converting the data as a numpy array raises the TypeError above. More weirdly, trying to convert it a second time works... Reading the file with OpenCV fails too.

Summary:

• first scenario

img_path = "crappy_file.png"
img = PIL.Image(img_path) # OK
data = np.asarray(img, dtype="float32") # KO (Keras way of getting image data)

• second scenario:

img_path = "crappy_file.png"
img = PIL.Image(img_path) # OK
data = np.asarray(img, dtype="float32") # KO
data = np.asarray(img, dtype="float32") # OK

• third scenario:

img_path = "crappy_file.png"
img = cv2.imread(img_path) # KO

As this bug arises very rarely (around 1 image over 100k, in my own tests), and seems quite hard to fix, I suggest for now to check the produced data as suggested in the Keras issue mentionned above, and to delete the involved image(s) by hand.

from deeposlandia.