endpoint to generate bitmaps for mnist or cifar10 on the fly about dataset-viewer HOT 5 CLOSED

There are indeed very few instances of raw image data in datasets - usually it's better to keep them as encoded in jpeg/png files to save disk space. The common way to have image/audio data is to simply have the path to the image/audio file

from dataset-viewer.

I agree: we can provide JPEG or PNG files from the endpoint, which are then linkable, cacheable, easier to manage on the frontend side. We will have to manage a special type for them: see #25 (maybe return both the raw values + the image URL)

re audio: from https://observablehq.com/@huggingface/types-of-the-datasets-columns, I understand that all the audio datasets have columns with a path to the audio file. Tensors are just used for images (Array2D for mnist: black and white, and Array3D for cifar: color)

from dataset-viewer.

After chatting with @lhoestq, I think that for all the types of images (see https://github.com/huggingface/moon-landing/pull/1040), I will provide the image inside the /rows endpoint response as:

image: {
  data: base64
  type: str (mimetype)
  filename: str (optional)
}

This would allow showing the images with https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/Data_URIs. And to allow downloading them as files with https://developer.mozilla.org/en-US/docs/Web/API/File/File.

This would help to avoid having to maintain a complex backend (have a cache of all the images, manage the uniqueness of the image URLs, have an internet-facing API, etc)

In some cases, the data will come directly from the datasets library, in other cases like mnist or cifar10, I will have to generate an image (one of https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/MIME_types#image_types) then provide the bytes.

Same idea for the audio files.

Note: https://github.com/ahupp/python-magic might be used to detect the mimetype

from dataset-viewer.

Finally: I store the data in a file, then serve the file as a static file, and I give its URL in the data. For example, see:

• the JSON: https://datasets-preview.huggingface.tech/rows?dataset=mnist
• the image asset: https://datasets-preview.huggingface.tech/assets/mnist/___/mnist/train/0/image/image.jpg

from dataset-viewer.

The support for image datasets is now followed here: #63

For audio: #70

from dataset-viewer.