This is a detailed description of the dataset, a datasheet for the dataset as proposed by Gebru et al.

Embrapa WGISD (Wine Grape Instance Segmentation Dataset) was created to provide images and annotation to study object detection and instance segmentation for image-based monitoring and field robotics in viticulture. It provides instances from five different grape varieties taken on field. These instances shows variance in grape pose, illumination and focus, including genetic and phenological variations such as shape, color and compactness.

Possible uses include relaxations of the instance segmentation problem: classification (Is a grape in the image?), semantic segmentation (What are the "grape pixels" in the image?), and object detection (Where are the grapes in the image?). The WGISD can also be used in grape variety identification.

The building of the WGISD dataset was supported by the Embrapa SEG Project 01.14.09.001.05.04, Image-based metrology for Precision Agriculture and Phenotyping, and the CNPq PIBIC Program (grants 161165/2017-6 and 125044/2018-6).

Each instance consists in a RGB image and an annotation describing grape clusters locations as bounding boxes. A subset of the instances also contains binary masks identifying the pixels belonging to each grape cluster. Each image presents at least one grape cluster. Some grape clusters can appear far at the background and should be ignored.

File names prefixes identify the variety observed in the instance.

The dataset consists of 300 images containing 4,432 grape clusters identified by bounding boxes. A subset of 137 images also contains binary masks identifying the pixels of each cluster. It means that from the 4,432 clusters, 2,020 of them presents binary masks for instance segmentation, as summarized in Table [table:GenInfoData].

General information about the dataset: the grape varieties and the associated identifying prefix, the date of image capture on field, number of images (instances) and the identified grapes clusters.

Each instance contains a 8-bits RGB image and a text file containing one bounding box description per line. These text files follows the "YOLO format"

CLASS CX CY W H

class is an integer defining the object class – the dataset presents only the grape class that is numbered 0, so every line starts with this “class zero” indicator. The center of the bounding box is the point (c_x, c_y), represented as float values because this format normalizes the coordinates by the image dimensions. To get the absolute position, use (2048 c_x, 1365 c_y). The bounding box dimensions are given by W and H, also normalized by the image size.

The instances presenting mask data for instance segmentation contain files presenting the .npz extension. These files are compressed archives for NumPy $n$-dimensional arrays. Each array is a H X W X n_clusters three-dimensional array where n_clusters is the number of grape clusters observed in the image. After assigning the NumPy array to a variable M, the mask for the i-th grape cluster can be found in M[:,:,i]. The i-th mask corresponds to the i-th line in the bounding boxes file.

The dataset also includes the original image files, presenting the full original resolution. The normalized annotation for bounding boxes allows easy identification of clusters in the original images, but the mask data will need to be properly rescaled if users wish to work on the original full resolution.

Everything is included in the dataset.

The dataset comes with specified train/test splits. The splits are found in lists stored as text files. There are also lists referring only to instances presenting binary masks.

Dataset recommended split.

Standard measures from the information retrieval and computer vision literature should be employed: precision and recall, F1-score and average precision as seen in COCO and Pascal VOC.

The first experiments run on this dataset are described in Grape detection, segmentation and tracking using deep neural networks and three-dimensional association by Santos et al.. See also the following video demo:

Images were captured at the vineyards of Guaspari Winery, located at Espírito Santo do Pinhal, São Paulo, Brazil (Lat -22.181018, Lon -46.741618). The winery staff performs dual pruning: one for shaping (after previous year harvest) and one for production, resulting in canopies of lower density. The image capturing was realized in April 2017 for Syrah and in April 2018 for the other varieties (see Table [table:GenInfoData]).

A Canon EOS REBEL T3i DSLR camera and a Motorola Z2 Play smartphone were used to capture the images. The cameras were located between the vines lines, facing the vines at distances around 1-2 meters. The EOS REBEL T3i camera captured 240 images, including all Syrah pictures. The Z2 smartphone grabbed 60 images covering all varieties except Syrah . The REBEL images were scaled to 2048 X 1365 pixels and the Z2 images to 2048 X 1536 pixels. More data about the capture process can be found in the Exif data found in the original image files, included in the dataset.

The authors of this paper. T. T. Santos, A. A. Santos and S. Avila captured the images in field. T. T. Santos, L. L. de Souza and S. Avila performed the annotation.

The rectangular bounding boxes identifying the grape clusters were annotated using the labelImg tool. The clusters can be under severe occlusion by leaves, trunks or other clusters. Considering the absence of 3-D data and on-site annotation, the clusters locations had to be defined using only a single-view image, so some clusters could be incorrectly delimited.

A subset of the bounding boxes was selected for mask annotation, using a novel tool developed by the authors and presented in this work. This interactive tool lets the annotator mark grape and background pixels using scribbles, and a graph matching algorithm developed by Noma et al. is employed to perform image segmentation to every pixel in the bounding box, producing a binary mask representing grape/background classification.

The following steps were taken to process the data:

1. Bounding boxes were annotated for each image using the labelImg tool.

2. Images were resized to W = 2048 pixels. This resolution proved to be practical to mask annotation, a convenient balance between grape detail and time spent by the graph-based segmentation algorithm.

3. A randomly selected subset of images were employed on mask annotation using the interactive tool based on graph matching.

4. All binaries masks were inspected, in search of pixels attributed to more than one grape cluster. The annotator assigned the disputed pixels to the most likely cluster.

5. The bounding boxes were fitted to the masks, which provided a fine tuning of grape clusters locations.

The original resolution images, containing the Exif data provided by the cameras, is available in the dataset.

The dataset is available at GitHub.

The dataset was released in July, 2019.

The data is released under Creative Commons BY-NC 4.0 (Attribution-NonCommercial 4.0 International license). There is a request to cite the corresponding paper if the dataset is used. For commercial use, contact Embrapa Agricultural Informatics business office.

There are no fees or restrictions. For commercial use, contact Embrapa Agricultural Informatics business office.

The dataset is hosted at Embrapa Agricultural Informatics and all comments or requests can be sent to Thiago T. Santos (maintainer).

There is no scheduled updates. In case of further updates, releases will be properly tagged at GitHub.

Contributors should contact the maintainer by e-mail.

The maintainers and their institutions are exempt from any liability, judicial or extrajudicial, for any losses or damages arising from the use of the data contained in the image database.