recommended configuration for a smaller batch size setting about pmtd HOT 10 CLOSED

Hi @JingChaoLiu , could you share your implementation of SyncBN? I try to use torch.nn.SyncBatchNorm in Pytorch 1.1, but it crashes in our program.

I don't know what is the authors' implementation, but I implement it by using torch.nn.BatchNorm2d and torch.nn.BatchNorm1d in the model part and
if distributed:
sync_bn_model = torch.nn.SyncBatchNorm.convert_sync_batchnorm(model)
model = torch.nn.parallel.DistributedDataParrel(sync_bn_model, device_ids=[local_rank], output_device=local_rank, )
in train_net.py.
Remember to change:
if isinstance(module, torch.nn.modules.batchnorm._BatchNorm): [torch/nn/modules/batchnorm.py (line 495)]
to
if isinstance(module, torch.nn.modules.batchnorm.BatchNorm2d):
Otherwise, the model will crash.

from pmtd.

I got NAN under the setting batch_size=36, LR=0.04,

In our earliest settings for baseline, we use just 8 cards with batch_size=16 and base_learning_rate=16 * 0.00125= 0.02 for warming up 2 epoch and training 40 epoch(a shrinking scheduler mentioned in #2). Though the F-measure is just 60%+, it seems to converge smoothly. Maybe you need to check the labels?

even when I use 1*binary_cross_entropy loss

The loss weight of mask branch keeps unchanged until the loss type is changed to l1_loss for pyramid label. The loss weight of binary_cross_entropy probably should be kept as 1 (haven`t done the loss weight experiments for binary_cross_entropy).

I calculate the cropped text area via cv2.findContours(). Is it OK?

Do you means calculating the text box from the corresponding predicted mask during the inference stage ?Yes, for the baseline, the text box is calculated by the cv2.findContours, and the contour with the max area is selected to be wrapped by the cv2.minAreaRect to output the final text box.

from pmtd.

from pmtd.

do you think I should change syncBN to group BN?

We didn't perform the experiments for group normalization(GN) provided by maskrcnn-benchmark. It is worth to try the GN.

Will the final performance be affected by the setting of batch size?

In our experiments, 8 cards with base_learing_rate=0.01, 16 cards with base_learing_rate=0.02 and 32 cards with base_learing_rate=0.04 shows no significant difference (within 0.1%).

what is your setting for FPN_POST_NMS_TOP_N_TRAIN?

These settings are: MODEL.RPN.FPN_POST_NMS_TOP_N_TRAIN = 2000 and MODEL.RPN.FPN_POST_NMS_PER_BATCH = False

I use cv2.fillPoly() and cv2.findContours() to find the cropped text areas

During the training stage, what type do text areas exist as before the data augmentation, binary masks of bbox_h * bbox_w or polygonsof point_num * {x, y}?

• For the data augmentation in the baseline, since the mask is binary and the cropped mask is just the same part of the original mask, The method 1 of

  • building the original binary mask before cropping
  • then cropping the original mask to get the cropped mask,

  and the method 2 of

  • cropping the text area as a polygon
  • then using the cropped polygon to build a binary mask,

  are both ok.

• For the PMTD, since the pyramid label is built according to the the cropped polygon, the method 1 of

  • building the binary mask before cropping,
  • cropping the original mask to get the cropped mask,
  • find contours from the cropped mask to get the cropped polygon,
  • and building the pyramid label from the cropped polygon,

  may be lengthy. The method 2 of

  • cropping the text area as a polygon
  • then using the cropped polygon to build a pyramid mask,

  is strongly recommended.

from pmtd.

Hi @JingChaoLiu , does cropping the text area as a polygon exist API?

from pmtd.

Both the libraries of pyclipper and Polygon3 can do this. Reimplement the PolygonInstance.crop(link) may be a proper way to do this.

from pmtd.

@JingChaoLiu Thanks!

from pmtd.

@JingChaoLiu Many thanks for your implementation details.

from pmtd.

Hi @JingChaoLiu , could you share your implementation of SyncBN? I try to use torch.nn.SyncBatchNorm in Pytorch 1.1, but it crashes in our program.

from pmtd.

Hi @JingChaoLiu , could you share your implementation of SyncBN? I try to use torch.nn.SyncBatchNorm in Pytorch 1.1, but it crashes in our program.

I don't know what is the authors' implementation, but I implement it by using torch.nn.BatchNorm2d and torch.nn.BatchNorm1d in the model part and
if distributed:
sync_bn_model = torch.nn.SyncBatchNorm.convert_sync_batchnorm(model)
model = torch.nn.parallel.DistributedDataParrel(sync_bn_model, device_ids=[local_rank], output_device=local_rank, )
in train_net.py.
Remember to change:
if isinstance(module, torch.nn.modules.batchnorm._BatchNorm): [torch/nn/modules/batchnorm.py (line 495)]
to
if isinstance(module, torch.nn.modules.batchnorm.BatchNorm2d):
Otherwise, the model will crash.

Thanks, it's really helpful

from pmtd.