Code Monkey home page Code Monkey logo

jialecao001 / sipmask Goto Github PK

View Code? Open in Web Editor NEW
334.0 334.0 53.0 22.79 MB

SipMask: Spatial Information Preservation for Fast Image and Video Instance Segmentation (ECCV2020)

Home Page: https://arxiv.org/pdf/2007.14772.pdf

License: MIT License

Python 39.93% Jupyter Notebook 53.55% Dockerfile 0.05% C++ 2.10% Cuda 4.33% Shell 0.03% Makefile 0.01% Batchfile 0.01%
detection image-instance-segmentation multi-object-tracking-segmentation object-detection segmentation tracking video-instance-segmentation

sipmask's People

Contributors

jialecao001 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

sipmask's Issues

SipMask-VIS tracking implementation

Hi,

In your paper you say that you match instances across frames similar to MaskTrack R-CNN. Where in your code do you perform this matching? and where can I adjust the parameters such as how many frames in the past to consider (beta in the MOTS paper) and the distance threshold (delta in the MOTS paper)?

Low score in codalab with released codes.

Hello,
Thank you for your interesting work and nice release of codes.
I am trying to reproduce your results about VIS dataset.
I successfully executed your codes and got the results.json file.
However, when I upload the file to Codalab server, I got a very low score (0.2% AP).
I can't determinate whether it's my environment problem (because there is no error alarm) or it's the problem of Codalab.
Would you mind maybe provide your json file which achieves good score in codalab? Or maybe you have some better advices for me to diagnose my issue.

Attached please find my results.json
results_vis_sipmask_ms_1x_final.zip

how to decrease the number of threads

hi there!

i was able to successfully run sipmask on a jetson nano. however, after the demo generates the results, it prints "segmentation fault (core dumped)".
i did the process once again and by running htop i noticed that when i run the demo, i have 8 different PIDs running the command. i'm thinking this might be what's causing the segmentantion fault ?
do you have any idea/suggestion on how to tackle this problem?

thanks in advance.

'SipMask is not in the detector registry'

greetings, thanks for sharing your work!
i'm trying to run the demo on an ec2:

python3 ./demo/sipmask_demo.py ./configs/sipmask/sipmask_r50_caffe_fpn_gn_1x.py ./sipmask_r50_caffe_1x.pth ./demo/demo.jpg --out ./demo/aa.jpg

but i keep getting the following error:

File "/home/ubuntu/.local/lib/python3.6/site-packages/mmcv/utils/registry.py", line 164, in build_from_cfg f'{obj_type} is not in the {registry.name} registry') KeyError: 'SipMask is not in the detector registry'

i followed all the steps on the installation of mmcv and mmdetection, and then cloned the sipmask repo. do you know what might be causing this?
thanks in advance!

training failed to converge

hello,

I've just tried training on my own dataset 4 classes , 24 images.
but after 90k iterations loss has not progressed passed 1.56:
iter: 90000 loss: 1.5658 (1.5819) loss_cls: 0.0000 (0.0114) loss_reg: 0.9772 (0.9775) loss_centerness: 0.5881 (0.5927) loss_mask: 0.0000 (0.0003) time: 0.3531 (0.3745) data: 0.0145 (0.0164) lr: 0.000010 max mem: 2017

Evaluate annotation type bbox
DONE (t=0.94s).
Accumulating evaluation results...
DONE (t=0.03s).
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.000
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.000
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.000
Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.000
Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.000
Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.000
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.000
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.000
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.000
Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.000
Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.000
Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.000
(test dataset matches train dataset)

there seems to have been very little progress since iter 5000
iter: 5000 loss: 1.5704 (1.8018) loss_cls: 0.0025 (0.2015) loss_reg: 0.9779 (0.9775) loss_centerness: 0.5880 (0.6171) loss_mask: 0.0000 (0.0057) time: 0.3798 (0.3738) data: 0.0161 (0.0164) lr: 0.001000 max mem: 2017

I'm using a modified sipmask_R_50_FPN_1x.yaml configuration:

FCOS with improvements

MODEL:
META_ARCHITECTURE: "GeneralizedRCNN"
RPN_ONLY: True
SIPMASK_ON: True
BACKBONE:
CONV_BODY: "R-50-FPN-RETINANET"
RESNETS:
BACKBONE_OUT_CHANNELS: 256
RETINANET:
USE_C5: False # FCOS uses P5 instead of C5x
NUM_CLASSES : 4
SIPMASK:
# normalizing the regression targets with FPN strides
NORM_REG_TARGETS: True
# positioning centerness on the regress branch.
# Please refer to tianzhi0549/FCOS#89 (comment)
CENTERNESS_ON_REG: True
# using center sampling and GIoU.
# Please refer to https://github.com/yqyao/FCOS_PLUS
CENTER_SAMPLING_RADIUS: 1.5
IOU_LOSS_TYPE: "giou"
NUM_CLASSES : 4
ROI_KEYPOINT_HEAD:
NUM_CLASSES : 4
ROI_KEYPOINT_HEAD:
NUM_CLASSES : 4
FCOS:
NUM_CLASSES : 4

DATASETS:
TRAIN: ("ffr2coco",)
TEST: ("ffr2coco",)
INPUT:
MIN_SIZE_TRAIN: (720,)
MAX_SIZE_TRAIN: 1280
MIN_SIZE_TEST: 720
MAX_SIZE_TEST: 1280
PIXEL_MEAN : [103, 103, 103]
PIXEL_STD: [66.0, 66.0, 66.0]
DATALOADER:
SIZE_DIVISIBILITY: 32
SOLVER:
BASE_LR: 0.001
WEIGHT_DECAY: 0.0001
STEPS: (60000, 80000)
MAX_ITER: 90000
IMS_PER_BATCH: 1
WARMUP_METHOD: "constant"
WARMUP_ITERS : 0
CHECKPOINT_PERIOD : 1000
TEST:
IMS_PER_BATCH : 1

any ideas ?

Omer.

About the evaluation in the codalab

Hi, when I try to upload the json file to the codalab for vis, the website gives a piece of error information "Invalid file type (application/json)." Is it normal?

number of GPUS

May I ask how many gpu cards you have used to train sipmask-mmdetection. Because I found that in mmdetection, we need to change the learning rate if number of GPU cards are different

SipMask-VIS custom dataset

I'm trying to use SipMask-VIS on a custom dataset. Training seems to go fine, however when testing I get really poor results. I don't really understand why. I have trained SipMask-mmdetection with similar config on my custom data with great results and I load from this checkpoint when I train SipMask-VIS.

Training results

![Figure_1](https://user-images.githubusercontent.com/33979503/111085003-9412ce80-8515-11eb-87c3-d37a92f980f3.png)

SipMask-VIS config

model = dict( type='SipMask', pretrained='open-mmlab://resnet50_caffe', backbone=dict( with_cp=True, type='ResNet', depth=50, num_stages=4, out_indices=(0, 1, 2, 3), frozen_stages=1, norm_cfg=dict(type='BN', requires_grad=False), style='caffe'), neck=dict( type='FPN', in_channels=[256, 512, 1024, 2048], out_channels=256, start_level=1, add_extra_convs=True, extra_convs_on_inputs=False, # use P5 num_outs=5, relu_before_extra_convs=True), bbox_head=dict( type='SipMaskHead', num_classes=2, in_channels=256, stacked_convs=3, feat_channels=256, strides=[8, 16, 32, 64, 128], loss_cls=dict( type='FocalLoss', use_sigmoid=True, gamma=2.0, alpha=0.25, loss_weight=1.0), loss_bbox=dict(type='IoULoss', loss_weight=1.0), loss_centerness=dict( type='CrossEntropyLoss', use_sigmoid=True, loss_weight=1.0), center_sampling=True, center_sample_radius=1.5)) # training and testing settings train_cfg = dict( assigner=dict( type='MaxIoUAssigner', pos_iou_thr=0.5, neg_iou_thr=0.4, min_pos_iou=0, ignore_iof_thr=-1), allowed_border=-1, pos_weight=-1, debug=False) test_cfg = dict( nms_pre=200, min_bbox_size=0, score_thr=0.03, nms=dict(type='nms', iou_thr=0.5), max_per_img=10) # dataset settings dataset_type = 'SausagesDataset' data_root = 'data_ytvos/' #classes = ('sausage',) img_norm_cfg = dict( mean=[102.9801, 115.9465, 122.7717], std=[1.0, 1.0, 1.0], to_rgb=False) data = dict( imgs_per_gpu=4, workers_per_gpu=1, train=dict( type=dataset_type, ann_file=data_root + 'annotations/train.json', img_prefix=data_root + 'train', img_scale=(640, 340), #classes=classes, img_norm_cfg=img_norm_cfg, size_divisor=32, flip_ratio=0.5, with_mask=True, with_crowd=True, with_label=True, with_track=True), val=dict( type=dataset_type, ann_file=data_root + 'annotations/val.json', img_prefix=data_root + 'val', img_scale=(640, 340), #classes=classes, img_norm_cfg=img_norm_cfg, size_divisor=32, flip_ratio=0, with_mask=True, with_crowd=True, with_label=True), test=dict( type=dataset_type, ann_file=data_root + 'annotations/test.json', img_prefix=data_root + 'test', img_scale=(640, 340), #classes=classes, img_norm_cfg=img_norm_cfg, size_divisor=32, flip_ratio=0, with_mask=False, with_label=False, test_mode=True))

optimizer

optimizer = dict(
type='SGD',
lr=0.005,
momentum=0.9,
weight_decay=0.0001,
paramwise_options=dict(bias_lr_mult=2., bias_decay_mult=0.))
optimizer_config = dict(grad_clip=None)

learning policy

lr_config = dict(
policy='step',
warmup='linear',
warmup_iters=1000,
warmup_ratio=1.0 / 80,
step=[8, 11])
checkpoint_config = dict(interval=1)

yapf:disable

log_config = dict(
interval=50,
hooks=[
dict(type='TextLoggerHook'),
# dict(type='TensorboardLoggerHook')
])

yapf:enable

runtime settings

total_epochs = 12
device_ids = range(1)
dist_params = dict(backend='nccl')
log_level = 'INFO'
work_dir = './work_dirs/sipmask_r50_fpn_1x'
load_from = './../SipMask-mmdetection/work_dirs/sipmask_r50_caffe_fpn_gn_1x_sausages/latest.pth'
resume_from = None
workflow = [('train', 1)]

Test result

Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = -1.000 Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = -1.000 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.000 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.000 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.000 Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.000 Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = -1.000 Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = -1.000

Any tip or help on what could be happening would be much appreciated!

Low FPS Score

@JialeCao001 Thanks for such a nice work. I am running the

python3 demo/webcam_demo.py configs/sipmask/sipmask_r50_caffe_fpn_ssd_gn_10x.py sipmask_r50_caffe_ssd_gn_10x.pth

I have follow hardware specs
-> I have Tesla K80 (12 GB Ram) with cuda 10.2
-> I am using sipmask_r50_caffe_ssd_gn_10x.pth
It giving me 6-9 fps , can you guide me what is the problem or why it is giving low fps. Thanks

Where is the config file sipmask_r50_caffe_fpn_gn_1x_4gpu.py of sipmask-vis?

Hello, thanks for your good work. I cannot find the config file sipmask_r50_caffe_fpn_gn_1x_4gpu.py in the vis project. There is just the configs/sipmask/sipmask_r50_caffe_fpn_gn_1x.py, and I have train with it, but got near 28 mAP in the evaluation website. So I want to ask what should I do more to get higher mAP. Thanks.

What version of dependencies are used?

Hey,

I have tried to run SipMask-VIS, but get
ImportError: cannot import name 'get_dist_info' from 'mmcv.runner.utils'
and upon checking mmcv, it seems like get_dist_info has been deprecated for quite some time. How would I most easily run your code? I will supply the docker file I used below.

ARG PYTORCH="1.4"
ARG CUDA="10.1"
ARG CUDNN="7"

FROM pytorch/pytorch:${PYTORCH}-cuda${CUDA}-cudnn${CUDNN}-devel

RUN apt-get update && apt-get install -y ffmpeg libglib2.0-0 libsm6 libxrender-dev libxext6 gcc git

# Install mmdetection
RUN conda install cython -y
RUN pip install mmcv-full==latest+torch1.4.0+cu101 -f https://openmmlab.oss-accelerate.aliyuncs.com/mmcv/dist/index.\
html
RUN git clone https://github.com/open-mmlab/mmdetection.git /mmdetection
WORKDIR /mmdetection
RUN pip install -e .
RUN python setup.py develop
RUN git clone https://github.com/youtubevos/cocoapi.git /cocoapi
WORKDIR /cocoapi/PythonAPI
RUN pip uninstall -y pycocotools
RUN python setup.py build_ext install
RUN git clone https://github.com/JialeCao001/SipMask.git /SipMask
WORKDIR /SipMask/SipMask-VIS
ENV TORCH_CUDA_ARCH_LIST="3.5;3.7;5.0;5.2;6.0+PTX;6.1+PTX;7.0+PTX;7.5+PTX"
RUN pip install -e .

Best regards,
Joakim

train on a custom dataset

hi! thanks for sharing your work :)

i am trying to train using a custom dataset, with only 1 class. i keep getting the following error, i believe it's because the number of classes is different from coco:

2020-11-10 18:53:53,782 - mmdet - INFO - load model from: open-mmlab://resnet50_caffe
2020-11-10 18:53:53,868 - mmdet - WARNING - The model and loaded state dict do not match exactly

unexpected key in source state_dict: conv1.bias, layer1.0.conv1.bias, layer1.0.conv2.bias, layer1.0.conv3.bias, layer1.0.downsample.0.bias, layer1.1.conv1.bias, layer1.1.conv2.bias, layer1.1.conv3.bias, layer1.2.conv1.bias, layer1.2.conv2.bias, layer1.2.conv3.bias, layer2.0.conv1.bias, layer2.0.conv2.bias, layer2.0.conv3.bias, layer2.0.downsample.0.bias, layer2.1.conv1.bias, layer2.1.conv2.bias, layer2.1.conv3.bias, layer2.2.conv1.bias, layer2.2.conv2.bias, layer2.2.conv3.bias, layer2.3.conv1.bias, layer2.3.conv2.bias, layer2.3.conv3.bias, layer3.0.conv1.bias, layer3.0.conv2.bias, layer3.0.conv3.bias, layer3.0.downsample.0.bias, layer3.1.conv1.bias, layer3.1.conv2.bias, layer3.1.conv3.bias, layer3.2.conv1.bias, layer3.2.conv2.bias, layer3.2.conv3.bias, layer3.3.conv1.bias, layer3.3.conv2.bias, layer3.3.conv3.bias, layer3.4.conv1.bias, layer3.4.conv2.bias, layer3.4.conv3.bias, layer3.5.conv1.bias, layer3.5.conv2.bias, layer3.5.conv3.bias, layer4.0.conv1.bias, layer4.0.conv2.bias, layer4.0.conv3.bias, layer4.0.downsample.0.bias, layer4.1.conv1.bias, layer4.1.conv2.bias, layer4.1.conv3.bias, layer4.2.conv1.bias, layer4.2.conv2.bias, layer4.2.conv3.bias

i changed the config file with the path to my dataset, which is in coco format.

should i do something else or are this changes enough? apparently the training is running fine.

thanks in advance!

how to perform validation during testing ?

hello,

I'm training using the Sipmask-benchmark version,
and I haven't been able to enable automatic validation every X iterations.
( noticed that other maskRcnn_benchmark users have a SOLVER.TEST_PERIOD parameter that controls this behavior )
is there a way to enable this functionality ?

thanks,
Omer.

SipMask-benchmark/demo/fcos_demo.py failed with 'NoneType' object is not subscriptable

hello

I've tried running
/SipMask/SipMask-benchmark$ python demo/fcos_demo.py --config-file configs/ffr/ffr-sipmask_R_50_FPN_1x.yaml --weights training_dir/ffr_50/model_0090000.pth --images-dir datasets/ffr/images/

returned following error:
Traceback (most recent call last):
File "demo/fcos_demo.py", line 113, in
main()
File "demo/fcos_demo.py", line 104, in main
composite = coco_demo.run_on_opencv_image(img)
File "/home/ubuntu/SipMask/SipMask-benchmark/demo/predictor.py", line 172, in run_on_opencv_image
predictions = self.compute_prediction(image)
File "/home/ubuntu/SipMask/SipMask-benchmark/demo/predictor.py", line 205, in compute_prediction
predictions = self.model(image_list)
File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/torch/nn/modules/module.py", line 532, in call
result = self.forward(*input, **kwargs)
File "/home/ubuntu/SipMask/SipMask-benchmark/fcos_core/modeling/detector/generalized_rcnn.py", line 51, in forward
proposals, proposal_losses = self.rpn(images, features, imge_metas, targets)
File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/torch/nn/modules/module.py", line 532, in call
result = self.forward(*input, **kwargs)
File "/home/ubuntu/SipMask/SipMask-benchmark/fcos_core/modeling/rpn/sipmask/sipmask.py", line 239, in forward
centerness, box_cof, box_mask, images.image_sizes, img_metas
File "/home/ubuntu/SipMask/SipMask-benchmark/fcos_core/modeling/rpn/sipmask/sipmask.py", line 257, in _forward_test
centerness, box_cof, box_mask, image_sizes, img_metas
File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/torch/nn/modules/module.py", line 532, in call
result = self.forward(*input, **kwargs)
File "/home/ubuntu/SipMask/SipMask-benchmark/fcos_core/modeling/rpn/sipmask/inference.py", line 163, in forward
boxlists = self.select_over_all_levels(boxlists, feat_masks, image_sizes, img_metas)
File "/home/ubuntu/SipMask/SipMask-benchmark/fcos_core/modeling/rpn/sipmask/inference.py", line 197, in select_over_all_levels
ori_shape = (img_metas[i][1],img_metas[i][0])
TypeError: 'NoneType' object is not subscriptable

please advise,
Omer

CUDA error: device-side assert triggered

你好,请问知道这个错误是为什么吗?

RuntimeError: cuda runtime error (59) : device-side assert triggered at /pytorch/aten/src/THC/THCTensorMathCompareT.cuh:69
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [0,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [1,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [2,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [3,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [4,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [5,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [6,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [7,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [8,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [9,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [10,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [11,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [12,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [13,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [14,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [15,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [16,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [17,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [18,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [19,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [20,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [21,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [22,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [23,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [24,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [25,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [26,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [27,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [28,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [29,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [30,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [31,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [32,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [33,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [34,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [35,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [36,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [37,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [38,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [39,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [40,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [41,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [42,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [43,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [44,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [45,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [46,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [47,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [48,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [49,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [50,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [51,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [52,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [53,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [54,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [55,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [56,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [57,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [58,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [59,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [60,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [61,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [62,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [63,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [64,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [65,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [66,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [67,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [68,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [69,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [70,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [71,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [72,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [73,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [74,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [75,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [76,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [77,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [78,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [79,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [80,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [81,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [82,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [83,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [84,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [85,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [86,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [87,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [88,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [89,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [90,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [91,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [92,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [93,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [94,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [95,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [96,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [97,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [98,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [99,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [100,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [101,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [102,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [103,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [104,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [105,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [106,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [107,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [108,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [109,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [110,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [111,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [112,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [113,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [114,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [115,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [116,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [117,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [118,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [119,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [120,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [121,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [122,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [123,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [124,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [125,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [126,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda ->auto::operator()(int)->auto: block: [20,0,0], thread: [127,0,0] Assertion index >= -sizes[i] && index < sizes[i] && "index out of bounds" failed.
terminate called after throwing an instance of 'c10::Error'
what(): CUDA error: device-side assert triggered (insert_events at /pytorch/c10/cuda/CUDACachingAllocator.cpp:564)
frame #0: std::function<std::string ()>::operator()() const + 0x11 (0x7f56a9601441 in /home/zsy/anaconda3/envs/sipmask/lib/python3.6/site-packages/torch/lib/libc10.so)
frame #1: c10::Error::Error(c10::SourceLocation, std::string const&) + 0x2a (0x7f56a9600d7a in /home/zsy/anaconda3/envs/sipmask/lib/python3.6/site-packages/torch/lib/libc10.so)
frame #2: + 0x13652 (0x7f56a8ba0652 in /home/zsy/anaconda3/envs/sipmask/lib/python3.6/site-packages/torch/lib/libc10_cuda.so)
frame #3: c10::TensorImpl::release_resources() + 0x50 (0x7f56a95f1ce0 in /home/zsy/anaconda3/envs/sipmask/lib/python3.6/site-packages/torch/lib/libc10.so)
frame #4: + 0x30facb (0x7f5643415acb in /home/zsy/anaconda3/envs/sipmask/lib/python3.6/site-packages/torch/lib/libtorch.so.1)
frame #5: + 0x1420bb (0x7f56a995c0bb in /home/zsy/anaconda3/envs/sipmask/lib/python3.6/site-packages/torch/lib/libtorch_python.so)
frame #6: + 0x3c30f4 (0x7f56a9bdd0f4 in /home/zsy/anaconda3/envs/sipmask/lib/python3.6/site-packages/torch/lib/libtorch_python.so)
frame #7: + 0x3c3141 (0x7f56a9bdd141 in /home/zsy/anaconda3/envs/sipmask/lib/python3.6/site-packages/torch/lib/libtorch_python.so)
frame #8: + 0x19b9fe (0x55ca87e189fe in /home/zsy/anaconda3/envs/sipmask/bin/python3.6)
frame #9: + 0xf2da8 (0x55ca87d6fda8 in /home/zsy/anaconda3/envs/sipmask/bin/python3.6)
frame #10: + 0xf2ec7 (0x55ca87d6fec7 in /home/zsy/anaconda3/envs/sipmask/bin/python3.6)
frame #11: + 0xf2787 (0x55ca87d6f787 in /home/zsy/anaconda3/envs/sipmask/bin/python3.6)
frame #12: + 0xf2617 (0x55ca87d6f617 in /home/zsy/anaconda3/envs/sipmask/bin/python3.6)
frame #13: + 0xf262d (0x55ca87d6f62d in /home/zsy/anaconda3/envs/sipmask/bin/python3.6)
frame #14: + 0xf262d (0x55ca87d6f62d in /home/zsy/anaconda3/envs/sipmask/bin/python3.6)
frame #15: + 0xf262d (0x55ca87d6f62d in /home/zsy/anaconda3/envs/sipmask/bin/python3.6)
frame #16: + 0xf262d (0x55ca87d6f62d in /home/zsy/anaconda3/envs/sipmask/bin/python3.6)
frame #17: + 0xf262d (0x55ca87d6f62d in /home/zsy/anaconda3/envs/sipmask/bin/python3.6)
frame #18: + 0xf262d (0x55ca87d6f62d in /home/zsy/anaconda3/envs/sipmask/bin/python3.6)
frame #19: + 0xf262d (0x55ca87d6f62d in /home/zsy/anaconda3/envs/sipmask/bin/python3.6)
frame #20: + 0xf262d (0x55ca87d6f62d in /home/zsy/anaconda3/envs/sipmask/bin/python3.6)
frame #21: PyDict_SetItem + 0x3da (0x55ca87db64ba in /home/zsy/anaconda3/envs/sipmask/bin/python3.6)
frame #22: PyDict_SetItemString + 0x4f (0x55ca87dbd4df in /home/zsy/anaconda3/envs/sipmask/bin/python3.6)
frame #23: PyImport_Cleanup + 0x99 (0x55ca87e22d49 in /home/zsy/anaconda3/envs/sipmask/bin/python3.6)
frame #24: Py_FinalizeEx + 0x61 (0x55ca87e8d061 in /home/zsy/anaconda3/envs/sipmask/bin/python3.6)
frame #25: Py_Main + 0x35e (0x55ca87e973ae in /home/zsy/anaconda3/envs/sipmask/bin/python3.6)
frame #26: main + 0xee (0x55ca87d6143e in /home/zsy/anaconda3/envs/sipmask/bin/python3.6)
frame #27: __libc_start_main + 0xf0 (0x7f56ae485840 in /lib/x86_64-linux-gnu/libc.so.6)
frame #28: + 0x1c3d0b (0x55ca87e40d0b in /home/zsy/anaconda3/envs/sipmask/bin/python3.6)

Regarding results.pkl.json file for SIPMASK-VIS

Hi Team,
I hope you're doing well!
I am reaching out to you regarding the interpretation of the results file, results.pkl.sjon for SIPMASK-VIS. There are multiple detections for each category on a single video. Presently, I am taking the one with the highest score. Is that correct?
Here are the values:

video_id:1, category_id:1, score:0.2766743004322052
video_id:1, category_id:2, score:0.17311915755271912
video_id:1, category_id:1, score:0.04480154812335968
video_id:1, category_id:2, score:0.050576336681842804
video_id:2, category_id:1, score:0.21141134202480316
video_id:2, category_id:2, score:0.10147302597761154
video_id:2, category_id:2, score:0.06389215588569641
video_id:3, category_id:1, score:0.26379796862602234
video_id:3, category_id:2, score:0.09652866423130035
video_id:3, category_id:2, score:0.058702562004327774
video_id:3, category_id:2, score:0.06718862056732178
video_id:3, category_id:1, score:0.03977232053875923
video_id:4, category_id:1, score:0.12127449363470078
video_id:4, category_id:2, score:0.19678737223148346
video_id:4, category_id:1, score:0.18680523335933685
video_id:4, category_id:2, score:0.04028315097093582

Thank you!

Test on the trained model

I have trained a model with 1 class in SipMask-benchmark. I want to run the inference script using my model. when I tried to execute demo/fcos_demo.py. I have changed the path to the weight file and changed the class values inside demo/fcos_demo.py and predictor.py.

I have renamed the model as FCOS_imprv_R_50_FPN_1x.pth

I got the below error
image

训练时 Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.

你好,麻烦问一下。 我现在可以跑通你的测试代码。 但是,我现在重新训练Youtube-vis时,使用train code会报错。
之前,测试时也会报这个错,但是前天经过你提供的修改版本后,test可以跑通了。但是,现在train会出现这个问题。

报错信息
rness: 0.6501, loss_mask: 0.6520, loss_match: 0.4234, match_acc: 79.0612, loss: 3.1077
2020-08-08 21:28:59,671 - INFO - Epoch [1][850/15336]   lr: 0.00425, eta: 16:25:55, time: 0.316, data_time: 0.015, memory: 3095, loss_cls: 0.7964, loss_bbox: 0.6386, loss_centerness: 0.6634, loss_mask: 0.6462, loss_match: 0.4421, match_acc: 79.8079, loss: 3.1867
2020-08-08 21:29:14,942 - INFO - Epoch [1][900/15336]   lr: 0.00450, eta: 16:22:40, time: 0.305, data_time: 0.012, memory: 3095, loss_cls: 0.7723, loss_bbox: 0.6340, loss_centerness: 0.6480, loss_mask: 0.6533, loss_match: 0.4121, match_acc: 80.7472, loss: 3.1197
2020-08-08 21:29:30,913 - INFO - Epoch [1][950/15336]   lr: 0.00475, eta: 16:22:01, time: 0.320, data_time: 0.012, memory: 3095, loss_cls: 0.7733, loss_bbox: 0.6439, loss_centerness: 0.6558, loss_mask: 0.6456, loss_match: 0.4613, match_acc: 77.0718, loss: 3.1798
2020-08-08 21:29:47,179 - INFO - Epoch [1][1000/15336]  lr: 0.00500, eta: 16:22:17, time: 0.325, data_time: 0.012, memory: 3095, loss_cls: 0.7749, loss_bbox: 0.6194, loss_centerness: 0.6524, loss_mask: 0.6529, loss_match: 0.3811, match_acc: 81.1160, loss: 3.0809
2020-08-08 21:30:02,198 - INFO - Epoch [1][1050/15336]  lr: 0.00500, eta: 16:18:52, time: 0.300, data_time: 0.011, memory: 3095, loss_cls: 0.7748, loss_bbox: 0.6533, loss_centerness: 0.6598, loss_mask: 0.6482, loss_match: 0.4230, match_acc: 81.8658, loss: 3.1591
2020-08-08 21:30:17,314 - INFO - Epoch [1][1100/15336]  lr: 0.00500, eta: 16:16:01, time: 0.302, data_time: 0.011, memory: 3095, loss_cls: 0.7779, loss_bbox: 0.6308, loss_centerness: 0.6568, loss_mask: 0.6341, loss_match: 0.3886, match_acc: 82.3207, loss: 3.0881
2020-08-08 21:30:33,515 - INFO - Epoch [1][1150/15336]  lr: 0.00500, eta: 16:16:16, time: 0.324, data_time: 0.011, memory: 3095, loss_cls: 0.7369, loss_bbox: 0.6382, loss_centerness: 0.6548, loss_mask: 0.6502, loss_match: 0.4324, match_acc: 81.9447, loss: 3.1126
2020-08-08 21:30:49,094 - INFO - Epoch [1][1200/15336]  lr: 0.00500, eta: 16:14:51, time: 0.311, data_time: 0.014, memory: 3095, loss_cls: 0.7456, loss_bbox: 0.6051, loss_centerness: 0.6448, loss_mask: 0.6525, loss_match: 0.3567, match_acc: 84.9854, loss: 3.0047
2020-08-08 21:31:04,446 - INFO - Epoch [1][1250/15336]  lr: 0.00500, eta: 16:13:03, time: 0.307, data_time: 0.012, memory: 3095, loss_cls: 0.7523, loss_bbox: 0.6132, loss_centerness: 0.6487, loss_mask: 0.6453, loss_match: 0.3773, match_acc: 81.1894, loss: 3.0367
2020-08-08 21:31:20,595 - INFO - Epoch [1][1300/15336]  lr: 0.00500, eta: 16:13:13, time: 0.323, data_time: 0.011, memory: 3095, loss_cls: 0.7536, loss_bbox: 0.5968, loss_centerness: 0.6511, loss_mask: 0.6463, loss_match: 0.4068, match_acc: 82.4096, loss: 3.0547
2020-08-08 21:31:36,113 - INFO - Epoch [1][1350/15336]  lr: 0.00500, eta: 16:11:54, time: 0.310, data_time: 0.011, memory: 3095, loss_cls: 0.7592, loss_bbox: 0.6125, loss_centerness: 0.6534, loss_mask: 0.6516, loss_match: 0.4166, match_acc: 81.3813, loss: 3.0933
2020-08-08 21:31:51,768 - INFO - Epoch [1][1400/15336]  lr: 0.00500, eta: 16:10:58, time: 0.313, data_time: 0.011, memory: 3095, loss_cls: 0.7279, loss_bbox: 0.5773, loss_centerness: 0.6434, loss_mask: 0.6333, loss_match: 0.4268, match_acc: 79.4155, loss: 3.0087
2020-08-08 21:32:07,332 - INFO - Epoch [1][1450/15336]  lr: 0.00500, eta: 16:09:53, time: 0.311, data_time: 0.011, memory: 3095, loss_cls: 0.7478, loss_bbox: 0.5974, loss_centerness: 0.6457, loss_mask: 0.6373, loss_match: 0.3742, match_acc: 82.5788, loss: 3.0025
2020-08-08 21:32:23,535 - INFO - Epoch [1][1500/15336]  lr: 0.00500, eta: 16:10:10, time: 0.324, data_time: 0.025, memory: 3095, loss_cls: 0.7332, loss_bbox: 0.6111, loss_centerness: 0.6494, loss_mask: 0.6385, loss_match: 0.3091, match_acc: 83.7850, loss: 2.9413
2020-08-08 21:32:38,794 - INFO - Epoch [1][1550/15336]  lr: 0.00500, eta: 16:08:33, time: 0.305, data_time: 0.012, memory: 3095, loss_cls: 0.7310, loss_bbox: 0.5800, loss_centerness: 0.6470, loss_mask: 0.6450, loss_match: 0.3471, match_acc: 85.5443, loss: 2.9502
2020-08-08 21:32:54,288 - INFO - Epoch [1][1600/15336]  lr: 0.00500, eta: 16:07:28, time: 0.310, data_time: 0.011, memory: 3095, loss_cls: 0.7496, loss_bbox: 0.5906, loss_centerness: 0.6543, loss_mask: 0.6394, loss_match: 0.3909, match_acc: 84.7658, loss: 3.0248
2020-08-08 21:33:10,256 - INFO - Epoch [1][1650/15336]  lr: 0.00500, eta: 16:07:19, time: 0.319, data_time: 0.012, memory: 3095, loss_cls: 0.7324, loss_bbox: 0.5914, loss_centerness: 0.6515, loss_mask: 0.6451, loss_match: 0.3490, match_acc: 83.4625, loss: 2.9694
2020-08-08 21:33:25,515 - INFO - Epoch [1][1700/15336]  lr: 0.00500, eta: 16:05:52, time: 0.305, data_time: 0.011, memory: 3095, loss_cls: 0.7436, loss_bbox: 0.6063, loss_centerness: 0.6452, loss_mask: 0.6301, loss_match: 0.3348, match_acc: 84.6653, loss: 2.9600
2020-08-08 21:33:41,107 - INFO - Epoch [1][1750/15336]  lr: 0.00500, eta: 16:05:05, time: 0.312, data_time: 0.012, memory: 3095, loss_cls: 0.7260, loss_bbox: 0.5829, loss_centerness: 0.6478, loss_mask: 0.6330, loss_match: 0.3358, match_acc: 84.9732, loss: 2.9256
2020-08-08 21:33:56,003 - INFO - Epoch [1][1800/15336]  lr: 0.00500, eta: 16:03:10, time: 0.298, data_time: 0.011, memory: 3095, loss_cls: 0.7380, loss_bbox: 0.5682, loss_centerness: 0.6520, loss_mask: 0.6421, loss_match: 0.4252, match_acc: 81.2897, loss: 3.0255
2020-08-08 21:34:11,874 - INFO - Epoch [1][1850/15336]  lr: 0.00500, eta: 16:02:55, time: 0.317, data_time: 0.011, memory: 3095, loss_cls: 0.7461, loss_bbox: 0.5867, loss_centerness: 0.6488, loss_mask: 0.6300, loss_match: 0.3946, match_acc: 81.6873, loss: 3.0062
2020-08-08 21:34:27,096 - INFO - Epoch [1][1900/15336]  lr: 0.00500, eta: 16:01:39, time: 0.305, data_time: 0.011, memory: 3095, loss_cls: 0.7425, loss_bbox: 0.5659, loss_centerness: 0.6462, loss_mask: 0.6339, loss_match: 0.3349, match_acc: 83.1951, loss: 2.9234
2020-08-08 21:34:42,450 - INFO - Epoch [1][1950/15336]  lr: 0.00500, eta: 16:00:38, time: 0.307, data_time: 0.011, memory: 3095, loss_cls: 0.7156, loss_bbox: 0.5853, loss_centerness: 0.6458, loss_mask: 0.6365, loss_match: 0.4714, match_acc: 81.8768, loss: 3.0547
2020-08-08 21:34:57,208 - INFO - Epoch [1][2000/15336]  lr: 0.00500, eta: 15:58:45, time: 0.295, data_time: 0.011, memory: 3095, loss_cls: 0.7369, loss_bbox: 0.5948, loss_centerness: 0.6464, loss_mask: 0.6382, loss_match: 0.3108, match_acc: 86.0842, loss: 2.9271
2020-08-08 21:35:12,865 - INFO - Epoch [1][2050/15336]  lr: 0.00500, eta: 15:58:13, time: 0.313, data_time: 0.011, memory: 3095, loss_cls: 0.7298, loss_bbox: 0.5639, loss_centerness: 0.6446, loss_mask: 0.6384, loss_match: 0.3719, match_acc: 82.5019, loss: 2.9486
2020-08-08 21:35:27,859 - INFO - Epoch [1][2100/15336]  lr: 0.00500, eta: 15:56:51, time: 0.301, data_time: 0.012, memory: 3095, loss_cls: 0.7314, loss_bbox: 0.5772, loss_centerness: 0.6464, loss_mask: 0.6288, loss_match: 0.4115, match_acc: 84.3310, loss: 2.9952
2020-08-08 21:35:43,216 - INFO - Epoch [1][2150/15336]  lr: 0.00500, eta: 15:55:57, time: 0.306, data_time: 0.011, memory: 3095, loss_cls: 0.7277, loss_bbox: 0.5800, loss_centerness: 0.6488, loss_mask: 0.6266, loss_match: 0.4211, match_acc: 82.1631, loss: 3.0040
2020-08-08 21:35:59,211 - INFO - Epoch [1][2200/15336]  lr: 0.00500, eta: 15:56:01, time: 0.320, data_time: 0.012, memory: 3095, loss_cls: 0.7260, loss_bbox: 0.5557, loss_centerness: 0.6417, loss_mask: 0.6312, loss_match: 0.3349, match_acc: 83.9794, loss: 2.8894
2020-08-08 21:36:14,111 - INFO - Epoch [1][2250/15336]  lr: 0.00500, eta: 15:54:34, time: 0.298, data_time: 0.011, memory: 3095, loss_cls: 0.7008, loss_bbox: 0.5723, loss_centerness: 0.6455, loss_mask: 0.6160, loss_match: 0.3169, match_acc: 86.5696, loss: 2.8514
2020-08-08 21:36:29,251 - INFO - Epoch [1][2300/15336]  lr: 0.00500, eta: 15:53:31, time: 0.303, data_time: 0.012, memory: 3095, loss_cls: 0.7353, loss_bbox: 0.5662, loss_centerness: 0.6443, loss_mask: 0.6367, loss_match: 0.3508, match_acc: 84.5352, loss: 2.9333
2020-08-08 21:36:44,466 - INFO - Epoch [1][2350/15336]  lr: 0.00500, eta: 15:52:34, time: 0.304, data_time: 0.011, memory: 3095, loss_cls: 0.7200, loss_bbox: 0.5537, loss_centerness: 0.6419, loss_mask: 0.6401, loss_match: 0.3553, match_acc: 84.2998, loss: 2.9111
2020-08-08 21:37:00,215 - INFO - Epoch [1][2400/15336]  lr: 0.00500, eta: 15:52:20, time: 0.315, data_time: 0.012, memory: 3095, loss_cls: 0.7434, loss_bbox: 0.5692, loss_centerness: 0.6481, loss_mask: 0.6277, loss_match: 0.3681, match_acc: 83.9581, loss: 2.9566
2020-08-08 21:37:15,480 - INFO - Epoch [1][2450/15336]  lr: 0.00500, eta: 15:51:30, time: 0.306, data_time: 0.011, memory: 3095, loss_cls: 0.7208, loss_bbox: 0.5515, loss_centerness: 0.6454, loss_mask: 0.6220, loss_match: 0.3207, match_acc: 85.4823, loss: 2.8605
2020-08-08 21:37:30,535 - INFO - Epoch [1][2500/15336]  lr: 0.00500, eta: 15:50:25, time: 0.301, data_time: 0.011, memory: 3095, loss_cls: 0.7279, loss_bbox: 0.5622, loss_centerness: 0.6490, loss_mask: 0.6312, loss_match: 0.3531, match_acc: 84.4417, loss: 2.9234
2020-08-08 21:37:45,963 - INFO - Epoch [1][2550/15336]  lr: 0.00500, eta: 15:49:51, time: 0.309, data_time: 0.011, memory: 3095, loss_cls: 0.7056, loss_bbox: 0.5183, loss_centerness: 0.6353, loss_mask: 0.6171, loss_match: 0.2979, match_acc: 85.8967, loss: 2.7742
2020-08-08 21:38:01,695 - INFO - Epoch [1][2600/15336]  lr: 0.00500, eta: 15:49:37, time: 0.315, data_time: 0.012, memory: 3095, loss_cls: 0.7048, loss_bbox: 0.5487, loss_centerness: 0.6448, loss_mask: 0.6169, loss_match: 0.3339, match_acc: 86.0841, loss: 2.8492
2020-08-08 21:38:17,266 - INFO - Epoch [1][2650/15336]  lr: 0.00500, eta: 15:49:12, time: 0.311, data_time: 0.012, memory: 3095, loss_cls: 0.6899, loss_bbox: 0.5335, loss_centerness: 0.6436, loss_mask: 0.6148, loss_match: 0.3327, match_acc: 85.6113, loss: 2.8145
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [32,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [33,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [34,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [35,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [36,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [37,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [38,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [39,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [40,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [41,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [42,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [43,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [44,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [45,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [46,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [47,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [48,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [49,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [50,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [51,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [52,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [53,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [54,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [55,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [56,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [57,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [58,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [59,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [60,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [61,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [62,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [63,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [64,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [65,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [66,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [67,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [68,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [69,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [70,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [71,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [72,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [73,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [74,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [75,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [76,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [77,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [78,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [79,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [80,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [81,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [82,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [83,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [84,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [85,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [86,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [87,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [88,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [89,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [90,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [91,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [92,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [93,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [94,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [95,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [96,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [97,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [98,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [99,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [100,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [101,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [102,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [103,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [104,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [105,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [106,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [107,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [108,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [109,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [110,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [111,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [112,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [113,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [114,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [115,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [116,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [117,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [118,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [119,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [120,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [121,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [122,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [123,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [124,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [125,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [126,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [127,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [0,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [1,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [2,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [3,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [4,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [5,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [6,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [7,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [8,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [9,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [10,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [11,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [12,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [13,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [14,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [15,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [16,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [17,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [18,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [19,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [20,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [21,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [22,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [23,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [24,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [25,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [26,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [27,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [28,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [29,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [30,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [23,0,0], thread: [31,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
Traceback (most recent call last):
  File "./tools/train.py", line 110, in <module>
    main()
  File "./tools/train.py", line 106, in main
    logger=logger)
  File "/home/zxq/code/SipMask-master(1)/SipMask-master/SipMask-VIS/tools/../mmdet/apis/train.py", line 60, in train_detector
    _non_dist_train(model, dataset, cfg, validate=validate)
  File "/home/zxq/code/SipMask-master(1)/SipMask-master/SipMask-VIS/tools/../mmdet/apis/train.py", line 221, in _non_dist_train
    runner.run(data_loaders, cfg.workflow, cfg.total_epochs)
  File "/home/zxq/anaconda3/envs/sipmask/lib/python3.6/site-packages/mmcv/runner/runner.py", line 358, in run
    epoch_runner(data_loaders[i], **kwargs)
  File "/home/zxq/anaconda3/envs/sipmask/lib/python3.6/site-packages/mmcv/runner/runner.py", line 264, in train
    self.model, data_batch, train_mode=True, **kwargs)
  File "/home/zxq/code/SipMask-master(1)/SipMask-master/SipMask-VIS/tools/../mmdet/apis/train.py", line 38, in batch_processor
    losses = model(**data)
  File "/home/zxq/anaconda3/envs/sipmask/lib/python3.6/site-packages/torch/nn/modules/module.py", line 493, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/zxq/anaconda3/envs/sipmask/lib/python3.6/site-packages/torch/nn/parallel/data_parallel.py", line 152, in forward
    outputs = self.parallel_apply(replicas, inputs, kwargs)
  File "/home/zxq/anaconda3/envs/sipmask/lib/python3.6/site-packages/torch/nn/parallel/data_parallel.py", line 162, in parallel_apply
    return parallel_apply(replicas, inputs, kwargs, self.device_ids[:len(replicas)])
  File "/home/zxq/anaconda3/envs/sipmask/lib/python3.6/site-packages/torch/nn/parallel/parallel_apply.py", line 83, in parallel_apply
    raise output
  File "/home/zxq/anaconda3/envs/sipmask/lib/python3.6/site-packages/torch/nn/parallel/parallel_apply.py", line 59, in _worker
    output = module(*input, **kwargs)
  File "/home/zxq/anaconda3/envs/sipmask/lib/python3.6/site-packages/torch/nn/modules/module.py", line 493, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/zxq/code/SipMask-master(1)/SipMask-master/SipMask-VIS/tools/../mmdet/core/fp16/decorators.py", line 49, in new_func
    return old_func(*args, **kwargs)
  File "/home/zxq/code/SipMask-master(1)/SipMask-master/SipMask-VIS/tools/../mmdet/models/detectors/base.py", line 86, in forward
    return self.forward_train(img, img_meta, **kwargs)
  File "/home/zxq/code/SipMask-master(1)/SipMask-master/SipMask-VIS/tools/../mmdet/models/detectors/single_stage.py", line 69, in forward_train
    *loss_inputs, gt_bboxes_ignore=gt_bboxes_ignore, gt_masks_list=gt_masks, ref_bboxes_list = ref_bboxes, gt_pids_list=gt_pids)
  File "/home/zxq/code/SipMask-master(1)/SipMask-master/SipMask-VIS/tools/../mmdet/core/fp16/decorators.py", line 127, in new_func
    return old_func(*args, **kwargs)
  File "/home/zxq/code/SipMask-master(1)/SipMask-master/SipMask-VIS/tools/../mmdet/models/anchor_heads/sipmask_head.py", line 506, in loss
    gt_mask_new[:gt_mask.shape[0], :shape[1], :shape[2]] = gt_mask[:gt_mask.shape[0], :shape[1], :shape[2]]
RuntimeError: CUDA error: device-side assert triggered
terminate called after throwing an instance of 'c10::Error'
  what():  CUDA error: device-side assert triggered (insert_events at ../c10/cuda/CUDACachingAllocator.cpp:564)
frame #0: c10::Error::Error(c10::SourceLocation, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) + 0x6d (0x7fa5fb1ea33d in /home/zxq/anaconda3/envs/sipmask/lib/python3.6/site-packages/torch/lib/libc10.so)
frame #1: <unknown function> + 0x11ea1 (0x7fa5fb20eea1 in /home/zxq/anaconda3/envs/sipmask/lib/python3.6/site-packages/torch/lib/libc10_cuda.so)
frame #2: c10::TensorImpl::release_resources() + 0x5c (0x7fa5fb1dc91c in /home/zxq/anaconda3/envs/sipmask/lib/python3.6/site-packages/torch/lib/libc10.so)
frame #3: torch::autograd::Variable::Impl::release_resources() + 0x7c (0x7fa5febec17c in /home/zxq/anaconda3/envs/sipmask/lib/python3.6/site-packages/torch/lib/libtorch.so.1)
frame #4: <unknown function> + 0x124c84 (0x7fa6234b8c84 in /home/zxq/anaconda3/envs/sipmask/lib/python3.6/site-packages/torch/lib/libtorch_python.so)
frame #5: <unknown function> + 0x35467c (0x7fa6236e867c in /home/zxq/anaconda3/envs/sipmask/lib/python3.6/site-packages/torch/lib/libtorch_python.so)
frame #6: <unknown function> + 0x3546b2 (0x7fa6236e86b2 in /home/zxq/anaconda3/envs/sipmask/lib/python3.6/site-packages/torch/lib/libtorch_python.so)
<omitting python frames>
frame #27: __libc_start_main + 0xe7 (0x7fa631940b97 in /lib/x86_64-linux-gnu/libc.so.6)

Aborted (core dumped)

Evaluation output names for SipMask-VIS

Hi Team,

I am reaching out regarding the output names during the evaluation of SipMask-VIS. The present code requires video ids to start from 1, however, it saves the output videos starting from 0, thereby mapping 1 to 0 and henceforth.

Also, I am not sure if the above mismatch is the reason that the value of average precision is 0 at the end, while the model does a good job in detecting and tracking the objects in the output frames.

Second, in the saved path, the output misses some frames in the sequence. However, in the results.pkl.json file, the length of segmentations corresponding to a video is correct. From my analysis, it is skipping frames with None as the segmentation value.

I really appreciate your thoughts if I am missing anything.

Thanks!

Results have no predicitons

Hi!
I run the test code

tools/test_video.py configs/sipmask/sipmask_r50_caffe_fpn_gn_1x.py work_dir/sipmask_r50_caffe_1x.pth --out sipmask.pkl --eval segm

using the pth file downloaded from your readme.

However, when the test finished, the outputs of the model is empty. Where should I fix?

About training process in the Youtube-VIS dataset.

My Config
# model settings
model = dict(
    type="SipMask",
    # pretrained="open-mmlab://resnet50_caffe",
    backbone=dict(
        type="ResNet",
        depth=50,
        num_stages=4,
        out_indices=(0, 1, 2, 3),
        frozen_stages=1,
        norm_cfg=dict(type="BN", requires_grad=False),
        style="caffe",
    ),
    neck=dict(
        type="FPN",
        in_channels=[256, 512, 1024, 2048],
        out_channels=256,
        start_level=1,
        add_extra_convs=True,
        extra_convs_on_inputs=False,  # use P5
        num_outs=5,
        relu_before_extra_convs=True,
    ),
    bbox_head=dict(
        type="SipMaskHead",
        num_classes=41,
        in_channels=256,
        stacked_convs=3,
        feat_channels=256,
        strides=[8, 16, 32, 64, 128],
        loss_cls=dict(
            type="FocalLoss", use_sigmoid=True, gamma=2.0, alpha=0.25, loss_weight=1.0
        ),
        loss_bbox=dict(type="IoULoss", loss_weight=1.0),
        loss_centerness=dict(
            type="CrossEntropyLoss", use_sigmoid=True, loss_weight=1.0
        ),
        center_sampling=True,
        center_sample_radius=1.5,
    ),
)
# training and testing settings
train_cfg = dict(
    assigner=dict(
        type="MaxIoUAssigner",
        pos_iou_thr=0.5,
        neg_iou_thr=0.4,
        min_pos_iou=0,
        ignore_iof_thr=-1,
    ),
    allowed_border=-1,
    pos_weight=-1,
    debug=False,
)
test_cfg = dict(
    nms_pre=200,
    min_bbox_size=0,
    score_thr=0.03,
    nms=dict(type="nms", iou_thr=0.5),
    max_per_img=10,
)
# dataset settings
dataset_type = "YTVOSDataset"
data_root = "/home/lart/Datasets/VideoSeg/YoutubeVIS2019/"
img_norm_cfg = dict(
    mean=[102.9801, 115.9465, 122.7717], std=[1.0, 1.0, 1.0], to_rgb=False
)
data = dict(
    imgs_per_gpu=2,
    workers_per_gpu=2,
    train=dict(
        type=dataset_type,
        ann_file=data_root + "Train/train.json",
        img_prefix=data_root + "Train/JPEGImages",
        img_scale=[(649, 360), (960, 480)],
        img_norm_cfg=img_norm_cfg,
        size_divisor=32,
        flip_ratio=0.5,
        with_mask=True,
        with_crowd=True,
        with_label=True,
        with_track=True,
    ),
    val=dict(
        type=dataset_type,
        ann_file=data_root + "Valid/valid.json",
        img_prefix=data_root + "Valid/JPEGImages",
        img_scale=(640, 360),
        img_norm_cfg=img_norm_cfg,
        size_divisor=32,
        flip_ratio=0,
        with_mask=True,
        with_crowd=True,
        with_label=True,
    ),
    test=dict(
        type=dataset_type,
        ann_file=data_root + "Valid/valid.json",
        img_prefix=data_root + "Valid/JPEGImages",
        img_scale=(640, 360),
        img_norm_cfg=img_norm_cfg,
        size_divisor=32,
        flip_ratio=0,
        with_mask=False,
        with_label=False,
        test_mode=True,
    ),
)

# optimizer
optimizer = dict(
    type="SGD",
    lr=0.005,
    momentum=0.9,
    weight_decay=0.0001,
    paramwise_options=dict(bias_lr_mult=2.0, bias_decay_mult=0.0),
)
optimizer_config = dict(grad_clip=None)
# learning policy
lr_config = dict(
    policy="step",
    warmup="linear",
    warmup_iters=1000,
    warmup_ratio=1.0 / 80,
    step=[8, 11],
)
checkpoint_config = dict(interval=1)
# yapf:disable
log_config = dict(
    interval=10,
    hooks=[
        dict(type="TextLoggerHook"),
        # dict(type='TensorboardLoggerHook')
    ],
)
# yapf:enable
# runtime settings
total_epochs = 12
device_ids = range(8)
dist_params = dict(backend="nccl")
log_level = "INFO"
work_dir = "./work_dirs/sipmask_r50_fpn_ms_1x"
load_from = None
resume_from = None
workflow = [("train", 1)]

When I train the code on the trainset of the Youtube-VIS, it outputs such an error message:

ERROR MESSAGE
➜ ./tools/dist_train.sh ./configs/sipmask/sipmask_r50_caffe_fpn_gn_ms_train.py 2
2020-08-08 13:08:35,462 - INFO - Distributed training: True
loading annotations into memory...
loading annotations into memory...
Done (t=7.55s)
creating index...
index created!
Done (t=7.63s)
creating index...
index created!
2020-08-08 13:09:34,491 - INFO - Start running, host: lart@god, work_dir: /home/lart/Coding/VISProj/SipMaskDockerProj/SipMask-VIS/work_dirs/sipmask_r50_fpn_ms_1x
2020-08-08 13:09:34,491 - INFO - workflow: [('train', 1)], max: 12 epochs
Traceback (most recent call last):
  File "./tools/train.py", line 110, in <module>
    main()
  File "./tools/train.py", line 106, in main
    logger=logger)
  File "/home/lart/Coding/VISProj/SipMaskDockerProj/SipMask-VIS/tools/../mmdet/apis/train.py", line 58, in train_detector
    _dist_train(model, dataset, cfg, validate=validate)
  File "/home/lart/Coding/VISProj/SipMaskDockerProj/SipMask-VIS/tools/../mmdet/apis/train.py", line 186, in _dist_train
    runner.run(data_loaders, cfg.workflow, cfg.total_epochs)
  File "/home/lart/miniconda3/envs/sipmask/lib/python3.6/site-packages/mmcv/runner/runner.py", line 358, in run
    epoch_runner(data_loaders[i], **kwargs)
  File "/home/lart/miniconda3/envs/sipmask/lib/python3.6/site-packages/mmcv/runner/runner.py", line 264, in train
    self.model, data_batch, train_mode=True, **kwargs)
  File "/home/lart/Coding/VISProj/SipMaskDockerProj/SipMask-VIS/tools/../mmdet/apis/train.py", line 38, in batch_processor
    losses = model(**data)
  File "/home/lart/miniconda3/envs/sipmask/lib/python3.6/site-packages/torch/nn/modules/module.py", line 493, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/lart/miniconda3/envs/sipmask/lib/python3.6/site-packages/mmcv/parallel/distributed.py", line 50, in forward
    return self.module(*inputs[0], **kwargs[0])
  File "/home/lart/miniconda3/envs/sipmask/lib/python3.6/site-packages/torch/nn/modules/module.py", line 493, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/lart/Coding/VISProj/SipMaskDockerProj/SipMask-VIS/tools/../mmdet/core/fp16/decorators.py", line 49, in new_func
    return old_func(*args, **kwargs)
  File "/home/lart/Coding/VISProj/SipMaskDockerProj/SipMask-VIS/tools/../mmdet/models/detectors/base.py", line 86, in forward
    return self.forward_train(img, img_meta, **kwargs)
  File "/home/lart/Coding/VISProj/SipMaskDockerProj/SipMask-VIS/tools/../mmdet/models/detectors/single_stage.py", line 66, in forward_train
    *loss_inputs, gt_bboxes_ignore=gt_bboxes_ignore, gt_masks_list=gt_masks, ref_bboxes_list = ref_bboxes, gt_pids_list=gt_pids)
  File "/home/lart/Coding/VISProj/SipMaskDockerProj/SipMask-VIS/tools/../mmdet/core/fp16/decorators.py", line 127, in new_func
    return old_func(*args, **kwargs)
  File "/home/lart/Coding/VISProj/SipMaskDockerProj/SipMask-VIS/tools/../mmdet/models/anchor_heads/sipmask_head.py", line 497, in loss
    loss_match += cross_entropy(prod_ext, cur_ids)
  File "/home/lart/Coding/VISProj/SipMaskDockerProj/SipMask-VIS/tools/../mmdet/models/losses/cross_entropy_loss.py", line 11, in cross_entropy
    loss = F.cross_entropy(pred, label, reduction='none')
  File "/home/lart/miniconda3/envs/sipmask/lib/python3.6/site-packages/torch/nn/functional.py", line 2056, in cross_entropy
    return nll_loss(log_softmax(input, 1), target, weight, None, ignore_index, None, reduction)
  File "/home/lart/miniconda3/envs/sipmask/lib/python3.6/site-packages/torch/nn/functional.py", line 1871, in nll_loss
    ret = torch._C._nn.nll_loss(input, target, weight, _Reduction.get_enum(reduction), ignore_index)
RuntimeError: invalid argument 2: non-empty vector or matrix expected at /tmp/pip-req-build-xea91aoh/aten/src/THCUNN/generic/ClassNLLCriterion.cu:31
Traceback (most recent call last):
  File "/home/lart/miniconda3/envs/sipmask/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/lart/miniconda3/envs/sipmask/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/lart/miniconda3/envs/sipmask/lib/python3.6/site-packages/torch/distributed/launch.py", line 235, in <module>
    main()
  File "/home/lart/miniconda3/envs/sipmask/lib/python3.6/site-packages/torch/distributed/launch.py", line 231, in main
    cmd=process.args)
subprocess.CalledProcessError: Command '['/home/lart/miniconda3/envs/sipmask/bin/python', '-u', './tools/train.py', '--local_rank=0', './configs/sipmask/sipmask_r50_caffe_fpn_gn_ms_train.py', '--launcher', 'pytorch']' returned non-zero exit status 1.

But, if I set the item from imgs_per_gpu=2 to imgs_per_gpu=4, it works well:

MESSAGE
➜ ./tools/dist_train.sh ./configs/sipmask/sipmask_r50_caffe_fpn_gn_ms_train.py 2
2020-08-08 13:10:36,074 - INFO - Distributed training: True
/home/lart/miniconda3/envs/sipmask/lib/python3.6/multiprocessing/semaphore_tracker.py:143: UserWarning: semaphore_tracker: There appear to be 14 leaked semaphores to clean up at shutdown
  len(cache))
loading annotations into memory...
loading annotations into memory...
Done (t=7.70s)
creating index...
index created!
Done (t=7.76s)
creating index...
index created!
2020-08-08 13:11:35,432 - INFO - Start running, host: lart@god, work_dir: /home/lart/Coding/VISProj/SipMaskDockerProj/SipMask-VIS/work_dirs/sipmask_r50_fpn_ms_1x
2020-08-08 13:11:35,432 - INFO - workflow: [('train', 1)], max: 12 epochs
2020-08-08 13:11:54,687 - INFO - Epoch [1][10/7669]     lr: 0.00011, eta: 2 days, 0:49:07, time: 1.910, data_time: 1.400, memory: 5320, loss_cls: 1.0477, loss_bbox: 2.2564, loss_centerness: 0.6610, loss_mask: 0.6855, loss_match: 1.6922, match_acc: 78.6566, loss: 6.3429
2020-08-08 13:11:59,198 - INFO - Epoch [1][20/7669]     lr: 0.00016, eta: 1 day, 6:08:53, time: 0.449, data_time: 0.025, memory: 5342, loss_cls: 1.0134, loss_bbox: 0.8200, loss_centerness: 0.6511, loss_mask: 0.6491, loss_match: 1.2980, match_acc: 80.7617, loss: 4.4316
^CTraceback (most recent call last):
  File "/home/lart/miniconda3/envs/sipmask/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/lart/miniconda3/envs/sipmask/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/lart/miniconda3/envs/sipmask/lib/python3.6/site-packages/torch/distributed/launch.py", line 235, in <module>
    main()
  File "/home/lart/miniconda3/envs/sipmask/lib/python3.6/site-packages/torch/distributed/launch.py", line 228, in main
    process.wait()
  File "/home/lart/miniconda3/envs/sipmask/lib/python3.6/subprocess.py", line 1477, in wait
    (pid, sts) = self._try_wait(0)
  File "/home/lart/miniconda3/envs/sipmask/lib/python3.6/subprocess.py", line 1424, in _try_wait
    (pid, sts) = os.waitpid(self.pid, wait_flags)
KeyboardInterrupt
VISProj/SipMaskDockerProj/SipMask-VIS via 🐍 v3.6.10 :: Anaconda, Inc. via C sipmask took 2m8s 
➜ Traceback (most recent call last):
  File "./tools/train.py", line 110, in <module>
    main()
  File "./tools/train.py", line 106, in main
    logger=logger)
  File "/home/lart/Coding/VISProj/SipMaskDockerProj/SipMask-VIS/tools/../mmdet/apis/train.py", line 58, in train_detector
    _dist_train(model, dataset, cfg, validate=validate)
  File "/home/lart/Coding/VISProj/SipMaskDockerProj/SipMask-VIS/tools/../mmdet/apis/train.py", line 186, in _dist_train
    runner.run(data_loaders, cfg.workflow, cfg.total_epochs)
  File "/home/lart/miniconda3/envs/sipmask/lib/python3.6/site-packages/mmcv/runner/runner.py", line 358, in run
    epoch_runner(data_loaders[i], **kwargs)
  File "/home/lart/miniconda3/envs/sipmask/lib/python3.6/site-packages/mmcv/runner/runner.py", line 271, in train
    self.call_hook('after_train_iter')
  File "/home/lart/miniconda3/envs/sipmask/lib/python3.6/site-packages/mmcv/runner/runner.py", line 229, in call_hook
    getattr(hook, fn_name)(self)
  File "/home/lart/Coding/VISProj/SipMaskDockerProj/SipMask-VIS/tools/../mmdet/core/utils/dist_utils.py", line 58, in after_train_iter
Traceback (most recent call last):
  File "./tools/train.py", line 110, in <module>
    main()
  File "./tools/train.py", line 106, in main
    logger=logger)
  File "/home/lart/Coding/VISProj/SipMaskDockerProj/SipMask-VIS/tools/../mmdet/apis/train.py", line 58, in train_detector
    _dist_train(model, dataset, cfg, validate=validate)
  File "/home/lart/Coding/VISProj/SipMaskDockerProj/SipMask-VIS/tools/../mmdet/apis/train.py", line 186, in _dist_train
    runner.run(data_loaders, cfg.workflow, cfg.total_epochs)
  File "/home/lart/miniconda3/envs/sipmask/lib/python3.6/site-packages/mmcv/runner/runner.py", line 358, in run
    epoch_runner(data_loaders[i], **kwargs)
  File "/home/lart/miniconda3/envs/sipmask/lib/python3.6/site-packages/mmcv/runner/runner.py", line 271, in train
    self.call_hook('after_train_iter')
  File "/home/lart/miniconda3/envs/sipmask/lib/python3.6/site-packages/mmcv/runner/runner.py", line 229, in call_hook
    getattr(hook, fn_name)(self)
  File "/home/lart/Coding/VISProj/SipMaskDockerProj/SipMask-VIS/tools/../mmdet/core/utils/dist_utils.py", line 53, in after_train_iter
    runner.outputs['loss'].backward()
  File "/home/lart/miniconda3/envs/sipmask/lib/python3.6/site-packages/torch/tensor.py", line 107, in backward
    runner.optimizer.step()
  File "/home/lart/miniconda3/envs/sipmask/lib/python3.6/site-packages/torch/optim/sgd.py", line 100, in step
    torch.autograd.backward(self, gradient, retain_graph, create_graph)
  File "/home/lart/miniconda3/envs/sipmask/lib/python3.6/site-packages/torch/autograd/__init__.py", line 93, in backward
    buf.mul_(momentum).add_(1 - dampening, d_p)
KeyboardInterrupt
    allow_unreachable=True)  # allow_unreachable flag
KeyboardInterrupt

ModuleNotFoundError: No module named 'pycocotools.ytvos'

Hi.

I followed the instruction to run the SipMask-VIS, following your readme (using docker build).

When I run test, "python tools/test_video.py configs/sipmask/sipmask_r50_caffe_fpn_gn_1x.py work_dir/sipmask_r50_caffe_1x.pth --out tmp.pkl --eval segm",

I got an error message
"
File "/workspace/sipmask/SipMask/SipMask-VIS/tools/../mmdet/core/evaluation/coco_utils.py", line 5, in
from pycocotools.ytvos import YTVOS
ModuleNotFoundError: No module named 'pycocotools.ytvos'
"

I tried to clone the repository https://github.com/youtubevos/cocoapi,
and install following the instruction of the youtubevos/cocoapi, but the problem have not been solved.

What should I do?

Questions about occlusion data

I have a confusion. If an instance is divided into two parts, how to label it, should it be marked as a closed polygonal box or two closed polygonal boxes?
For example, the person and the parallel bars in the picture below need to be marked, but the person on the right and the parallel bars block each other, and the person is divided into upper and lower parts. How should this person be marked?
1

Any plan for supporting CUDA 11 to support new 30 series GPU?

Hi,

I am trying to install SipMask-VIS. Unfortunately my 30 series GPU cannot work with CUDA version below 11. DO you have any suggestion to work around it? I noticed that the new mmdet version seems do not have such problem but this porject seems to be built on an old version

Non-existent config key: MODEL.SIPMASK_ON

hello Sipmask community,

This is my first attempt at using SipMask, so I guess this is some sort of installation error.

i've followed install.md :
installed FCOS ,
cloned SipMask,
cd SipMask-benchmark
python tools/test_net.py --config-file configs/sipmask/sipmask_R_50_FPN_1x.yaml MODEL.WEIGHT training_dir/SipMask_R50_1x.pth TEST.IMS_PER_BATCH 4

sadly, received the following error :
Traceback (most recent call last):
File "tools/test_net.py", line 97, in
main()
File "tools/test_net.py", line 49, in main
cfg.merge_from_file(args.config_file)
File "/usr/local/lib/python3.7/dist-packages/yacs/config.py", line 213, in merge_from_file
self.merge_from_other_cfg(cfg)
File "/usr/local/lib/python3.7/dist-packages/yacs/config.py", line 217, in merge_from_other_cfg
_merge_a_into_b(cfg_other, self, self, [])
File "/usr/local/lib/python3.7/dist-packages/yacs/config.py", line 478, in _merge_a_into_b
_merge_a_into_b(v, b[k], root, key_list + [k])
File "/usr/local/lib/python3.7/dist-packages/yacs/config.py", line 491, in _merge_a_into_b
raise KeyError("Non-existent config key: {}".format(full_key))
KeyError: 'Non-existent config key: MODEL.SIPMASK_ON'

any ideas?
Omer

Sipmask:mmdetection CUDA out of memory error

Hi,when training with the original code,I got RuntimeError as below:

pred_masks = torch.stack([pos_masks00, pos_masks01, pos_masks10, pos_masks11], dim=0)
RuntimeError: CUDA out of memory. Tried to allocate 464.00 MiB (GPU 1; 10.76 GiB total capacity; 7.33 GiB already alloca
ted; 97.19 MiB free; 1.19 GiB cached)

There was no one using GPU when I trained it. First I thought it might be a batchsize issue. But even though I changed the batch size from 16 to 1,the same error still occured .The only difference between batchsize 16 and batchsize 1 is ,when changed to batchsize 1,the training was able to run for half epoch,then the error occurred.

I wonder if the code didn't clear the grad during training .

BTW,my GPU is GeForce RTX 2080 Ti ,11019M.During training I used 2 GPUs.
My environment is
pytorch = 1.1.0 torchvision = 0.3.0 mmcv = 0.4.3

Please tell me how to deal with it.Thanks a lot!

SipMask-benchmark

@JialeCao001
Thank for sharing such a great work!
Iam testing your work based on maskrcnn-benchmark.
But there seems a missing of make_data_loader in fcos_core folder ?
image

关于SipMask-VIS的预训练模型

谢谢大佬分享工作。我下载了SipMask-mmdetection的训练模型当作VIS的预训练模型,但是加载模型时出现错误:
RuntimeError:While copying the parameter named bbox_head.fcos_cls.weight,whose dimension in the model are torch.size([40,256,3,3]) ang whose dimension in the checkpoint are torch.size([80,256,3,3]).

这个错误应该是由于coco和youtube-vis数据集类别不一致导致的,请问我是不是下载错了预训练模型。

SipMask-VIS Build Issue

Hi,

Many thanks for sharing your work on instance segmentation!

I am writing regarding the error during installation of SipMask-VIS code. On running setup.py file from the SipMask-VIS repo ( as python setup.py build_ext install), I am getting the following error:
"distutils.errors.DistutilsSetupError: each element of 'ext_modules' option must be an Extension instance or 2-tuple"

On debugging, the error is happening due to extension checking at function "def check_extensions_list(self, extensions)" in the file ".../lib/python3.7/distutils/command/build_ext.py".

This instance checking fails for mmcv extension: setuptools.extension.Extension('mmcv._flow_warp_ext'). It is strange though because the mmcv._flow_warp_ext is a correct extension instance.

At first, logically, it should not fail while instance checking because the extension is of correct instance (setuptools.extension.Extension). Second, this is happening while installing mmcv 1.1.1 (as best match for mmcv>=0.2.10) as part of the setup.py code flow.

How shall I resolve this error? I even tried pre-installing mmcv-full both directly and compiled from source, both the same error happens. Shall I explicitly pre-install mmcv=0.2.1 as mentioned in your settings?

Let me know if I can provide further details. Looking forward to hearing from you!

Thanks!

Speed shrink down exponential when input size larger

I trained the input like this:

test_pipeline = [
    dict(type='LoadImageFromFile'),
    # dict(type='LoadAnnotations', with_bbox=True, with_mask=True),
    dict(
        type='MultiScaleFlipAug',
        img_scale=(1280, 768),
        flip=False,
        transforms=[
            dict(type='Resize', keep_ratio=True),
            # dict(type='RandomFlip'),
            dict(type='Normalize', **img_norm_cfg),
            dict(type='Pad', size_divisor=32),
            dict(type='ImageToTensor', keys=['img']),
            dict(type='Collect', keys=['img']),
        ])
]

and my raw video resolution is just like this, but the speed I tested for only model forward time:

cost: 0.42196059226989746, fps: 2.369889554426383
cost: 0.44635605812072754, fps: 2.240363901881951
cost: 0.4053168296813965, fps: 2.467205718514231
cost: 0.4938929080963135, fps: 2.024730429627856
cost: 0.47973155975341797, fps: 2.0844990905205405
cost: 0.6344692707061768, fps: 1.576120461889321
cost: 0.4951167106628418, fps: 2.0197258110340113
cost: 0.4999728202819824, fps: 2.000108724782288
cost: 0.39091920852661133, fps: 2.5580733261203417
cost: 0.3721492290496826, fps: 2.6870941061831357
cost: 0.37459611892700195, fps: 2.6695418064245118
cost: 0.35617923736572266, fps: 2.8075752180164453
cost: 0.3825395107269287, fps: 2.6141090579107216
cost: 0.3750581741333008, fps: 2.6662530481060425
cost: 0.3960697650909424, fps: 2.5248077185856084
cost: 0.45439767837524414, fps: 2.200715469268297
cost: 0.4749162197113037, fps: 2.105634548779759
cost: 0.4713883399963379, fps: 2.1213931596351507

Up to 400 ms per frame, is that normal?

Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.

Thanks for your work.

My ENV:
  • Ubuntu 18.04.04
  • 2080Ti
  • cuda in my system: 9.2, V9.2.148
  • gcc, g++ 5.5.0
  • conda list:
➜ conda list
# packages in environment at /home/lart/miniconda3/envs/sipmask:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                        main    defaults
addict                    2.2.1                    pypi_0    pypi
blas                      1.0                         mkl    defaults
ca-certificates           2020.6.24                     0    defaults
certifi                   2020.6.20                py36_0    defaults
cffi                      1.14.0           py36he30daa8_1    defaults
chardet                   3.0.4                    pypi_0    pypi
cudatoolkit               9.2                           0    defaults
cudnn                     7.6.5                 cuda9.2_0    defaults
cycler                    0.10.0                   pypi_0    pypi
cython                    0.29.21                  pypi_0    pypi
decorator                 4.4.2                    pypi_0    pypi
freetype                  2.10.2               h5ab3b9f_0    defaults
idna                      2.10                     pypi_0    pypi
imagecorruptions          1.1.0                    pypi_0    pypi
imageio                   2.9.0                    pypi_0    pypi
intel-openmp              2020.1                      217    defaults
jpeg                      9b                   h024ee3a_2    defaults
kiwisolver                1.2.0                    pypi_0    pypi
lcms2                     2.11                 h396b838_0    defaults
ld_impl_linux-64          2.33.1               h53a641e_7    defaults
libedit                   3.1.20191231         h14c3975_1    defaults
libffi                    3.3                  he6710b0_2    defaults
libgcc-ng                 9.1.0                hdf63c60_0    defaults
libpng                    1.6.37               hbc83047_0    defaults
libstdcxx-ng              9.1.0                hdf63c60_0    defaults
libtiff                   4.1.0                h2733197_1    defaults
lz4-c                     1.9.2                he6710b0_1    defaults
matplotlib                3.3.0                    pypi_0    pypi
mkl                       2020.1                      217    defaults
mkl-service               2.3.0            py36he904b0f_0    defaults
mkl_fft                   1.1.0            py36h23d657b_0    defaults
mkl_random                1.1.1            py36h0573a6f_0    defaults
mmcv                      0.2.10                   pypi_0    pypi
mmdet                     1.0rc0+unknown            dev_0    <develop>
ncurses                   6.2                  he6710b0_1    defaults
networkx                  2.4                      pypi_0    pypi
ninja                     1.10.0           py36hfd86e86_0    defaults
numpy                     1.19.1           py36hbc911f0_0    defaults
numpy-base                1.19.1           py36hfa32c7d_0    defaults
olefile                   0.46                     py36_0    defaults
opencv-python             4.3.0.36                 pypi_0    pypi
openssl                   1.1.1g               h7b6447c_0    defaults
pillow                    7.2.0            py36hb39fc2d_0    defaults
pip                       20.1.1                   py36_1    defaults
pycocotools               2.0                      pypi_0    pypi
pycparser                 2.20                       py_2    defaults
pyparsing                 2.4.7                    pypi_0    pypi
python                    3.6.10               h7579374_2    defaults
python-dateutil           2.8.1                    pypi_0    pypi
pytorch                   1.1.0           cuda92py36h65efead_0    defaults
pywavelets                1.1.1                    pypi_0    pypi
pyyaml                    5.3.1                    pypi_0    pypi
readline                  8.0                  h7b6447c_0    defaults
requests                  2.24.0                   pypi_0    pypi
scikit-image              0.17.2                   pypi_0    pypi
scipy                     1.5.2                    pypi_0    pypi
setuptools                49.2.0                   py36_0    defaults
six                       1.15.0                     py_0    defaults
sqlite                    3.32.3               h62c20be_0    defaults
tifffile                  2020.7.24                pypi_0    pypi
tk                        8.6.10               hbc83047_0    defaults
torchvision               0.3.0           cuda92py36h0601742_0    defaults
urllib3                   1.25.10                  pypi_0    pypi
wheel                     0.34.2                   py36_0    defaults
xz                        5.2.5                h7b6447c_0    defaults
yapf                      0.30.0                   pypi_0    pypi
zlib                      1.2.11               h7b6447c_3    defaults
zstd                      1.4.5                h9ceee32_0    defaults

When I test the pretrained model vis_sipmask_r50_fpn_1x.pth (provided by you) on the valid set of the Youtube-VIS, I will get the following error message:

ERROR MESSAGE
SipMask/SipMask-VIS on  master [!+?] via 🐍 v3.6.10 :: Anaconda, Inc. via C sipmask took 15s 
➜ CUDA_VISIBLE_DEVICES=1 python tools/test_video.py ./configs/sipmask/sipmask_r50_caffe_fpn_gn_1x.py ./work_dirs/vis_sipmask_r50_fpn_1x.pth --out results.pkl --eval bbox --show
loading annotations into memory...
Done (t=0.00s)
creating index...
index created!
Traceback (most recent call last):
  File "/home/lart/miniconda3/envs/sipmask/lib/python3.6/site-packages/mmcv/runner/checkpoint.py", line 67, in load_state_dict
    own_state[name].copy_(param)
RuntimeError: The expanded size of the tensor (1) must match the existing size (3) at non-singleton dimension 3.  Target sizes: [512, 768, 1, 1].  Tensor sizes: [512, 768, 3, 3]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "tools/test_video.py", line 122, in <module>
    main()
  File "tools/test_video.py", line 91, in main
    load_checkpoint(model, args.checkpoint)
  File "/home/lart/miniconda3/envs/sipmask/lib/python3.6/site-packages/mmcv/runner/checkpoint.py", line 162, in load_checkpoint
    load_state_dict(model, state_dict, strict, logger)
  File "/home/lart/miniconda3/envs/sipmask/lib/python3.6/site-packages/mmcv/runner/checkpoint.py", line 73, in load_state_dict
    name, own_state[name].size(), param.size()))
RuntimeError: While copying the parameter named bbox_head.fovea_track.weight, whose dimensions in the model are torch.Size([512, 768, 1, 1]) and whose dimensions in the checkpoint are torch.Size([512, 768, 3, 3]).

When I change as follows, that will get even stranger error messages:

# SipMask-VIS/mmdet/models/anchor_heads/sipmask_head.py
        # self.fovea_track = nn.Conv2d(self.feat_channels * 3, 512, 1, padding=0)
        self.fovea_track = nn.Conv2d(self.feat_channels * 3, 512, 3, padding=0)
ERROR MESSAGE
SipMask/SipMask-VIS on  master [!+?] via 🐍 v3.6.10 :: Anaconda, Inc. via C sipmask took 30s 
➜ CUDA_VISIBLE_DEVICES=1 python tools/test_video.py ./configs/sipmask/sipmask_r50_caffe_fpn_gn_1x.py ./work_dirs/vis_sipmask_r50_fpn_1x.pth --out results.pkl --eval bbox --show
loading annotations into memory...
Done (t=0.00s)
creating index...
index created!
unexpected key in source state_dict: bbox_head.track_convs.3.conv.weight, bbox_head.track_convs.3.gn.weight, bbox_head.track_convs.3.gn.bias

[                                                  ] 0/8289, elapsed: 0s, ETA:/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [64,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [65,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [66,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [67,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [68,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [69,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [70,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [71,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [72,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [73,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [74,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [75,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [76,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [77,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [78,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [79,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [80,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [81,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [82,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [83,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [84,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [85,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [86,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [87,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [88,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [89,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [90,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [91,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [92,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [93,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [94,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [95,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [96,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [97,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [98,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [99,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [100,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [101,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [102,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [103,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [104,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [105,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [106,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [107,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [108,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [109,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [110,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [111,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [112,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [113,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [114,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [115,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [116,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [117,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [118,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [119,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [120,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [121,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [122,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [123,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [124,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [125,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [126,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [127,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [0,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [1,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [2,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [3,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [4,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [5,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [6,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [7,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [8,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [9,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [10,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [11,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [12,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [13,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [14,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [15,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [16,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [17,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [18,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [19,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [20,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [21,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [22,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [23,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [24,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [25,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [26,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [27,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [28,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [29,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [30,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [31,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [32,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [33,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [34,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [35,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [36,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [37,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [38,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [39,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [40,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [41,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [42,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [43,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [44,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [45,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [46,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [47,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [48,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [49,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [50,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [51,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [52,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [53,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [54,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [55,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [56,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [57,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [58,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [59,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [60,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [61,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [62,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [0,0,0], thread: [63,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [64,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [65,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [66,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [67,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [68,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [69,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [70,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [71,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [72,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [73,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [74,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [75,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [76,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [77,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [78,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [79,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [80,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [81,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [82,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [83,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [84,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [85,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [86,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [87,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [88,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [89,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [90,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [91,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [92,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [93,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [94,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [95,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [96,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [97,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [98,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [99,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [100,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [101,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [102,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [103,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [104,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [105,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [106,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [107,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [108,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [109,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [110,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [111,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [112,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [113,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [114,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [115,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [116,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [117,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [118,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [119,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [120,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [121,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [122,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [123,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [124,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [125,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [126,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [127,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [0,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [1,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [2,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [3,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [4,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [5,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [6,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [7,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [8,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [9,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [10,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [11,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [12,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [13,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [14,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [15,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [16,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [17,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [18,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [19,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [20,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [21,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [22,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [23,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [24,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [25,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [26,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [27,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [28,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [29,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [30,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [31,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [32,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [33,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [34,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [35,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [36,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [37,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [38,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [39,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [40,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [41,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [42,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [43,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [44,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [45,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [46,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [47,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [48,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [49,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [50,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [51,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [52,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [53,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [54,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [55,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [56,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [57,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [58,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [59,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [60,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [61,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [62,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/tmp/pip-req-build-xea91aoh/aten/src/ATen/native/cuda/IndexKernel.cu:53: lambda [](int)->auto::operator()(int)->auto: block: [2,0,0], thread: [63,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
Traceback (most recent call last):
  File "tools/test_video.py", line 122, in <module>
    main()
  File "tools/test_video.py", line 104, in main
    outputs = single_test(model, data_loader, False, save_path=args.save_path)
  File "tools/test_video.py", line 23, in single_test
    result = model(return_loss=False, rescale=not show, **data)
  File "/home/lart/miniconda3/envs/sipmask/lib/python3.6/site-packages/torch/nn/modules/module.py", line 493, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/lart/miniconda3/envs/sipmask/lib/python3.6/site-packages/torch/nn/parallel/data_parallel.py", line 150, in forward
    return self.module(*inputs[0], **kwargs[0])
  File "/home/lart/miniconda3/envs/sipmask/lib/python3.6/site-packages/torch/nn/modules/module.py", line 493, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/lart/Coding/VISProj/SipMask/SipMask-VIS/tools/../mmdet/core/fp16/decorators.py", line 49, in new_func
    return old_func(*args, **kwargs)
  File "/home/lart/Coding/VISProj/SipMask/SipMask-VIS/tools/../mmdet/models/detectors/base.py", line 88, in forward
    return self.forward_test(img, img_meta, **kwargs)
  File "/home/lart/Coding/VISProj/SipMask/SipMask-VIS/tools/../mmdet/models/detectors/base.py", line 79, in forward_test
    return self.simple_test(imgs[0], img_metas[0], **kwargs)
  File "/home/lart/Coding/VISProj/SipMask/SipMask-VIS/tools/../mmdet/models/detectors/single_stage.py", line 73, in simple_test
    bbox_list = self.bbox_head.get_bboxes(*bbox_inputs)
  File "/home/lart/Coding/VISProj/SipMask/SipMask-VIS/tools/../mmdet/core/fp16/decorators.py", line 127, in new_func
    return old_func(*args, **kwargs)
  File "/home/lart/Coding/VISProj/SipMask/SipMask-VIS/tools/../mmdet/models/anchor_heads/sipmask_head.py", line 674, in get_bboxes
    mask = masks[i].cpu().numpy()
RuntimeError: CUDA error: device-side assert triggered
terminate called after throwing an instance of 'c10::Error'
  what():  CUDA error: device-side assert triggered (insert_events at ../c10/cuda/CUDACachingAllocator.cpp:564)
frame #0: c10::Error::Error(c10::SourceLocation, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) + 0x6d (0x7fec585d933d in /home/lart/miniconda3/envs/sipmask/lib/python3.6/site-packages/torch/lib/libc10.so)
frame #1: <unknown function> + 0x11ea1 (0x7fec585fdea1 in /home/lart/miniconda3/envs/sipmask/lib/python3.6/site-packages/torch/lib/libc10_cuda.so)
frame #2: c10::TensorImpl::release_resources() + 0x5c (0x7fec585cb91c in /home/lart/miniconda3/envs/sipmask/lib/python3.6/site-packages/torch/lib/libc10.so)
frame #3: torch::autograd::Variable::Impl::release_resources() + 0x7c (0x7fec5bfdb17c in /home/lart/miniconda3/envs/sipmask/lib/python3.6/site-packages/torch/lib/libtorch.so.1)
frame #4: <unknown function> + 0x124c84 (0x7fec808a7c84 in /home/lart/miniconda3/envs/sipmask/lib/python3.6/site-packages/torch/lib/libtorch_python.so)
frame #5: <unknown function> + 0x35467c (0x7fec80ad767c in /home/lart/miniconda3/envs/sipmask/lib/python3.6/site-packages/torch/lib/libtorch_python.so)
frame #6: <unknown function> + 0x3546b2 (0x7fec80ad76b2 in /home/lart/miniconda3/envs/sipmask/lib/python3.6/site-packages/torch/lib/libtorch_python.so)
<omitting python frames>
frame #23: __libc_start_main + 0xe7 (0x7fec8f26bb97 in /lib/x86_64-linux-gnu/libc.so.6)

Can you help me solve this problem?

Inference visualization error using mmdetection

Hello,
I congratulate you on your work, it is great, i have some errors while visualizing the predictions using the test.py script as well as the high level API by mmdetection, the error states that key error=results[ann_info], it seems like in test model, the build dataset is not making a ann_info list, can you guide me which script you used for the testing images using sipmask, i have got the evaluation results but saving images and viewing them returns an error, thanks in advance

ImportError: cannot import name 'deform_conv_cuda'

Traceback (most recent call last):
File "./tools/test_video.py", line 10, in
from mmdet import datasets
File "/ytvis/SipMask/SipMask-VIS/tools/../mmdet/datasets/init.py", line 2, in
from .cityscapes import CityscapesDataset
File "/ytvis/SipMask/SipMask-VIS/tools/../mmdet/datasets/cityscapes.py", line 1, in
from .coco import CocoDataset
File "/ytvis/SipMask/SipMask-VIS/tools/../mmdet/datasets/coco.py", line 4, in
from .custom import CustomDataset
File "/ytvis/SipMask/SipMask-VIS/tools/../mmdet/datasets/custom.py", line 10, in
from .extra_aug import ExtraAugmentation
File "/ytvis/SipMask/SipMask-VIS/tools/../mmdet/datasets/extra_aug.py", line 5, in
from mmdet.core.evaluation.bbox_overlaps import bbox_overlaps
File "/ytvis/SipMask/SipMask-VIS/tools/../mmdet/core/init.py", line 6, in
from .post_processing import * # noqa: F401, F403
File "/ytvis/SipMask/SipMask-VIS/tools/../mmdet/core/post_processing/init.py", line 1, in
from .bbox_nms import multiclass_nms, multiclass_nms_idx
File "/ytvis/SipMask/SipMask-VIS/tools/../mmdet/core/post_processing/bbox_nms.py", line 3, in
from mmdet.ops.nms import nms_wrapper
File "/ytvis/SipMask/SipMask-VIS/tools/../mmdet/ops/init.py", line 2, in
from .dcn import (DeformConv, DeformConvPack, DeformRoIPooling,
File "/ytvis/SipMask/SipMask-VIS/tools/../mmdet/ops/dcn/init.py", line 1, in
from .deform_conv import (DeformConv, DeformConvPack, ModulatedDeformConv,
File "/ytvis/SipMask/SipMask-VIS/tools/../mmdet/ops/dcn/deform_conv.py", line 9, in
from . import deform_conv_cuda
ImportError: cannot import name 'deform_conv_cuda'

I tried to install mmdet using cmd and dockerfile given in SipMask-VIS repo while all come into this error. This error should because the mmdet.ops has been deleted and moved the functions to mmcv.ops. Since you use a customed mmdet file in SipMask-VIS, I cannot use the recently released mmcv. Could you tell me which mmcv you used or update the docker? So that I can install it either. Thank you!

About CropSplit and CropSplitGt in mmdet.ops

Hi, When I use your code, I found one question about CropSplit and CropSplitGt in mmdet.ops. May I ask the function of the CropSplit. I found that you define self.crop_cuda = CropSplit(2) and self.crop_gt_cuda = CropSplitGt(2) at first, so what's the meaning of "2"? I have print the input feature map size of pred_masks = self.crop_cuda(pred_masks, bbox_dt), which is "[4,288,288, number of coefficient]", and output size is [288,288,number of coefficient]. May I ask what have done in CropSplit?

Batched inference support

Does the model support batched inference support?

I removed the 1 image per GPU constraint and sent a tensor of shape [32, 3, 704, 1344] into the model (SipMask-mmdetection/mmdet/configs/sipmask/sipmask_r101_caffe_fpn_gn_ms_4x.py), but the output from the model is the same for shapes [32, 3, 704, 1344] and [1, 3, 704, 1344]

Sipmask-VIS :output_size of sipmask_r50_caffe_1x.pth do not match

First thanks for providing the code of this wonderful work!
Here's the problem.
When loading from './../SipMask-mmdetection/work_dirs/sipmask_r50_caffe_1x.pth',
the output size of this pretrained model do not match it of the VIS model.Since the number of classes of the pretrained model is **80,**different from it (which is 40) of the VIS model.
Please tell me how to deal with this problem. Thank you!!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.