代码能够正常运行，但显存不足的问题仍未解决 about balanced-dataparallel HOT 12 CLOSED

batch_szie = 100
gpu0_bsz = 50

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 456.81       Driver Version: 456.81       CUDA Version: 11.1     |
|-------------------------------+----------------------+----------------------+
| GPU  Name            TCC/WDDM | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  TITAN Xp           WDDM  | 00000000:AF:00.0  On |                  N/A |
| 50%   81C    P2   253W / 250W |   9340MiB / 12288MiB |     75%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
|   1  TITAN Xp           WDDM  | 00000000:D8:00.0 Off |                  N/A |
| 26%   49C    P2    61W / 250W |   1649MiB / 12288MiB |      3%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

len(inputs):  2
self.device_ids[:len(inputs)] [0, 1]
replicas: 2
        Loss:   55605.012
start predicting:  2020-11-10 15:07:37.934568
len(inputs):  2
self.device_ids[:len(inputs)] [0, 1]
replicas: 2
len(inputs):  2
self.device_ids[:len(inputs)] [0, 1]
replicas: 2
Traceback (most recent call last):
  File "e:/pythonFile/chengzuo/TASGC/src/main.py", line 104, in <module>    
    main()
  File "e:/pythonFile/chengzuo/TASGC/src/main.py", line 80, in main
    hit, mrr = train_test(model, train_data, test_data)
  File "e:\pythonFile\chengzuo\TASGC\src\model.py", line 278, in train_test 
    targets, scores = forward(model, i, test_data)
  File "e:\pythonFile\chengzuo\TASGC\src\model.py", line 238, in forward    
    return targets, model.module.compute_scores(seq_hidden, mask)
  File "e:\pythonFile\chengzuo\TASGC\src\model.py", line 108, in compute_scores
    scores = torch.sum(a * b, -1)  # b,n
RuntimeError: CUDA out of memory. Tried to allocate 1.61 GiB (GPU 0; 12.00 GiB total capacity; 7.49 GiB already allocated; 820.44 MiB free; 8.73 GiB reserved in total by PyTorch)

set CUDA_VISIBLE_DEVICES=0,1 #设置
python main.py #然后再运行自己的程序

os.environ["CUDA_VISIBLE_DEVICES"] = '1, 0'