goudetolivier / dlmcol_wvcp Goto Github PK

Hi,
Excuse me. When I run this code, I get the following error. Can you give me some advice? Thanks.

  (py3) root@gzm:/media/gzm/1430A2F330A2DACE1/graph_color/DLMCOL_WVCP# ./runscript_array_job_WVCP.sh 

23/04/2022 01:29:44 init
23/04/2022 01:29:44 cuda:0
23/04/2022 01:29:49 k : 82
23/04/2022 01:30:01 nb_conflict : 0.0
23/04/2022 01:30:01 score : 889.0
23/04/2022 01:30:01 nb_uncolored : 0
23/04/2022 01:30:01 score test : 889.0
23/04/2022 01:30:01 nb_conflict test : 0.0
23/04/2022 01:30:02 ############################
23/04/2022 01:30:02 Start TABU
23/04/2022 01:30:02 ############################
23/04/2022 01:30:03 iter tabu : 0
23/04/2022 01:30:03 stats phi
23/04/2022 01:30:03 min 1.5
23/04/2022 01:30:03 mean 1.5
23/04/2022 01:30:03 max 1.5
23/04/2022 01:30:19 Call to cuMemcpyDtoH results in CUDA_ERROR_LAUNCH_TIMEOUT
Traceback (most recent call last):
File "main.py", line 74, in
name_expe,
File "/media/gzm/1430A2F330A2DACE1/graph_color/DLMCOL_WVCP/dlmcol/main_WVCP.py", line 336, in main_WVCP
offsprings_pop = offsprings_pop_gpu_memory.copy_to_host()
File "/home/gzm/anaconda3/envs/py3/lib/python3.7/site-packages/numba/cuda/cudadrv/devices.py", line 224, in _require_cuda_context
return fn(*args, **kws)
File "/home/gzm/anaconda3/envs/py3/lib/python3.7/site-packages/numba/cuda/cudadrv/devicearray.py", line 262, in copy_to_host
stream=stream)
File "/home/gzm/anaconda3/envs/py3/lib/python3.7/site-packages/numba/cuda/cudadrv/driver.py", line 2403, in device_to_host
fn(host_pointer(dst), device_pointer(src), size, *varargs)
File "/home/gzm/anaconda3/envs/py3/lib/python3.7/site-packages/numba/cuda/cudadrv/driver.py", line 300, in safe_cuda_api_call
self._check_error(fname, retcode)
File "/home/gzm/anaconda3/envs/py3/lib/python3.7/site-packages/numba/cuda/cudadrv/driver.py", line 335, in _check_error
raise CudaAPIError(retcode, msg)
numba.cuda.cudadrv.driver.CudaAPIError: [702] Call to cuMemcpyDtoH results in CUDA_ERROR_LAUNCH_TIMEOUT
23/04/2022 01:30:19 add pending dealloc: module_unload ? bytes
23/04/2022 01:30:19 add pending dealloc: cuMemFree_v2 81920 bytes
23/04/2022 01:30:19 add pending dealloc: module_unload ? bytes
23/04/2022 01:30:19 add pending dealloc: cuMemFree_v2 4000 bytes
23/04/2022 01:30:19 add pending dealloc: cuMemFree_v2 250000 bytes
23/04/2022 01:30:19 add pending dealloc: cuMemFree_v2 327680 bytes
23/04/2022 01:30:19 add pending dealloc: cuMemFree_v2 40960000 bytes
23/04/2022 01:30:19 add pending dealloc: cuMemFree_v2 81920 bytes
23/04/2022 01:30:19 add pending dealloc: cuMemFree_v2 81920 bytes
23/04/2022 01:30:19 add pending dealloc: cuMemFree_v2 838860800 bytes
23/04/2022 01:30:19 add pending dealloc: cuMemFree_v2 838860800 bytes
23/04/2022 01:30:19 add pending dealloc: cuMemFree_v2 81920 bytes
23/04/2022 01:30:19 add pending dealloc: cuMemFree_v2 40960000 bytes
23/04/2022 01:30:19 dealloc: cuMemFree_v2 81920 bytes
23/04/2022 01:30:19 Call to cuMemFree results in CUDA_ERROR_LAUNCH_TIMEOUT
Traceback (most recent call last):
File "/home/gzm/anaconda3/envs/py3/lib/python3.7/weakref.py", line 648, in _exitfunc
f()
File "/home/gzm/anaconda3/envs/py3/lib/python3.7/weakref.py", line 572, in call
return info.func(*info.args, **(info.kwargs or {}))
File "/home/gzm/anaconda3/envs/py3/lib/python3.7/site-packages/numba/cuda/cudadrv/driver.py", line 1804, in deref
mem.free()
File "/home/gzm/anaconda3/envs/py3/lib/python3.7/site-packages/numba/cuda/cudadrv/driver.py", line 1617, in free
self._finalizer()
File "/home/gzm/anaconda3/envs/py3/lib/python3.7/weakref.py", line 572, in call
return info.func(*info.args, **(info.kwargs or {}))
File "/home/gzm/anaconda3/envs/py3/lib/python3.7/site-packages/numba/cuda/cudadrv/driver.py", line 1285, in core
deallocations.add_item(driver.cuMemFree, handle, size)
File "/home/gzm/anaconda3/envs/py3/lib/python3.7/site-packages/numba/cuda/cudadrv/driver.py", line 970, in add_item
self.clear()
File "/home/gzm/anaconda3/envs/py3/lib/python3.7/site-packages/numba/cuda/cudadrv/driver.py", line 981, in clear
dtor(handle)
File "/home/gzm/anaconda3/envs/py3/lib/python3.7/site-packages/numba/cuda/cudadrv/driver.py", line 300, in safe_cuda_api_call
self._check_error(fname, retcode)
File "/home/gzm/anaconda3/envs/py3/lib/python3.7/site-packages/numba/cuda/cudadrv/driver.py", line 335, in _check_error
raise CudaAPIError(retcode, msg)
numba.cuda.cudadrv.driver.CudaAPIError: [702] Call to cuMemFree results in CUDA_ERROR_LAUNCH_TIMEOUT

Best.