Deadlock during memory free about cuarrays.jl HOT 5 CLOSED

In reference to #691, changing allocated_lock in src/memory/binned.jl to a ReentrantLock appears to alleviate this issue, and it's not too much slower:

julia> l1=Threads.SpinLock();

julia> l2=ReentrantLock();

julia> @btime Base.@lock $l1 begin end
  20.569 ns (0 allocations: 0 bytes)

julia> @btime Base.@lock $l2 begin end
  37.242 ns (0 allocations: 0 bytes)

Perhaps changing all/some of the locks in CuArrays to a ReentrantLock could serve as a stopgap?

from cuarrays.jl.

Perhaps changing all/some of the locks in CuArrays to a ReentrantLock could serve as a stopgap?

No, the code does not support that. I'll have a look.

from cuarrays.jl.

Ah I thought they'd be interchangeable. Could you elaborate why the code wouldn't support a ReentrantLock?

from cuarrays.jl.

A reentrant lock can be taken by the same task. But in this case, we're protecting against concurrent access of datastructures that are used in regular code as well as from finalizers. So while we're modifying the map and have the lock, a GC run could trigger our finalizer that, potentially on the same task, will be able to acquire the lock and start modifying the same datastructure too.

from cuarrays.jl.

I see. So there's no guarantee that finalizes are run in a separate task and swapping allocated_lock to a ReentrantLock only masked that underlying problem. Thanks for clarifying! I tried to grok the code in gc.c/task.c etc. but.. there's a lot there :)

from cuarrays.jl.