cudnn support about cudarc HOT 4 CLOSED

or just let each thread spawn a new handle (no data is associated to a handle)

from cudarc.

Re thread safety I think if we enforce that the handles are Arc<Mutex<CudnnHandle>>, we should be fine

from cudarc.

@M1ngXU let's split the cudnn pr into a couple separate PRs, so we just have what we need for dfdx:

1. Add sys.rs/bindgen.sh/result.rs & link to cudnn in build.rs (you already have this in the big PR, just need to split it out)
   1. i think to start we just need to create/destroy handle, cudnnSetStream
2. Add tensor descriptors cudnnCreateTensorDescriptor, cudnnDestroyTensorDescriptor, cudnnSetTensorNdDescriptor
3. Add reduction support
4. Add conv2d support
5. Add pool2d support

from cudarc.

@M1ngXU let's split the cudnn pr into a couple separate PRs, so we just have what we need for dfdx:

1. Add sys.rs/bindgen.sh/result.rs & link to cudnn in build.rs (you already have this in the big PR, just need to split it out)

   1. i think to start we just need to create/destroy handle, cudnnSetStream

2. Add tensor descriptors cudnnCreateTensorDescriptor, cudnnDestroyTensorDescriptor, cudnnSetTensorNdDescriptor

3. Add reduction support

4. Add conv2d support

5. Add pool2d support

ok, i'll do number one now

from cudarc.