Comments (7)
RS-Coop
commented on July 26, 2024
1
Is #41 referencing the Discourse thread that I reference here?
from nnlibcuda.jl.
DhairyaLGandhi
commented on July 26, 2024
Most ops are expected to be AD'd and hence don't require rules. Ideally, we would not hold onto rrules either, relying on the AD to compose based on lower level ops where appropriate. It's not an issue with the rules not existing, from my perspective. Are you trying to do higher order differentiation?
from nnlibcuda.jl.
RS-Coop
commented on July 26, 2024
Yes, Hessian vector products using mixed forward over backward.
from nnlibcuda.jl.
ToucheSir
commented on July 26, 2024
I don't think this is because of a lack of frules (neither Zygote nor ForwardDiff use them), but some explicit rrule overrides. Opened #41 yesterday to track just that.
from nnlibcuda.jl.
RS-Coop
commented on July 26, 2024
Do you want me to leave this up given that the error is indentified incorrectly and the true issue is being tracked?
from nnlibcuda.jl.
DhairyaLGandhi
commented on July 26, 2024
Yeah I think we can close this for now
from nnlibcuda.jl.
RS-Coop
commented on July 26, 2024
Okay!
from nnlibcuda.jl.
Related Issues (17)
- CUDNN bad param for backward pass with Float16 HOT 4
- `batched_gemm!` with CuArrays doesn't work HOT 3
- BatchNorm causes error during gradient computation HOT 7
- gpu `scatter` with `CartesianIndex` not supported HOT 2
- Slow ∇softmax! compared with generic version. HOT 15
- Interesting resources for optimizing various ops
- NNlibCUDA Heisenbug in conv! with nonzero beta HOT 7
- TagBot trigger issue HOT 20
- Do we need src/cudnn/activations.jl?
- conv on complex numbers
- Unconstrained element type on activations causing errors with e.g. Complex, ForwardDiff.Dual HOT 2
- `relu` propagates NaN on CPU but not on GPU HOT 7
- Maxpool misbehaving in some edge cases HOT 4
- SVectors on GPU CuArray cannot use index from CuArray HOT 2
- Disable `CUDNN_SOFTMAX_FAST` or use a separate math mode variable for `softmax` HOT 1
- Float16 CUDA `conv` broken on 5D tensors HOT 8
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from nnlibcuda.jl.