Comments (9)
Hey @osofr, I will take a look at this today and see what I can do. The egregiousness of the performance hit is actually a good thing here, because something very simple has gone wrong; most likely a kernel just slipped through the cracks.
So just to clarify, the slowdown caused by a statement of the form x[i,:] = ...
, right, but the forward pass x[i,:]
is fine? Is this call falling back to scalar indexing (it will error if you call CuArrays.allowscalar(false)
beforehand)?
from cuarrays.jl.
Thank you so much @MikeInnes! Yes, this is exactly right. The forward pass is fine and is much faster on GPU. It errors out when setting CuArrays.allowscalar(false)
, at least the last example below. I haven't checked if it will also error out when calling back
, but I am fairly certain this is the case. Thank you so much for your help! I wish I knew how to fix this myself, will hopefully learn something from your fix.
xg = rand(2,100) |> gpu;
repl = zeros(100) |> gpu;
@time xg[1,:] = repl;
from cuarrays.jl.
Ok, so this is happening just because we don't have a setindex kernel on the GPU; pinging @SimonDanisch who can hopefully implement one in GPUArrays fairly quickly.
from cuarrays.jl.
Ok, thanks!
@SimonDanisch, anything I can do to expedite this? Can this kernel be borrowed from somewhere? Am I the first person to try to overwrite a row/column in GPU array?
from cuarrays.jl.
Can you try Pkg.update()
? Looks like this was fixed on master and just not tagged yet.
from cuarrays.jl.
Ok, fairly certain I tried this yesterday, but will do so right now. Just to confirm, I should update GPUArrays
? Will take me a few minutes to spin up GPU instance, but will get back shortly.
from cuarrays.jl.
Yeah, GPUArrays has the fix. If not, would be useful to narrow down what the setindex
is being called with.
from cuarrays.jl.
Woohoo, it works! Not as fast as CPU, but it will do. I will re-run my Flux training code, just to confirm that the performance has improved and will close the issue then. Just out of curiosity, would you mind pointing me to the GPUArrays
commit that fixes this setindex issue? I'd like to understand it for future hacking. Thank you so much @MikeInnes!
julia> using Flux, CuArrays
julia> testx = rand(2,100);
julia> x = param(testx);
julia> idx = (1,:); # (1, Colon())
julia> l = Flux.getindex(x,idx...);
julia> l2 = sum(l);
julia> Flux.back!(l2);
julia> @time Flux.back!(l2);
0.000011 seconds (9 allocations: 2.844 KiB)
julia> xg = param(testx) |> gpu;
julia> l = Flux.getindex(xg,idx...);
julia> l2 = sum(l);
julia> Flux.back!(l2);
julia> @time Flux.back!(l2);
0.000246 seconds (104 allocations: 2.844 KiB)
from cuarrays.jl.
I think you're just timing kernel launch overhead there, which is significant when the size of the array is small. Hopefully the times for the overall model are much better.
Here's the GPU code for setindex!
. Hacking on GPU kernels in Julia is pretty easy and straightforward now, so I'd definitely recommend playing around with it, maybe via CUDAnative.
from cuarrays.jl.
Related Issues (20)
- similar(PermutedDimsArray(::CuArray)) isa Array HOT 1
- In CuArrays v2.0, GPU operation takes hours to run for the first time HOT 5
- sum!(y::CuVector, x::CuMatrix) throws InvalidIRError error
- Where can I find
- Where can I find All the using instructions of CuArrays? HOT 3
- add implicit float conversion to math functions HOT 4
- Multiplication between mixed types doesn't drop leading dimensions HOT 2
- Very slow 4D broadcast in 2.0.1 HOT 1
- Failed to detect installed CUDA version. HOT 1
- Sum function is slow HOT 8
- CURAND_STATUS_PREEXISTING_FAILURE with v2.0.1 but not v1.7.3 HOT 8
- Deadlock during memory free HOT 5
- Indexing CuArrays with Empty Ranges Errors HOT 5
- Sum, any, etc. with function is no longer implemented HOT 7
- Training Halts when Using CuArrarys HOT 6
- CUBLAS initialization HOT 1
- Performance issue with v2.1.0 compared with v1.7.3 HOT 4
- .+ CartesianIndices: InvalidIRError: compiling kernel broadcast HOT 1
- Package fails to load HOT 4
- Project.toml becoming stale (many notable package downgrades) HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from cuarrays.jl.