Will ReverseDiff.jl support branching? about reversediff.jl HOT 3 CLOSED

I imagine the best thing to do at the moment is to splice in ForwardDiff for these.

from reversediff.jl.

Sorry, do you mean using ReverseDiff.@forward? I didn't know about it but it seems to be working well:

julia> using ReverseDiff

julia> ReverseDiff.@forward elu(x) = x > 0 ? x : expm1(x)
elu (generic function with 1 method)

julia> x = ones(3)
3-element Array{Float64,1}:
 1.0
 1.0
 1.0

julia> ∇elu! = ReverseDiff.compile_gradient(x -> sum(elu.(x)), x)
(::#301) (generic function with 1 method)

julia> ∇elu!(similar(x), x)
3-element Array{Float64,1}:
 1.0
 1.0
 1.0

julia> ∇elu!(similar(x), -x)
3-element Array{Float64,1}:
 0.367879
 0.367879
 0.367879

from reversediff.jl.

ReverseDiff.jl's re-recording API (gradient/gradient!, jacobian/jacobian!, etc.) does support branching, so that's what you should use if your function branches on differentiable input. The performance of this should be similar to (if not better than) other autograd-like approaches.

If you would like to use the pre-recording API (ReverseDiff.compile_gradient!, ReverseDiff.compile_jacobian! etc.), then you can splice the branch into ForwardDiff using @forward (as you've discovered).

In the future, we plan to provide an @source macro which will allow you to splice reverse-mode source-to-source AD, which may be able to handle simple branches.

from reversediff.jl.