liorsinai / denoisingdiffusion.jl Goto Github PK

Hi @LiorSinai,

Thank you for sharing your implementation of denoising diffusion probabilistic models. I'm excited to see this being available in Julia!

This isn't an issue but rather a request for clarification: when I went over your implementation of GaussianDiffusion, I was surprised to see that your sampling procedure in the reverse process is different from the one described in the original paper by Jonathan Ho et al.

Specifically, in your implementation, you compute x_start (x0 in Ho's paper) from the output of the denoising network and then use that to sample from q_{posterior}(x_{t-1} | x_t, x_0) = q(x_t | x_{t-1}, x_0) q(x_{t-1} | x_0) / q(x_t | x_0)

I cannot find a reference to this sampling strategy in the literature, so I was wondering if you could share the motivation (or a reference) for this parameterization. In particular, the original paper includes the comment:

There is also the possibility of predicting x0, but we found this to lead to worse sample quality early in our experiments.

Running this from the tutorial script:

for frac in [0.0, 0.25, 0.5, 0.75, 1]
local p
timestep = max(1, ceil(Int, frac * num_timesteps))
Xt = q_sample(diffusion, X, timestep)
p = scatter(Xt[1, :], Xt[2, :], alpha=0.5, label="",
aspectratio=:equal,
xlims = (-2, 2), ylims=(-2, 2),
title="t=$timestep"
)
push!(canvases, p)
end

I get this error:
ERROR: UndefVarError: cpu not defined
Stacktrace:
[1] q_sample(diffusion::GaussianDiffusion{Vector{Float32}}, x_start::Matrix{Float64}, timestep::Int64)
@ Main ./REPL[44]:1
[2] top-level scope
@ ./REPL[49]:4

Flux v0.14 introduced the breaking change of package extensions, which means some functionality will not be unloaded unless other packages are loaded. Flux v0.14 uses this for its GPU functionality so that it can be loaded faster when GPUs are not used. So for example, CUDA functionality will not be loaded unless CUDA is explicitly loaded.

If you used Flux 0.14 you may encounter this error:

ERROR: UndefVarError: CUDA not defined

Temporary resolution: downgrade to Flux 0.13 OR replace using Flux.CUDA with using CUDA, cuDNN.

The upgrade to version 0.4 (Julia 1.9 and Flux 0.14) has broken loading old models with BSON.

One issue is that Flux.GroupNorm was changed. See FluxML/Flux.jl@5d3eaa9.

That can be fixed with a local version of BSON and an altered version of constructtype:

function constructtype(T, Ts) 
  if (Symbol(T) == Symbol("Flux.GroupNorm"))
   return T{map(normalize_typeparams, Ts[1:3])...}
  end
  (length(Ts) == 0) ? T : T{map(normalize_typeparams, Ts)...}
end

However there is then another error:

ERROR: BoundsError: attempt to access DataType at index [1]

I have been unable to fix this.