Is Measurements.jl compatible with Optim.jl and ForwardDiff.jl, or am I facing a gotcha? about measurements.jl HOT 12 OPEN

I'm aware of some issues with Optim.jl, but due to a bug in that package: JuliaNLSolvers/Optim.jl#823. I believe Measurements.jl can't work with ForwardDiff because of the limitations of the latter package.

In the end, I don't think there is anything I can do here

from measurements.jl.

Can't believe I missed that post in the Optim page! Thanks for pointing this out! It's helpful as it tells me that this is a fundamental bug and not something I'm doing incorrectly when feeding it in.

For anyone else struggling with this: I'm attempting the following workaround (as long as your main case allows for analytical solutions that work with Measurements.jl)

1. Define a second constructor for using Measurement types with
2. Strip all Measurement types to only their values via measurement.val
3. Optimize the value normally (no measurement types)
4. Define a new optimizer for the uncertainty component only (now that you've solved the actual value)
   a. The optimized function should only take in the uncertainty, the value should be static now.
5. Use a black box optimizer to solve the uncertainty. I'm going to attempt this with BlackBoxOptim.jl
6. Reconstruct the solved Measurement with the solved value and uncertainty

from measurements.jl.

@giordano A few of us from the Humans of Julia discord noticed that the line 92 in src/Measurements.jl is the culprit in the ForwardDiff dual numbers. Do you know about this?

When using autodiff=true in Optim.jl, the following loop occurs

ERROR: LoadError: StackOverflowError:
Stacktrace:
 [1] measurement(val::ForwardDiff.Dual{ForwardDiff.Tag{Main.PengRobinson.PRVolumeSolver.var"#target_function#1"{StaticArrays.SVector{2, Main.PengRobinson.PengRobinsonChemical{Float64, Float64, Float64, Float64, Float64}}, StaticArrays.SVector{2, Measurement{Float64}}, StaticArrays.SMatrix{2, 2, Float64, 4}, Float64, Float64, Measurement{Float64}}, Float64}, Float64, 1}, err::ForwardDiff.Dual{ForwardDiff.Tag{Main.PengRobinson.PRVolumeSolver.var"#target_function#1"{StaticArrays.SVector{2, Main.PengRobinson.PengRobinsonChemical{Float64, Float64, Float64, Float64, Float64}}, StaticArrays.SVector{2, Measurement{Float64}}, StaticArrays.SMatrix{2, 2, Float64, 4}, Float64, Float64, Measurement{Float64}}, Float64}, Float64, 1}) (repeats 79984 times)
   @ Measurements ~\.julia\dev\Measurements\src\Measurements.jl:92
in expression starting at c:\git\julia-polymer-membranes\PolymerMembranes.jl\test.jl:30

In the case that you use Optim.jl's finite differencing, the issue doesn't occur, so it seems the issue is that line calling itself until the Stack Overflow Error happens. If you want to know more about what I was doing / talk to us over at HoJ, I'm more than happy to explain it!

from measurements.jl.

That's not the problem, the culprit is ForwardDiff being unable to deal with anything more specific than Real, the StackOverflowError you see is only of symptom of their limitation. Quoting from the documentation of ForwardDiff

The target function must be written generically enough to accept numbers of type T<:Real as input (or arrays of these numbers). The function doesn't require a specific type signature, as long as the type signature is generic enough to avoid breaking this rule. This also means that any storage assigned used within the function must be generic as well (see this comment for an example).

But Measurement <: AbstractFloat and I'm not going to change the subtyping.

from measurements.jl.

That's completely understandable, and thanks for the quick responses!

So far, it seems that using Optim.jl while breaking the components into uncertainties and values and solving them independently works well, do you have any advice about how I might go about splitting a measurement object into this without running into issues?

Applying the same technique to other optimization libraries like BlackBoxOptim.jl, which (shouldn't) do any autodiff-like things with the functions creates very strange results. There, the same stackoverflow issue happens without even doing anything other than just using Measurements in the same code. Here's a tiny (ish) example done on Julia 1.6.0.

using BlackBoxOptim
# using Measurements  # uncomment this after confirming it runs correctly
function solve_volume_uncertainty_bbox(p_uncertainty, v_value, t)
    function target_function(volume_uncertainty_guess_l_mol)
        resulting_pressure = 4
        return (resulting_pressure - p_uncertainty)^2
    end
    res = bboptimize(target_function; SearchRange=(0.0, 10.0), NumDimensions=1, Method=:de_rand_1_bin)  
    return best_candidate(res)[1]
end

bbox = solve_volume_uncertainty_bbox(0.1, 20.08405, 273.15)
println(bbox)

It may also just be something that happens on my machine. I'm hoping that this might prove useful to you in some way, because I'm super excited that this library exists. In my field, errors are basically unseen in most literature (I'm planning on citing your library in mine as well, once I get my underlying calculations working!), as its too lengthy to do the analytical calculations by hand. The ultimate version of this would be to somehow generally extend it to optimization as well as direct or, for the lack of a better word: analytical as opposed to iterative, solutions are only a subset of the calculations done.

from measurements.jl.

That's extremely bizarre, but since Measurements isn't directly involved apart from loading it (and thus adding a new data type) my guess is that BlackBoxOptim is doing something fishy with type-inference (looking also at the output, it goes crazy when inferring the type of some containers). I'd report them this issue (and we'd be up to three packages not working correctly out-of-the-box because of their limitations 🙂)

from measurements.jl.

BTW, in JuliaNLSolvers/Optim.jl#823 I had some suggestions about how to fix Optim.jl (those are genuinely issues in the package, unrelated to Measurements.jl). If you're interested in using Optim.jl together with Measurements.jl I'd recommend you to comment on that issue to make the maintainer aware that there is interest in getting it fixed 🙂

from measurements.jl.

For the record, BlackBoxOptim seems to go haywire because of this log(::Real, ::Measurement) method

from measurements.jl.

Thanks @giordano! This is extremely helpful, how did you track down the BlackBoxOptim.jl issue to those lines? I was never quite able to narrow it down further than things happening in my own code. This was mainly because the stackoverflow error caused by importing Measurements was just a whole bunch of spam in the terminal, followed by the correct answer (Measurements propagated through correctly!) Is this what it showed up to you as? Or did you get a formal exception followed by the process exiting early?

I'll send it over to them and see if they'd be willing to work on it! Ideally we'd have at least one optimization library that works without tomfoolery in Measurements. BlackBoxOptim would be a nice one! They have the BorgMOEA algorithm implemented which can handle multiple objective functions, and would be perfect for optimizing both the value and uncertainty in a single optimization process.

from measurements.jl.

how did you track down the BlackBoxOptim.jl issue to those lines?

Binary search: I commented half of the code of Measurements.jl until I couldn't trigger the error, then repeat 🙂

from measurements.jl.

Whelp, I'm adding that to my bag of tricks. Thanks again!

from measurements.jl.

ok, im trying to make optim work with measurements, and the main show problem comes on the use of the @sprintf macro for printing

from measurements.jl.