sciml / truncatedstacktraces.jl Goto Github PK

28.0 4.0 3.0 65 KB

Simpler stacktraces for the Julia Programming Language

License: MIT License

Julia 100.00%

truncatedstacktraces.jl's Introduction

TruncatedStacktraces.jl: Truncated and Simpler Stacktraces for the Julia Programming Language

Don't you wish Julia stacktraces were simpler? Introducing TruncatedStacktraces.jl! The purpose of this package is to give package authors a single uniform system for implementing truncation of type printing in stack traces.

Note Starting v1.10 a similar feature is inbuilt into julia. Starting julia v1.10, this package does nothing!

Enabling TruncatedStacktraces.jl

TruncatedStacktraces.jl is currently disabled by default, as it causes invalidations which will slow down package loading.

It can be enabled using Preferences.jl. To enable it, create a LocalPreferences.toml with the following entry:

[TruncatedStacktraces]
disable = false

Alternatively, you can generate the LocalPreferences.toml using:

using Preferences, UUIDs

using TruncatedStacktraces
Preferences.set_preferences!(TruncatedStacktraces, "disable" => false)

# OR if you don't want to load TruncatedStacktraces.jl

Preferences.set_preferences!(UUID("781d530d-4396-4725-bb49-402e4bee1e77"), "disable" => false)

In either case, you need to reload your packages (depending on TruncatedStacktraces) for the change to take effect.

TruncatedStacktraces is known to create invalidations, to remove these simply set the preference to disable it!

Users: How to Interact with TruncatedStacktraces.jl

If a package you are using is making use of TruncatedStacktraces.jl, you will see shorter stack traces. Everything is easier to read by default! This looks like:

 [14] initialize!(integrator::ODEIntegrator{true, Tsit5{Static.False, …}, Vector{Float64}, Float64, …}, cache::Tsit5Cache{Vector{Float64}, …})
    @ OrdinaryDiffEq C:\Users\accou\.julia\packages\OrdinaryDiffEq\0Pm1I\src\perform_step\low_order_rk_perform_step.jl:766

But if you want to see the type in full glory, say to share with developers on Discourse, then you can opt to show the entire stacktrace via simply running:

TruncatedStacktraces.VERBOSE[] = true

then if you run the code to error again, it will print out exactly what everyone wants to read:

 [14] initialize!(integrator::OrdinaryDiffEq.ODEIntegrator{Tsit5{typeof(OrdinaryDiffEq.trivial_limiter!), typeof(OrdinaryDiffEq.trivial_limiter!), Static.False}, true, Vector{Float64}, Nothing, Float64, SciMLBase.NullParameters, Float64, Float64, Float64, Float64, Vector{Vector{Float64}}, ODESolution{Float64, 2, Vector{Vector{Float64}}, Nothing, Nothing, Vector{Float64}, Vector{Vector{Vector{Float64}}}, ODEProblem{Vector{Float64}, Tuple{Float64, Float64}, true, SciMLBase.NullParameters, ODEFunction{true, SciMLBase.AutoSpecialize, FunctionWrappersWrappers.FunctionWrappersWrapper{Tuple{FunctionWrappers.FunctionWrapper{Nothing, Tuple{Vector{Float64}, Vector{Float64}, SciMLBase.NullParameters, Float64}}, FunctionWrappers.FunctionWrapper{Nothing, Tuple{Vector{ForwardDiff.Dual{ForwardDiff.Tag{DiffEqBase.OrdinaryDiffEqTag, Float64}, Float64, 1}}, Vector{ForwardDiff.Dual{ForwardDiff.Tag{DiffEqBase.OrdinaryDiffEqTag, Float64}, Float64, 1}}, SciMLBase.NullParameters, Float64}}, FunctionWrappers.FunctionWrapper{Nothing, Tuple{Vector{ForwardDiff.Dual{ForwardDiff.Tag{DiffEqBase.OrdinaryDiffEqTag, Float64}, Float64, 1}}, Vector{Float64}, SciMLBase.NullParameters, ForwardDiff.Dual{ForwardDiff.Tag{DiffEqBase.OrdinaryDiffEqTag, Float64}, Float64, 1}}}, FunctionWrappers.FunctionWrapper{Nothing, Tuple{Vector{ForwardDiff.Dual{ForwardDiff.Tag{DiffEqBase.OrdinaryDiffEqTag, Float64}, Float64, 1}}, Vector{ForwardDiff.Dual{ForwardDiff.Tag{DiffEqBase.OrdinaryDiffEqTag, Float64}, Float64, 1}}, SciMLBase.NullParameters, ForwardDiff.Dual{ForwardDiff.Tag{DiffEqBase.OrdinaryDiffEqTag, Float64}, Float64, 1}}}}, false}, LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED), Nothing, Nothing}, Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}, SciMLBase.StandardODEProblem}, Tsit5{typeof(OrdinaryDiffEq.trivial_limiter!), typeof(OrdinaryDiffEq.trivial_limiter!), Static.False}, OrdinaryDiffEq.InterpolationData{ODEFunction{true, SciMLBase.AutoSpecialize, FunctionWrappersWrappers.FunctionWrappersWrapper{Tuple{FunctionWrappers.FunctionWrapper{Nothing, Tuple{Vector{Float64}, Vector{Float64}, SciMLBase.NullParameters, Float64}}, FunctionWrappers.FunctionWrapper{Nothing, Tuple{Vector{ForwardDiff.Dual{ForwardDiff.Tag{DiffEqBase.OrdinaryDiffEqTag, Float64}, Float64, 1}}, Vector{ForwardDiff.Dual{ForwardDiff.Tag{DiffEqBase.OrdinaryDiffEqTag, Float64}, Float64, 1}}, SciMLBase.NullParameters, Float64}}, FunctionWrappers.FunctionWrapper{Nothing, Tuple{Vector{ForwardDiff.Dual{ForwardDiff.Tag{DiffEqBase.OrdinaryDiffEqTag, Float64}, Float64, 1}}, Vector{Float64}, SciMLBase.NullParameters, ForwardDiff.Dual{ForwardDiff.Tag{DiffEqBase.OrdinaryDiffEqTag, Float64}, Float64, 1}}}, FunctionWrappers.FunctionWrapper{Nothing, Tuple{Vector{ForwardDiff.Dual{ForwardDiff.Tag{DiffEqBase.OrdinaryDiffEqTag, Float64}, Float64, 1}}, Vector{ForwardDiff.Dual{ForwardDiff.Tag{DiffEqBase.OrdinaryDiffEqTag, Float64}, Float64, 1}}, SciMLBase.NullParameters, ForwardDiff.Dual{ForwardDiff.Tag{DiffEqBase.OrdinaryDiffEqTag, Float64}, Float64, 1}}}}, false}, LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED), Nothing, Nothing}, Vector{Vector{Float64}}, Vector{Float64}, Vector{Vector{Vector{Float64}}}, OrdinaryDiffEq.Tsit5Cache{Vector{Float64}, Vector{Float64}, Vector{Float64}, typeof(OrdinaryDiffEq.trivial_limiter!), typeof(OrdinaryDiffEq.trivial_limiter!), Static.False}}, DiffEqBase.DEStats, Nothing}, ODEFunction{true, SciMLBase.AutoSpecialize, FunctionWrappersWrappers.FunctionWrappersWrapper{Tuple{FunctionWrappers.FunctionWrapper{Nothing, Tuple{Vector{Float64}, Vector{Float64}, SciMLBase.NullParameters, Float64}}, FunctionWrappers.FunctionWrapper{Nothing, Tuple{Vector{ForwardDiff.Dual{ForwardDiff.Tag{DiffEqBase.OrdinaryDiffEqTag, Float64}, Float64, 1}}, Vector{ForwardDiff.Dual{ForwardDiff.Tag{DiffEqBase.OrdinaryDiffEqTag, Float64}, Float64, 1}}, SciMLBase.NullParameters, Float64}}, FunctionWrappers.FunctionWrapper{Nothing, Tuple{Vector{ForwardDiff.Dual{ForwardDiff.Tag{DiffEqBase.OrdinaryDiffEqTag, Float64}, Float64, 1}}, Vector{Float64}, SciMLBase.NullParameters, ForwardDiff.Dual{ForwardDiff.Tag{DiffEqBase.OrdinaryDiffEqTag, Float64}, Float64, 1}}}, FunctionWrappers.FunctionWrapper{Nothing, Tuple{Vector{ForwardDiff.Dual{ForwardDiff.Tag{DiffEqBase.OrdinaryDiffEqTag, Float64}, Float64, 1}}, Vector{ForwardDiff.Dual{ForwardDiff.Tag{DiffEqBase.OrdinaryDiffEqTag, Float64}, Float64, 1}}, SciMLBase.NullParameters, ForwardDiff.Dual{ForwardDiff.Tag{DiffEqBase.OrdinaryDiffEqTag, Float64}, Float64, 1}}}}, false}, LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED), Nothing, Nothing}, OrdinaryDiffEq.Tsit5Cache{Vector{Float64}, Vector{Float64}, Vector{Float64}, typeof(OrdinaryDiffEq.trivial_limiter!), typeof(OrdinaryDiffEq.trivial_limiter!), Static.False}, OrdinaryDiffEq.DEOptions{Float64, Float64, Float64, Float64, PIController{Rational{Int64}}, typeof(DiffEqBase.ODE_DEFAULT_NORM), typeof(LinearAlgebra.opnorm), Nothing, CallbackSet{Tuple{}, Tuple{}}, typeof(DiffEqBase.ODE_DEFAULT_ISOUTOFDOMAIN), typeof(DiffEqBase.ODE_DEFAULT_PROG_MESSAGE), typeof(DiffEqBase.ODE_DEFAULT_UNSTABLE_CHECK), DataStructures.BinaryHeap{Float64, DataStructures.FasterForward}, DataStructures.BinaryHeap{Float64, DataStructures.FasterForward}, Nothing, Nothing, Int64, Tuple{}, Tuple{}, Tuple{}}, Vector{Float64}, Float64, Nothing, OrdinaryDiffEq.DefaultInit}, cache::OrdinaryDiffEq.Tsit5Cache{Vector{Float64}, Vector{Float64}, Vector{Float64}, typeof(OrdinaryDiffEq.trivial_limiter!), typeof(OrdinaryDiffEq.trivial_limiter!), Static.False})
    @ OrdinaryDiffEq C:\Users\accou\.julia\packages\OrdinaryDiffEq\0Pm1I\src\perform_step\low_order_rk_perform_step.jl:766

Beautiful. You can turn it back into the not beautiful short stacktrace with the command:

TruncatedStacktraces.VERBOSE[] = false

How to Opt A Package Into TruncatedStacktraces.jl

Opting into TruncatedStacktraces.jl is easy: for every type that you want to omit the printing of something, use the macro TruncatedStacktraces.@truncate_stacktrace like:

TruncatedStacktraces.@truncate_stacktrace ODEProblem 3 1 2

where 3 1 2 gives the order of the types to print, with indices corresponding to the original type. For example, on a type MyType{T1,T2,T3,T4}, this will change the stacktrace printing to default to MyType{T3,T1,T2,…}.

For any new error exception you add to your package, make sure to include the note from TruncatedStacktraces.jl on how to effect the type printing. This is done by adding println(io, VERBOSE_MSG) to the bottom of any error message.

Default values

TruncatedStacktraces.VERBOSE[] defaults to false for non-CI workflows and to true for CI jobs.
TruncatedStacktraces.DISABLE defaults to true.

How It's Implemented

This is done by writing an overload on Base.show on the DataType which is conditional on TruncatedStacktraces.VERBOSE[]. For example, the following does this for the SciMLBase.ODEProblem:

@static if !TruncatedStacktraces.DISABLE
function Base.show(io::IO,
                   t::Type{<:ODEProblem{uType, tType, isinplace}}) where {uType, tType, isinplace}
    if TruncatedStacktraces.VERBOSE[]
        invoke(show, Tuple{IO, Type}, io, t)
    else
        print(io, "ODEProblem{$isinplace,$uType,$tType,…}")
    end
end
end

FAQ: Why is this not in Base Julia?

There are attempts like JuliaLang/julia#48444, but no one agrees on what exactly to do and how to make it perfect. So until people agree, we can use this solution as a nice hack that gets the job done 90%.

Related Projects

Check out https://github.com/BioTurboNick/AbbreviatedStackTraces.jl which doesn't change type printing but instead the number of calls which are shown.

truncatedstacktraces.jl's People

Contributors

Stargazers

Watchers

Forkers

avik-pal ven-k ranocha

truncatedstacktraces.jl's Issues

VERBOSE does not change anything

julia> using TruncatedStacktraces;

julia> TruncatedStacktraces.VERBOSE[] = false;

julia> v
ERROR: UndefVarError: v not defined

Some of the types have been truncated in the stacktrace for improved reading. To emit complete information
in the stack trace, evaluate `TruncatedStacktraces.VERBOSE[] = true` and re-run the code.


julia> TruncatedStacktraces.VERBOSE[] = true;

julia> v
ERROR: UndefVarError: v not defined

Some of the types have been truncated in the stacktrace for improved reading. To emit complete information
in the stack trace, evaluate `TruncatedStacktraces.VERBOSE[] = true` and re-run the code.

Doesn't look like VERBOSE does anything here - I also question whether this error message should be printed when no types are even present in the error message. Perhaps the init function in the src file needs to be something like

function __init__()
    for type in InteractiveUtils.subtypes(Exception)
        if type == MethodError
            Base.Experimental.register_error_hint(type) do io, e, args, kwargs
                VERBOSE[] && println(io, VERBOSE_MSG)
            end
        else
            Base.Experimental.register_error_hint(type) do io, e
                VERBOSE[] && println(io, VERBOSE_MSG)
            end
        end
    end
end

or maybe VERBOSE[] goes before Base.Experimental.register_error_hint(type) - I'm not really sure what register_error_hint does.

Has trouble with @show

Start with Example 3 of PreallocationTools, and add an @show as shown below. This will throw:

ERROR: UndefVarError: `specialize` not defined

Some of the types have been truncated in the stacktrace for improved reading. To emit complete information
in the stack trace, evaluate `TruncatedStacktraces.VERBOSE[] = true` and re-run the code.

Stacktrace:
  [1] show(io::IOContext{IOBuffer}, t::┌ Error: Error trying to display an error.
└ @ VSCodeServer c:\Users\elber\.vscode\extensions\julialang.language-julia-1.38.2\scripts\packages\VSCodeServer\src\eval.jl:347

Here's the code that causes the error:

using LinearAlgebra, OrdinaryDiffEq, PreallocationTools, OptimizationOptimJL, Optimization
function foo(du, u, p, t)
    tmp = p[2]
    A = reshape(p[1], size(tmp.du))
    tmp = get_tmp(tmp, u)
    mul!(tmp, A, u)
    @. du = u + tmp
    nothing
end

coeffs = -collect(0.1:0.1:0.4)
cache = DiffCache(zeros(2,2), levels = 3)
prob = ODEProblem(foo, ones(2, 2), (0., 1.0), (coeffs, cache))
realsol = solve(prob, TRBDF2(), saveat = 0.0:0.1:10.0, reltol = 1e-8)

function objfun(x, prob, realsol, cache)
    @show eltype(x).(prob.u0)
    prob = remake(prob, u0 = eltype(x).(prob.u0), p = (x, cache))
    sol = solve(prob, TRBDF2(), saveat = 0.0:0.1:10.0, reltol = 1e-8)

    ofv = 0.0
    if any((s.retcode != :Success for s in sol))
        ofv = 1e12
    else
        ofv = sum((sol.-realsol).^2)
    end
    return ofv
end
fn(x,p) = objfun(x, p[1], p[2], p[3])
optfun = OptimizationFunction(fn, Optimization.AutoForwardDiff())
optprob = OptimizationProblem(optfun, zeros(length(coeffs)), (prob, realsol, cache))
solve(optprob, Newton())

LocalPreferences.toml disable = true is not reliable

Just adding

[TruncatedStacktraces]
disable = true

to LocalPreferences.toml and then restarting the Julia REPL is apparently not sufficient to disable TruncatedStacktraces
(if TruncatedStacktraces is not part of the environment, ie is present in Manifest.toml but not Project.toml).

It does work if TruncatedStacktraces is also added to the environment

(this is testing with Julia 1.8.3)

Package breaks stacktraces

julia> @snoopr using DiffEqBase
7913-element Vector{Any}:
Error showing value of type Vector{Any}:
ERROR: UndefVarError: `isinplace` not defined

Some of the types have been truncated in the stacktrace for improved reading. To emit complete information
in the stack trace, evaluate `TruncatedStacktraces.VERBOSE[] = true` and re-run the code.

Stacktrace:
  [1] show(io::IOContext{IOBuffer}, t::
SYSTEM (REPL): showing an error caused an error

Honestly, this is just BS. Do you guys hate people trying to improve the SciML ecosystem?

Register

@JuliaRegistrator register()

Stacktrace piracy

I just tried to reproduce some Turing issue, and noticed that the TruncatedStacktraces output showed up in the stacktrace even though no to-be-truncated types are involved and TruncatedStacktraces is not a direct dependency of any of the involved packages. I assume TruncatedStacktraces "hijacked" the stacktrace since Turing depends on SciMLBase (for MAP estimation, completely unrelated to my example):

julia> sample(model2() | (; x=0.5), NUTS{Turing.EnzymeAD}(), 10)
ERROR: UndefVarError: EnzymeAD not defined

Some of the types have been truncated in the stacktrace for improved reading. To emit complete information
in the stack trace, evaluate `TruncatedStacktraces.VERBOSE[] = true` and re-run the code.

Stacktrace:
 [1] getproperty(x::Module, f::Symbol)
   @ Base ./Base.jl:31
 [2] top-level scope
   @ REPL[42]:1

I think it is both confusing and not desired that the TruncatedStacktraces message shows up in such cases.

I'm not sure what's the best way to resolve this issue but it seems wrong to just add it to absolutely every stacktrace whenever some involved package im- or explicitly depends on TruncatedStacktraces.

Using `@isdefined` for more safe type printing

From @KristofferC

Also, in cases like
print(io, "AnalyticalProblem{$isinplace,$uType,$tType,…}") one should run @isdefined isinplace && @isdefined uType && @isdefined tType before running that code. Note that this is broken on 1.9 though: JuliaLang/julia#49027

SciML/SciMLBase.jl#419

TagBot trigger issue

This issue is used to trigger TagBot; feel free to unsubscribe.

If you haven't already, you should update your TagBot.yml to include issue comment triggers.
Please see this post on Discourse for instructions and more details.

If you'd like for me to do this for you, comment TagBot fix on this issue.
I'll open a PR within a few hours, please be patient!

Extremely slow import for other packages (DataFrames, Plots, ...) loaded after SciML packages

TruncatedStacktraces is apparently adding minutes to startup time, apparently due to a slowdown in import / using of other packages loaded after OrdinaryDiffEq. This is particularly bad with Julia 1.8 (? 1.8.3 tested here), resulting in several minutes increase to startup time.

Example timings for two sequences, with TruncatedStacktraces disabled and enabled (in LocalPreferences.toml)

using OrdinaryDiffEq; import DataFrames; using Plots
import DataFrames; using Plots; using OrdinaryDiffEq

Julia 1.8.3, linux

SciMLBase 1.91.2, TruncatedStackTraces 1.2.0

	using OrdinaryDiffEq	import DataFrames	using Plots	using OrdinaryDiffEq	total
disable = true	20.8	5.8	14.7	-	41.3
disable = true	-	3.0	7.4	20.3	30.7
disable = false	31.0	72.9	51.0	-	155.9
disable = false	-	3.1	7.4	32.0	42.5

Julia 1.9.0-beta4, Windows (NB: also different computer to above)

SciMLBase 1.91.1, TruncatedStackTraces 1.2.0

	using OrdinaryDiffEq	import DataFrames	using Plots	using OrdinaryDiffEq	total
disable = false	21.6	17.4	30.7	-	69.7
disable = false	-	2.5	6.4	23.7	32.6

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.