`realloc(): invalid pointer` about parsers.jl HOT 11 CLOSED

Using the following to include Parsers.jl in a system image will fail on 0.2.20 or 0.2.18:

RUN julia -e 'using PackageCompiler; compile_package("Parsers", force=true)'

Like you said, the alternative build_sysimg allows compilation to succeed but gives the realloc error at runtime.

from parsers.jl.

Seems likely to have been introduced by #20, based on the backtrace.

from parsers.jl.

I just ran all the CSV.jl tests and didn't see any issues; is there a way you could share the file, even if only privately? ([email protected])

from parsers.jl.

I'll try to pull the data tomorrow

from parsers.jl.

@simonbyrne, do you have any ideas on what might be happening here? My two thoughts are

• are we doing something bad w/ the mutating operations here? not accounting for things growing when they're not supposed to or something?
• are we getting some weird precompile issue (since we pre-allocate the NUMS that get used during scale) and their pointers are somehow getting invalidated/corrupt and then we try to resize them?

from parsers.jl.

The second one could be the case. Tough to say without a reproducible example.

from parsers.jl.

It appears that the issue has to do with using PackageCompiler. I can reproduce the problem by running the tests for Parsers. The issue can be reproduced using this Dockerfile:

FROM julia:1.0.3-stretch

RUN apt-get update && apt-get install -y gcc && \
    julia -e 'using Pkg; Pkg.add("PackageCompiler"); using PackageCompiler; compile_package(force=true)'

RUN julia -e 'using Pkg, PackageCompiler; Pkg.add(PackageSpec(name="Parsers", version="0.2.20"))'

# Fails with "Task cannot be serialized"
# RUN julia -e 'using PackageCompiler; compile_package("Parsers", force=true)'

RUN echo "using Parsers" > userimg.jl && \
    julia -e 'using PackageCompiler: build_sysimg, default_sysimg_path; build_sysimg(default_sysimg_path(), "userimg.jl")'

CMD ["julia", "-e", "using Pkg; Pkg.test(\"Parsers\")"]

$ docker build -t parsers .
...

$ docker run --rm parsers
...
*** Error in `/usr/local/julia/bin/julia': realloc(): invalid next size: 0x00007f0cb087c4d0 ***
...
signal (6): Aborted
in expression starting at /root/.julia/packages/Parsers/v5u2B/test/floats.jl:1
gsignal at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
abort at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
unknown function (ip: 0x7f0cb58d8bff)
unknown function (ip: 0x7f0cb58defc5)
unknown function (ip: 0x7f0cb58e213b)
realloc at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
jl_gc_counted_realloc_with_old_size at /buildworker/worker/package_linux64/build/src/gc.c:2777
__gmpz_realloc at /usr/local/julia/bin/../lib/julia/libgmp.so (unknown line)
__gmpz_mul at /usr/local/julia/bin/../lib/julia/libgmp.so (unknown line)
mul! at ./gmp.jl:132 [inlined]
mul! at ./gmp.jl:134 [inlined]
scale at /root/.julia/packages/Parsers/v5u2B/src/floats.jl:80 [inlined]
scale at /root/.julia/packages/Parsers/v5u2B/src/floats.jl:114 [inlined]
#_defaultparser#46 at /root/.julia/packages/Parsers/v5u2B/src/floats.jl:239 [inlined]
_defaultparser at /root/.julia/packages/Parsers/v5u2B/src/floats.jl:126 [inlined]
#defaultparser#45 at /root/.julia/packages/Parsers/v5u2B/src/floats.jl:123 [inlined]
defaultparser at /root/.julia/packages/Parsers/v5u2B/src/floats.jl:123
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2184
...

Using Parsers 0.2.18 does not have this issue.

from parsers.jl.

@omus So if you enable the commented-out line, it fails to compile, but with sysimg it succeeds compilation but gives the realloc error at runtime?

from parsers.jl.

So if I'm understanding correctly, when Parsers.jl is included in the sysimg, the pre-allocated BigInts are somehow borked, but normal precompilation is fine. Right?

from parsers.jl.

So if I'm understanding correctly, when Parsers.jl is included in the sysimg, the pre-allocated BigInts are somehow borked, but normal precompilation is fine. Right?

That is also my understanding

from parsers.jl.

You can't mutate BigInts in the system image. Since this use case needs resize_nthreads! in __init__ anyway, might as well initialize the global BigInts there.

from parsers.jl.