Hi, just came across a missing symbol issue (not sure if it's related to Julia 1.8.3). I've been using unchanged code for some time and fairly sure this issue is upstream from anything I'm doing.

Opening the issue here because I'm not seeing any reference to the symbol EVP_idea_cbc: https://github.com/JuliaWeb/OpenSSL.jl/search?q=EVP_idea_cbc

julia: symbol lookup error: /home/dehann/.julia/artifacts/53037ac9d528ee46c3526799b407ee52b7c224f3/lib/libssl.so: undefined symbol: EVP_idea_cbc, version OPENSSL_1_1_0

The artifact libssl.so file does exist:

$ ~/.julia/artifacts/53037ac9d528ee46c3526799b407ee52b7c224f3/lib/libssl.so
/home/dehann/.julia/artifacts/53037ac9d528ee46c3526799b407ee52b7c224f3/lib/libssl.so: symbolic link to libssl.so.1.1

The symbol does exist in the .so file:

$ nm -gD ~/.julia/artifacts/53037ac9d528ee46c3526799b407ee52b7c224f3/lib/libssl.so | grep -i EVP_idea
                 U EVP_idea_cbc

MWE

Trying to boil it down to MWE, here is the shortest I could get so far:

This load order works fine

using OpenSSL
using RobotOS # only has one dependency: PyCall.jl

This breaks

using RobotOS # only has one dependency: PyCall.jl
using OpenSSL # flip load order with RobotOS

this gives the error above undefined symbol: EVP_idea_cbc, version OPENSSL_1_1_0. My best guess at the moment is that the RobotOS is calling a few Python packages through PyCall, and one of the lines of code there somewhere is looking for this symbol from OpenSSL. Probably not worth going through the Python code (since that's not going to change anyway), and since it looks like the EVP_idea_cbc symbol does not seem be present here in OpenSSL.jl.

Next best guess is that something with newest Julia is exposing a bug deeper in the stack. Either way the error is difficult to parse and took me a long time to boil it down this far.

OpenSSL_jll Wrapper

Dont know these binary wrappers too well myself yet, but nothing in terms of special cases etc:

• https://github.com/JuliaBinaryWrappers/OpenSSL_jll.jl/blob/fd2fb4d01957cee3aecc1d537f520a99cb6b3d87/src/wrappers/x86_64-linux-gnu.jl#L1

[edited]

Version Info

julia> versioninfo()
Julia Version 1.8.3
Commit 0434deb161e (2022-11-14 20:14 UTC)
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 12 × Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-13.0.1 (ORCJIT, skylake)
  Threads: 6 on 12 virtual cores
Environment:
  LD_LIBRARY_PATH = /opt/ros/noetic/lib
  JULIA_NUM_THREADS = 6

Ubuntu 20.04

Here "moderate":
https://www.openssl.org/news/secadv/20231024.txt

but here "high severity" so I thought I would let you know:
GHSA-xw78-pcr6-wrg8

I don't know if the latter alarming, or if this applies to Julia. Is only part of openssl used? Should all issues be taken seriously (just in case), could some use the OpenSSL.jl and use the full API, more than is exposed, if that applies?

Ok to report here (in general) or better at OpenSSL_jll (or at JuliaLang)? For quickest fix. This is public by now, assuming ok to open public issue (better than in private, at least in such case, I would never know of non-public...), rather than email or some other non-public.

I'm sure this is a very, very minor thing, but shouldn't this work?

julia> EvpMD2()
ERROR: could not load symbol "EVP_md2":
dlsym(0x8e716ac0, EVP_md2): symbol not found
Stacktrace:
 [1] EvpMD2()
   @ OpenSSL ~/.julia/packages/OpenSSL/8wxMC/src/OpenSSL.jl:1174
 [2] top-level scope
   @ REPL[3]:1

This works:

julia> EvpMD5()
OpenSSL.EvpDigest(Ptr{Nothing} @0x00000001177484d0)

Seems like EvpMD2 is exported. I couldn't find anything about Evp_md2, apart from this line. I don't know if this helps though..

Julia Version 1.9.1
Commit 147bdf428cd (2023-06-07 08:27 UTC)
Platform Info:
  OS: macOS (arm64-apple-darwin22.4.0)
  CPU: 8 × Apple M1
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-14.0.6 (ORCJIT, apple-m1)
  Threads: 1 on 4 virtual cores

Loading OpenSSL is creating an issue for the MS SQL Server ODBC driver. Any help or guidance would be appreciated. Example below is ran with UBI8 minimal container image. ODBC works fine without OpenSSL loaded, but errors when it is loaded. Using msodbcsql17-17.9.1.1, but had same issue with various other versions of the SQL Server ODBC driver.

(@v1.8) pkg> st
Status `/usr/local/julia/depot/environments/v1.8/Project.toml`
  [be6f12e9] ODBC v1.1.2
  [4d8831e6] OpenSSL v1.3.1

julia> using ODBC

julia> conn = ODBC.Connection("driver=ODBC Driver 17 for SQL Server;pwd=$password;uid=$user;server=$server;database=$database");

julia> ODBC.disconnect!(conn)

julia> exit()
[root@0f4b46074e95 /]# julia
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.8.2 (2022-09-29)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

julia> using ODBC, OpenSSL


julia> conn = ODBC.Connection("driver=ODBC Driver 17 for SQL Server;pwd=$password;uid=$user;server=$server;database=$database");
ERROR: 01000: [unixODBC][Driver Manager]Can't open lib '/opt/microsoft/msodbcsql17/lib64/libmsodbcsql-17.9.so.1.1' : file not found
Stacktrace:
 [1] error(s::String)
   @ Base ./error.jl:35
 [2] driverconnect(connstr::String)
   @ ODBC.API /usr/local/julia/depot/packages/ODBC/9VZTC/src/API.jl:111
 [3] connect
   @ /usr/local/julia/depot/packages/ODBC/9VZTC/src/API.jl:349 [inlined]
 [4] ODBC.Connection(dsn::String; user::Nothing, password::Nothing, extraauth::Nothing)
   @ ODBC /usr/local/julia/depot/packages/ODBC/9VZTC/src/dbinterface.jl:57
 [5] ODBC.Connection(dsn::String)
   @ ODBC /usr/local/julia/depot/packages/ODBC/9VZTC/src/dbinterface.jl:54
 [6] top-level scope
   @ REPL[3]:1

julia> versioninfo()
Julia Version 1.8.2
Commit 36034abf260 (2022-09-29 15:21 UTC)
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 2 × 12th Gen Intel(R) Core(TM) i7-12800H
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-13.0.1 (ORCJIT, goldmont)
  Threads: 1 on 2 virtual cores

This has the same title as #9 mentioned by @dehann because I have the same problem. I found a work around and I can probably submit a patch...when we can figure out how and where it's actually happening. Read through the linked issue for background, which I won't repeat here.

The players:

• OpenSSL
• OpenSSL_jll
• PyCall
• py-julia

Base OS is Ubuntu 20.04 but I'm also testing it on Alpine Linux to build inside a docker container via a CI system. Let's stick with Ubuntu for this example. On my computer, here's all the things with the name libssl.so in them.

/home/lee/.julia/artifacts/3a83c270789c2c05ff731923907ab1ccac10dfeb/lib/libssl.so
/home/lee/.julia/artifacts/3a83c270789c2c05ff731923907ab1ccac10dfeb/lib/libssl.so.1.1
/home/lee/.julia/artifacts/3a83c270789c2c05ff731923907ab1ccac10dfeb/logs/OpenSSL/update_linkage_libssl.so.1.1_libcrypto.so.1.1.log.gz
/home/lee/.julia/artifacts/3a83c270789c2c05ff731923907ab1ccac10dfeb/logs/OpenSSL/update_linkage_openssl_libssl.so.1.1.log.gz
/home/lee/.julia/artifacts/3a83c270789c2c05ff731923907ab1ccac10dfeb/logs/OpenSSL/update_rpath_libssl.so.1.1_libcrypto.so.1.1.log.gz
/home/lee/.julia/artifacts/53037ac9d528ee46c3526799b407ee52b7c224f3/lib/libssl.so
/home/lee/.julia/artifacts/53037ac9d528ee46c3526799b407ee52b7c224f3/lib/libssl.so.1.1
# ... some stuff in snapcraft we can ignore
/usr/lib/x86_64-linux-gnu/libssl.so
/usr/lib/x86_64-linux-gnu/libssl.so.1.1

The last two on the list seem important, as those are what the OS distributes but it looks like the Julia binary is pulled from this package on install time and it is distributing its own C shared lib. That's the one that throws the EVP_idea_cbc symbol error. I suspect because it was compiled differently. What if...I just replace the Julia one? Let's try!

lee@MS3ZRMZ-LT:~$ ls -l /home/lee/.julia/artifacts/53037ac9d528ee46c3526799b407ee52b7c224f3/lib/libssl.so
lrwxrwxrwx 1 lee lee 13 Dec 18 21:33 /home/lee/.julia/artifacts/53037ac9d528ee46c3526799b407ee52b7c224f3/lib/libssl.so -> libssl.so.1.1
lee@MS3ZRMZ-LT:~$ rm /home/lee/.julia/artifacts/53037ac9d528ee46c3526799b407ee52b7c224f3/lib/libssl.so
lee@MS3ZRMZ-LT:~$ ln -s /usr/lib/x86_64-linux-gnu/libssl.so.1.1 /home/lee/.julia/artifacts/53037ac9d528ee46c3526799b407ee52b7c224f3/lib/libssl.so
lee@MS3ZRMZ-LT:~$ ls -l /home/lee/.julia/artifacts/53037ac9d528ee46c3526799b407ee52b7c224f3/lib/libssl.so
lrwxrwxrwx 1 lee lee 39 Jan 14 01:21 /home/lee/.julia/artifacts/53037ac9d528ee46c3526799b407ee52b7c224f3/lib/libssl.so -> /usr/lib/x86_64-linux-gnu/libssl.so.1.1

That removes the symbol error!

This obviously sucks and I spent about 6 hours debugging this across a number of projects in my organization. We're doing earth orbital simulation through custom Julia packages (some planned to be open source!) which are then called from Python data APIs using py-julia, which uses PyCall behind the scenes.

All three bug reports I've found in these projects mention either PyCall or py-julia. In my case, I suspect libssl is being called because I have some initialization function that downloads planetary body data models. That's pure speculation and really difficult to debug due to this data passing back and forth through eval().

PyCall has quite an essay under the installing section devoted to how a python interpreter is picked from Julia. I suspect this might be my problem and/or close to the other people's problems.

In my case I'm using pyenv and I have many python interpreters. I'm also using the poetry dependency manager, which does some python path tricks that I don't fully understand.

but I digress

It seems like it could be helpful for this library to search through the base OS for libssl.so before using the binary from the JLL package. That's the best I can do for now. I expect to reference this issue across the other projects, get thrown around from one project to the next and perhaps post on the Julia discourse. But I'm starting here.

I guess that is why we have this weird workaround:
port, server = Sockets.listenany(10000)
iob = connect(port)
sob = accept(server)
local cert_pem
try
write(iob, x509_certificate)
cert_pem = String(readavailable(sob))
finally
close(iob)
close(sob)
close(server)
end

instead of doing this
private_key_io = IOBuffer()
write(private_key_io, evp_pkey)

public_key_io = IOBuffer()
write(public_key_io, x509_certificate)

It's great to see 3.0 supported in Julia. I just thought should the Julia wrapper version also be 3.0? At least if it was a breaking change.

It's unclear if it was, maybe only in some situations (that do not apply to Julia?):
https://www.openssl.org/docs/man3.0/man7/migration_guide.html

https://www.openssl.org/blog/blog/2021/09/07/OpenSSL3.Final/

One reason for 3.0 version was new Apache license, mention it here?

I did also see some security fixes:

Fixed in OpenSSL 3.1.1 (gi commit) (Affected since 3.1.0)
Fixed in OpenSSL 3.0.9 (git commit) (Affected since 3.0.0)
[..]

If affecting Julia (we're still at "OpenSSL_jll v3.0.8"?), go straight to 3.1 (or 3.2? is that a dev version?)?

It's unclear whether there is something fundamental in OpenSSL.jl or HTTP.jl's usage of it, but we noticed in a RelationalAI workload that executed thousands of concurrent requests (GET, PUT, etc.) to cloud storage that there was slowdown in these heavily saturated scenarios.

on both Ubuntu and macOS, with julia 1.10.3 and 1.9.4, and OpenSSL master and v1.4.3.

(jl_CVKGWL) pkg> test OpenSSL
     Testing OpenSSL
      Status `/private/var/folders/s5/8d629n5d7nsf37f60_91wzr40000gq/T/jl_nQ6JBi/Project.toml`
  [d1d4a3ce] BitFlags v0.1.8
  [4d8831e6] OpenSSL v1.4.3
  [458c3c95] OpenSSL_jll v3.0.13+1
  [ade2ca70] Dates
  [6462fe0b] Sockets
  [8dfed614] Test
  [14a3606d] MozillaCACerts_jll v2023.1.10
      Status `/private/var/folders/s5/8d629n5d7nsf37f60_91wzr40000gq/T/jl_nQ6JBi/Manifest.toml`
  [d1d4a3ce] BitFlags v0.1.8
  [692b3bcd] JLLWrappers v1.5.0
  [4d8831e6] OpenSSL v1.4.3
  [21216c6a] Preferences v1.4.3
  [458c3c95] OpenSSL_jll v3.0.13+1
  [56f22d72] Artifacts
  [2a0f44e3] Base64
  [ade2ca70] Dates
  [b77e0a4c] InteractiveUtils
  [8f399da3] Libdl
  [56ddb016] Logging
  [d6f4376e] Markdown
  [de0858da] Printf
  [9a3f8284] Random
  [ea8e919c] SHA v0.7.0
  [9e88b42a] Serialization
  [6462fe0b] Sockets
  [fa267f1f] TOML v1.0.3
  [8dfed614] Test
  [4ec0a83e] Unicode
  [14a3606d] MozillaCACerts_jll v2023.1.10
     Testing Running tests...
Test Summary: | Pass  Total  Time
OpenSSL       |    6      6  0.0s
Test Summary: | Pass  Total  Time
RandomBytes   |    1      1  0.0s
Test Summary: | Pass  Total  Time
BigNumbers    |    4      4  0.0s
asn1_time = May  6 21:35:19 2026 GMT
Test Summary: | Pass  Total  Time
Asn1Time      |    4      4  0.0s
Test Summary: | Pass  Total  Time
X509Name      |    3      3  0.0s
Test Summary: | Pass  Total  Time
ReadPEMCert   |    4      4  0.0s
Test Summary:            |Time
StackOf{X509Certificate} | None  0.0s
Test Summary:   | Pass  Total  Time
StackOf{BigNum} |    2      2  0.0s
Test Summary: |Time
X509Store     | None  0.0s
Test Summary: | Pass  Total  Time
HttpsConnect  |    8      8  6.5s
Test Summary: | Pass  Total  Time
ClosedStream  |    1      1  0.5s
Test Summary: | Pass  Total  Time
NoCloseStream |    2      2  2.9s
Test Summary: | Pass  Total  Time
Hash          |    1      1  0.1s
Test Summary:         | Pass  Total  Time
SelfSignedCertificate |    1      1  0.2s
Test Summary:       |Time
SignCertCertificate | None  0.4s
Test Summary: | Pass  Total  Time
ErrorTaskTLS  |    5      5  0.0s
Test Summary: | Pass  Total  Time
PKCS12        |    5      5  0.1s
String(read(enc_data)) = "OpenSSL Julia"
String(read(enc_data)) = "\x1a%\xfc\xb9en\xb3\xb8ЇN\xec.\r\x90\xc6"
String(read(enc_data)) = ">C\xf5D'6\xab\xc4\xcfک\a\x11\xa2\xaa*"
String(read(enc_data)) = "\x81\xb1W1֢\xecؔ\"\xefX\xf3"
Test Summary: | Pass  Total  Time
Encrypt       |    4      4  0.0s
Test Summary:          | Pass  Total  Time
StackOf{X509Extension} |    8      8  0.0s
Test Summary:       | Pass  Total  Time
SerializePrivateKey |    2      2  0.3s
Test Summary: |Time
DSA           | None  0.0s
Test Summary: |Time
X509Attribute | None  0.0s
Test Summary: |Time
SSLServer     | None  0.0s
Test Summary: | Pass  Unhandled Task ERROR: OpenSSL.OpenSSLError("error:FFFFFFFFFFFFFFFF:system library::Unknown error: 2147483647")
Stacktrace:
 [1] ssl_accept
   @ ~/.julia/packages/OpenSSL/hXs2T/src/ssl.jl:340 [inlined]
 [2] accept
   @ ~/.julia/packages/OpenSSL/hXs2T/src/ssl.jl:557 [inlined]
 [3] test_server()
   @ Main ~/.julia/packages/OpenSSL/hXs2T/test/http_helpers.jl:38
 [4] (::var"#21#23")()
   @ Main ~/.julia/packages/OpenSSL/hXs2T/test/runtests.jl:573
Total  TimeUnhandled Task ERROR: IOError: read: connection reset by peer (ECONNRESET)
Stacktrace:
 [1] wait_readnb(x::TCPSocket, nb::Int64)
   @ Base ./stream.jl:410
 [2] eof(s::TCPSocket)
   @ Base ./stream.jl:106
 [3] connect(ssl::SSLStream; require_ssl_verification::Bool)
   @ OpenSSL ~/.julia/packages/OpenSSL/hXs2T/src/ssl.jl:497
 [4] connect
   @ ~/.julia/packages/OpenSSL/hXs2T/src/ssl.jl:489 [inlined]
 [5] test_client()
   @ Main ~/.julia/packages/OpenSSL/hXs2T/test/http_helpers.jl:72
 [6] (::var"#22#24")()
   @ Main ~/.julia/packages/OpenSSL/hXs2T/test/runtests.jl:574

VersionNumber |    3      3  0.0s
     Testing OpenSSL tests passed 

Hello everyone,

I'm running a Docker where I use OpenSSL as a dependency. Unfortunately, I cannot update to Julia 1.9 since the system image creation fails due to a line in OpenSSL.jl. Every other (older) Julia version works perfectly fine.

The line 508 in OpenSSL.jl for the call of the version number fails:

@static if version_number() ≥ v"3"

Leads to the following error, originally coming from GoogleCloud.jl:

Step #5: ERROR: LoadError: could not load symbol "OpenSSL_version_num":
Step #5: /usr/local/julia/bin/julia: undefined symbol: OpenSSL_version_num
Step #5: Stacktrace:
Step #5:   [1] version_number()
Step #5:     @ OpenSSL ~/.julia/packages/OpenSSL/8wxMC/src/OpenSSL.jl:495
Step #5:   [2] top-level scope
Step #5:     @ none:1
Step #5:   [3] eval(m::Module, e::Any)
Step #5:     @ Core ./boot.jl:370
Step #5:   [4] var"@static"(__source__::LineNumberNode, __module__::Module, ex::Any)
Step #5:     @ Base ./osutils.jl:19
Step #5:   [5] include
Step #5:     @ ./Base.jl:457 [inlined]
Step #5:   [6] _require(pkg::Base.PkgId, env::String)
Step #5:     @ Base ./loading.jl:1836
Step #5:   [7] _require_prelocked(uuidkey::Base.PkgId, env::String)
Step #5:     @ Base ./loading.jl:1660
Step #5:   [8] macro expansion
Step #5:     @ ./loading.jl:1648 [inlined]
Step #5:   [9] macro expansion
Step #5:     @ ./lock.jl:267 [inlined]
Step #5:  [10] require(into::Module, mod::Symbol)
Step #5:     @ Base ./loading.jl:1611
Step #5:  [11] include
Step #5:     @ ./Base.jl:457 [inlined]
Step #5:  [12] _require(pkg::Base.PkgId, env::String)
Step #5:     @ Base ./loading.jl:1836
Step #5:  [13] _require_prelocked(uuidkey::Base.PkgId, env::String)
Step #5:     @ Base ./loading.jl:1660
Step #5:  [14] macro expansion
Step #5:     @ ./loading.jl:1648 [inlined]
Step #5:  [15] macro expansion
Step #5:     @ ./lock.jl:267 [inlined]
Step #5:  [16] require(into::Module, mod::Symbol)
Step #5:     @ Base ./loading.jl:1611
Step #5:  [17] include(mod::Module, _path::String)
Step #5:     @ Base ./Base.jl:457
Step #5:  [18] include(x::String)
Step #5:     @ GoogleCloud ~/.julia/packages/GoogleCloud/cyJtU/src/GoogleCloud.jl:4
Step #5:  [19] top-level scope
Step #5:     @ ~/.julia/packages/GoogleCloud/cyJtU/src/GoogleCloud.jl:17
Step #5:  [20] include
Step #5:     @ ./Base.jl:457 [inlined]
Step #5:  [21] _require(pkg::Base.PkgId, env::Nothing)
Step #5:     @ Base ./loading.jl:1836
Step #5:  [22] _require_prelocked(uuidkey::Base.PkgId, env::Nothing)
Step #5:     @ Base ./loading.jl:1660
Step #5:  [23] _require_prelocked
Step #5:     @ ./loading.jl:1658 [inlined]
Step #5:  [24] macro expansion
Step #5:     @ ./lock.jl:267 [inlined]
Step #5:  [25] require(uuidkey::Base.PkgId)
Step #5:     @ Base ./loading.jl:1655
Step #5:  [26] top-level scope
Step #5:     @ /tmp/jl_Yl6PGfGrqb:36
Step #5: in expression starting at /root/.julia/packages/OpenSSL/8wxMC/src/OpenSSL.jl:508

Unfortunately, I cannot reproduce this for a normal precompilation and I therefore don't know how to provide a MWE.
As a similar issue occurred for the Unitful.jl package earlier, I guess this is somehow related to to this MR.
Is there any way we can fix this?

Hello,

and thank you for the work on this library so far.

I'm having some issues when trying to extract the DER encoding from a X509Certificate. Using this library together with OpenSSL_jll we have access to i2d_x509 which does this conversion. However, it produces a segmentation fault.

using OpenSSL, OpenSSL_jll

function der(cert::X509Certificate)
   buf = Vector{UInt8}(undef, 2048)
   len = @ccall libssl.i2d_X509(cert::X509Certificate, buf::Ptr{UInt8})::Cint
   @info len
end

# ------------ output
# Segmentation fault: 11 in expression starting at REPL[10]:1
# ASN1_put_object at /lib/libcrypto.3.dylib (unknown line)
# ASN1_item_ex_i2d at /lib/libcrypto.3.dylib (unknown line)
# Allocations: 660735 (Pool: 660175; Big: 560); GC: 1

As noted in other issues, it seems I/O is somewhat problematic currently. But maybe I'm doing something wrong (I also tried i2d_x509_bio with the BIO struct provided by this library, but it also produces errors like this).

Thank you for any help.

This issue is used to trigger TagBot; feel free to unsubscribe.

If you haven't already, you should update your TagBot.yml to include issue comment triggers.
Please see this post on Discourse for instructions and more details.

If you'd like for me to do this for you, comment TagBot fix on this issue.
I'll open a PR within a few hours, please be patient!