Bug: GGML can no longer be statically linked to llama.cpp due to the source code reorganization about llama.cpp HOT 9 CLOSED

Some features of llama.cpp require using ggml. If llama.cpp is built as a shared library linking statically to ggml, then applications cannot use these features without linking their own copy of ggml - which may not work at all. Is it really that much of a problem to bundle the ggml shared library alongside llama.cpp?

@mudler BUILD_SHARED_LIBS is the default in most cases:

from llama.cpp.

Yes. I agree with you. I got lbc++ statically linking, but gomp and openblas are always dynamically linked. You can reclose this if you want.

from llama.cpp.

It should be linked statically without the -DBUILD_SHARED_LIBS=ON parameter (edit: BUILD_SHARED_LIBS is the default in most cases). What is not possible at the moment is to build llama.cpp as a shared library and ggml as a static library, but I am not sure that's really a problem.

from llama.cpp.

Thanks a lot for the reply @slaren!

This flag indeed links GGML statically into llama.cpp. However, in this configuration, llama.cpp is also generated as a static library. I'm trying to compile llama.cpp as a dynamic library.

For me, the ideal option would be to have both GGML_BUILD_SHARED_LIBS and LLAMA_BUILD_SHARED_LIBS.

from llama.cpp.

I'm hitting the same issue here - however it's more dodgy in my case as I'm linking also grpc - and, before the mentioned PR that's how the linker was looking like on the resulting binaries, and everything was working fine:

       linux-vdso.so.1 (0x00007ffe3a6b6000)
        libgomp.so.1 => /lib64/libgomp.so.1 (0x00007f784dbfc000)
        libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007f784d800000)
        libm.so.6 => /lib64/libm.so.6 (0x00007f784db15000)
        libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f784dae6000)
        libc.so.6 => /lib64/libc.so.6 (0x00007f784d400000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f784f882000)

now, with shared mode on there are more shared libraries used, but that breaks badly my linker now:

/usr/lib64/gcc/x86_64-suse-linux/13/../../../../x86_64-suse-linux/bin/ld: ../../bin/grpc-server: hidden symbol `_ZN6google8protobuf2io19EpsCopyOutputStream16WriteRawFallbackEPKviPh' in /home/mudler/_git/LocalAI/backend/cpp/grpc/installed_packages/lib64/libprotobuf.a(coded_stream.cc.o) is referenced by DSO

and yeah, the symbol is hidden in libprotobuf.a - but the pity of that is that I can't always control how libprotobuf is built (indeed, it fails in mac with homebrew now)

It should be linked statically without the -DBUILD_SHARED_LIBS=ON parameter. What is not possible at the moment is to build llama.cpp as a shared library and ggml as a static library, but I am not sure that's really a problem.

Sadly that has no effect here - I never enabled that explicitly before. I 'm going to try to disable it explicitly and see if that's helping

Update: Thanks for the hint @slaren ! specifying -DBUILD_SHARED_LIBS=OFF did the trick here - I guess now -DBUILD_SHARED_LIBS=ON is implied by default?

from llama.cpp.

Some features of llama.cpp require using ggml. If llama.cpp is built as a shared library linking statically to ggml, then applications cannot use these features without linking their own copy of ggml - which may not work at all. Is it really that much of a problem to bundle the ggml shared library alongside llama.cpp?

In my case it is, because I'm using llama.cpp with gRPC, and that is by default static - and mixing gRPC static with shared ggml and llama cpp breaks the linker as some of the protobuf symbols are hidden when building from a DSO.

@mudler BUILD_SHARED_LIBS is the default in most cases:

llama.cpp/CMakeLists.txt

Lines 31 to 41 in f675b20

	if (EMSCRIPTEN)
	set(BUILD_SHARED_LIBS_DEFAULT OFF)
	option(LLAMA_WASM_SINGLE_FILE "llama: embed WASM inside the generated llama.js" ON)
	else()
	if (MINGW)
	set(BUILD_SHARED_LIBS_DEFAULT OFF)
	else()
	set(BUILD_SHARED_LIBS_DEFAULT ON)
	endif()
	endif()

gotcha - thanks @slaren ! - it clicked here now thanks to your previous comment. I thought that was defaulting to ON before as well when I was looking at the PR diff, but it looks it wasn't (at least here), so now I had to disable it explicitly to get things going. I have yet to see what LocalAI's CI tells, but locally at least now builds fine as before.

from llama.cpp.

Thanks folks for what you've shared.

Some features of llama.cpp require using ggml. If llama.cpp is built as a shared library linking statically to ggml, then applications cannot use these features without linking their own copy of ggml

I don't believe that was the case unless I missed something. I used to use pretty much all of the llama.cpp API by compiling llama.cpp as a shared library, statically linking ggml, at least on Windows, macOS, and Linux systems.

Is it really that much of a problem to bundle the ggml shared library alongside llama.cpp?

Good point. I would not argue that it is a real problem. Plus I understand the willingness to keep ggml as a separate binary for the long run and/or for building apps using llama.cpp, Whisper.cpp, and probably other upcoming projects.

-> I'm reviewing my compilation and deployment pipelines to follow this approach.

If you don't mind, I'm marking this case as resolved.

from llama.cpp.

I'm having the same issue. Tried with both GCC and Clang in MSYS2.
Even when setting GGML_STATIC to true, there's still dependencies to libc++, pthread ,etc

LLMA_STATIC from before was working and doing what we needed.

What are the flags now to get llama-server linked statically to libc++, etc?

from llama.cpp.

LLAMA_STATIC never really worked here as it always linked on libgomp and failed linking, same GGML_STATIC (but maybe caused by my requirements? ). Did you tried to set BUILD_SHARED_LIBS to OFF?

from llama.cpp.