vaibhavsagar / experiments Goto Github PK

From https://vaibhavsagar.com/blog/2018/01/03/static-haskell-nix/

"Using '<function>' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking"

but these don’t seem to cause any issues in practice

I think that isn't quite accurate -- you are probably just lucky and ran the compiled program on a machine that happened to have the same (or compatible) version of glibc as your build system has.

glibc is not really compatible with static linking. It loads stuff dynamically at run time and assumes to find the same glibc version as on the machine the was compiled. You'll get hard errors if that isn't the case.

See https://stackoverflow.com/questions/8657908/deploying-yesod-to-heroku-cant-build-statically#answer-8658468 for some details.

As it's said there:

Contrary to popular belief, static linking [of glibc] produces less, not more, portable executables on Linux.

To be really more portable, you have to link against a libc that doesn't force dynamic behaviour as glibc does. For example, the musl libc. That's what Alpine Linux uses, and the reason that in the past stack supported building statically linked binaries in an Alpine Docker image (but this is currently broken with the latest Alpine and GHC releases).

You may want to update the article with this info, otherwise people will get some bad surprises.

Hopefully that will also motivate more people to improve the static linking situation, e.g. by making musl work nicely to solve the issue.

When the node in a HAMT is full, you can avoid all the popcounting overhead, which usually makes it worth special casing. Was sad when after 5 minutes of adding random nodes to finally get the root to be made of all 1s that it didn't switch from Many to another case, like we have in unordered-containers. ;)

Hi! I saw you at ZuriHac, but didn't get a chance to chat more.

I'm trying https://vaibhavsagar.com/blog/2018/01/03/static-haskell-nix/ but have some issues with it:

Pinning nixpkgs

With the nixpkgs version on my system, simply running nix-build doesn't work. I get:

/nix/store/bm7pb1s7rx1ad80706b5xqrznq7fgpgx-gcc-7.3.0/lib/gcc/x86_64-unknown-linux-gnu/7.3.0/libgcc_eh.a(unwind-dw2-fde-dip.o): In function `_Unwind_Find_FDE':
(.text+0x1d21): undefined reference to `pthread_mutex_unlock'
/nix/store/bm7pb1s7rx1ad80706b5xqrznq7fgpgx-gcc-7.3.0/lib/gcc/x86_64-unknown-linux-gnu/7.3.0/libgcc_eh.a(unwind-dw2-fde-dip.o): In function `__register_frame_info_bases.part.6':
(.text+0x1834): undefined reference to `pthread_mutex_unlock'
/nix/store/bm7pb1s7rx1ad80706b5xqrznq7fgpgx-gcc-7.3.0/lib/gcc/x86_64-unknown-linux-gnu/7.3.0/libgcc_eh.a(unwind-dw2-fde-dip.o): In function `__register_frame_info_table_bases':
(.text+0x1944): undefined reference to `pthread_mutex_unlock'
collect2: error: ld returned 1 exit status
`cc' failed in phase `Linker'. (Exit code: 1)
builder for '/nix/store/v3mwl9jnw29hizhw6rcngs7nrh50c5aj-blank-me-up-0.1.0.0.drv' failed with exit code 1

But if I use a different version of nixpkgs, then it works:

NIX_PATH=nixpkgs=https://github.com/NixOS/nixpkgs/archive/9fdd97fad85ea422549055b8a784139b621a8cdf.tar.gz nix-build

succeeds and creates a nice statically linked executable. That is the latest release-17.09 nixpkgs commit. It doesn't work with the latest release-18.03 nixpkgs commit, so I think pinning is quite important for reproducibility.

Mention in the post what the latest approach to build is

In the blog post you first describe the approach with $ $(nix-build static.nix)/bin/fhs and so on.

Only much later, in

Edit 1: This turned out to be fairly easy.

do you create default.nix. But you don't say what do do with it, so a Haskeller not familiar with nix may not guess that they can just run nix-build at this stage.

It would be nice if you could update the post, mentioning directly something like "the following below is my old approach, right now you can just clone my repo and run nix-build, it will pick up the default.nix I made in Edit 1".

Dependency on non-nix stuff in the old approach and build failure

In your old approach you do

$ cabal install --only-dependencies --extra-include-dirs=/usr/include --extra-lib-dirs=/usr/lib

That doesn't look very nixy -- e.g. /usr/include doesn't exist in nix's concept, and I'm quite sure that can't work on NixOS. What does that do?

Also, on my Ubuntu the next step

$ cabal configure --disable-executable-dynamic --disable-shared --ghc-option=-optl=-static --ghc-option=-optl=-L/usr/lib

failed with

cabal: Encountered missing dependencies:
scotty -any

Where is scotty supposed to come from in that approach?

Thanks for the post, it is very helpful!

OK you totally sniped me with this, but here we go:

I've managed to esentially fully solve #5, doing fully static linking, by not using glibc, but a fully musl based tool chain.

Here: https://github.com/nh2/static-haskell-nix

At the time of writing, it's at this commit.

This is based on work of @dtzWill (such as NixOS/nixpkgs#34645 and especially this NixOS/nixpkgs#37598 (comment)).

CC @angerman @bgamari, some of your work is paying off big time here.