ghc-nix is an attempt to augment GHC with the ability to use Nix as a caching
layer. This has the potential to dramatically speed up "pure Nix" builds by
being able to re-use build artefacts for files that haven't changed. It also has
the potential to speed up developers collaborating on work, by allowing them to
pull down pre-built artefacts from a shared build server, rather than building
locally.
When you do cabal build, Cabal calls out to GHC with a single exec call,
primarily to ghc --make. GHC actually has a build system built in, and GHC
will form a dependency graph and start doing minimal recompilation where
possible. However, this is only useful if you have some kind of reusable
directory to keep the cache artefacts - and that doesn't exist on build machines
like Hydra.
The plan then is to replace ghc --make with something that can use the Nix
store. cabal build has the -w option to give it another compiler, but it
will still call that executable with the same arguments. If we can impersonate
GHC, then we can get the caching we desire - that's what this project does.
When ghc-nix is called with --make, we use the GHC API to form a dependency
graph, as ghc --make would. However, rather than just doing the build, instead
we transform this dependency graph into a series of Nix expressions, each of
which will build a single .hs source file into .o and .hi files. Finally,
we aggregate all the build artefacts and place the files where they need to go
for Cabal to continue with the next phase (linking).
However, just doing Nix builds is not enough, we need one more experimental Nix feature to get something useful - content-addressable store paths. Usually, the objects in a Nix store are hashed based on all of the dependencies that are required to produce the store path. This ultimately comes down to an exact hash of all the source code. However, this means that if you make object-file-preserving changes, you still end up recompiling everything downstream. For example, a single comment change causes all dependents to rebuild, even though they can't observe a change.
Once we've built the .o and .hi files, we rewrite the resulting store path
to one who's hash is based on the contents of these files, and not the the files
that built them. This cuts the connection between an object file and its source
code, allowing early cut off.
This is supported as an experimental Nix feature - nix make-content-addressable.
This project is still in very early days, so it's not too easy to use... yet.
Maybe you can help here! To try ghc-nix out you will need:
-
A version of Nix that is at least
4511f09b490fad4ce0dcfbcd7c4fd83b11e7df46. -
nix-commandandca-referencesenabled as experimental features.
Heres's a snippet of my configuration.nix that should get you this.
{
systemFeatures =
[ "benchmark" "big-parallel" "kvm" "nixos-test" "recursive-nix" "nix-command" "ca-references" ];
extraOptions =
''
experimental-features = recursive-nix nix-command ca-references
'';
package =
let
src =
pkgs.fetchFromGitHub
{ owner =
"NixOS";
repo =
"nix";
rev =
"9f7b4d068cc106e5d902dc6f52bf46d4a057fd00";
sha256 =
"187pfanj0g49ng5smfi8rwkq1l3r43mf85yv390h0ars050fxfik";
};
in
( import "${src}/release.nix" { nix = src; officialRelease = true; } ).build.x86_64-linux;
}ghc-nix is built like a normal Haskell project, but you will have to actually
use cabal install to have this work. I do this, in the ghc-nix directory:
cabal install --installdir=./bin --overwrite-policy=alwaysYou can now finally use ghc-nix by going to a Cabal project and running:
cabal build -w /path/to/ghc-nix/bin/ghc-nixYou will need a working ghc on your PATH, too (this will hopefully change in
the future).
If you get problems with packages not being found and you're using Nix, you might need to run:
NIX_GHC=$(type -p ghc)
eval $(grep export "$NIX_GHC")
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent