Code Monkey home page Code Monkey logo

ghc-nix's Introduction

ghc-nix

demo

ghc-nix is an attempt to augment GHC with the ability to use Nix as a caching layer. This has the potential to dramatically speed up "pure Nix" builds by being able to re-use build artefacts for files that haven't changed. It also has the potential to speed up developers collaborating on work, by allowing them to pull down pre-built artefacts from a shared build server, rather than building locally.

What's the Plan?

When you do cabal build, Cabal calls out to GHC with a single exec call, primarily to ghc --make. GHC actually has a build system built in, and GHC will form a dependency graph and start doing minimal recompilation where possible. However, this is only useful if you have some kind of reusable directory to keep the cache artefacts - and that doesn't exist on build machines like Hydra.

The plan then is to replace ghc --make with something that can use the Nix store. cabal build has the -w option to give it another compiler, but it will still call that executable with the same arguments. If we can impersonate GHC, then we can get the caching we desire - that's what this project does.

When ghc-nix is called with --make, we use the GHC API to form a dependency graph, as ghc --make would. However, rather than just doing the build, instead we transform this dependency graph into a series of Nix expressions, each of which will build a single .hs source file into .o and .hi files. Finally, we aggregate all the build artefacts and place the files where they need to go for Cabal to continue with the next phase (linking).

However, just doing Nix builds is not enough, we need one more experimental Nix feature to get something useful - content-addressable store paths. Usually, the objects in a Nix store are hashed based on all of the dependencies that are required to produce the store path. This ultimately comes down to an exact hash of all the source code. However, this means that if you make object-file-preserving changes, you still end up recompiling everything downstream. For example, a single comment change causes all dependents to rebuild, even though they can't observe a change.

Once we've built the .o and .hi files, we rewrite the resulting store path to one who's hash is based on the contents of these files, and not the the files that built them. This cuts the connection between an object file and its source code, allowing early cut off.

This is supported as an experimental Nix feature - nix make-content-addressable.

Trying this Out

This project is still in very early days, so it's not too easy to use... yet. Maybe you can help here! To try ghc-nix out you will need:

Nix changes

  • A version of Nix that is at least 4511f09b490fad4ce0dcfbcd7c4fd83b11e7df46.

  • nix-command and ca-references enabled as experimental features.

Heres's a snippet of my configuration.nix that should get you this.

{
  systemFeatures =
    [ "benchmark" "big-parallel" "kvm" "nixos-test" "recursive-nix" "nix-command" "ca-references" ];

  extraOptions =
    ''
    experimental-features = recursive-nix nix-command ca-references
    '';

  package =
    let
      src =
        pkgs.fetchFromGitHub
          { owner =
              "NixOS";

            repo =
              "nix";

            rev =
              "9f7b4d068cc106e5d902dc6f52bf46d4a057fd00";

            sha256 =
              "187pfanj0g49ng5smfi8rwkq1l3r43mf85yv390h0ars050fxfik";
          };

    in
    ( import "${src}/release.nix" { nix = src; officialRelease = true; } ).build.x86_64-linux;
}

Building ghc-nix

ghc-nix is built like a normal Haskell project, but you will have to actually use cabal install to have this work. I do this, in the ghc-nix directory:

cabal install --installdir=./bin --overwrite-policy=always

Using ghc-nix

You can now finally use ghc-nix by going to a Cabal project and running:

cabal build -w /path/to/ghc-nix/bin/ghc-nix

You will need a working ghc on your PATH, too (this will hopefully change in the future).

If you get problems with packages not being found and you're using Nix, you might need to run:

NIX_GHC=$(type -p ghc)
eval $(grep export "$NIX_GHC")

ghc-nix's People

Contributors

matthewbauer avatar ocharles avatar gabriella439 avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.