nix-community / nixpkgs-fmt Goto Github PK

View Code? Open in Web Editor NEW

560.0 12.0 33.0 3.91 MB

Nix code formatter for nixpkgs [maintainer=@zimbatm]

Home Page: https://nix-community.github.io/nixpkgs-fmt/

License: Apache License 2.0

Rust 67.99% Nix 27.76% Shell 1.10% HTML 3.14%

nix formatter rust-crate nixpkgs cli pre-commit-hook

nixpkgs-fmt's Introduction

nixpkgs-fmt - Nix code formatter for nixpkgs

STATUS: beta

This project's goal is to format the nix code in nixpkgs to increase the consistency of the code found there. Ideally automatically with pre-commit hooks and later ofborg enforcing the format.

Demo

You can try nixpkgs-fmt in your browser. The page also provides a way for you to submit code samples if you find the output not satisfying: https://nix-community.github.io/nixpkgs-fmt/

Design decisions

You might ask yourself; why do we need yet another nix code formatter?

The main goal of nixpkgs-fmt is to provide some overall consistency in the nix code submitted to nixpkgs, our main package repository.

At this point it's important to understand that there are multiple possible outputs for a code formatter. Those outputs will depend on multiple conflicting desires and depending on how much weight is being put on each requirement the output will change.

For nixpkgs-fmt we have a few of these:

Minimize merge conflicts. nixpkgs is seen in a lot of pull-requests and we want to avoid them getting unnecessarily stale.
Only expand, don't collapse. It's up to the developer to choose if an element should be on a single line or multiple lines.
Respect the developer's expressivity. Empty lines can be useful as a way to separate blocks of code.
Only change the indent of one (+/-) per line. Not sure why but it seems like a good thing.

Corollary rules:

because of (1). The format is quite close to what exists in nixpkgs already.
because of (1). Don't align values vertically, a single line change can introduce a very big diff.
because of (1). Avoid too many rules. More rules means more formatting changes that create merge conflicts.
because of (2). Don't enforce line lengths. Line length limits also create complicated heuristics.

At the time where we started this project none of the other formatters were weighted that way.

To implement this, we needed a whitespace and comment-preserving parser which rnix provides to us. Then create an engine that follows the AST and patches the tree with rewrite rules. The nice thing about this design is that it also works on incomplete or broken nix code. We are able to format up to the part that is missing/broken, which makes it great for potential editor integration.

Most of the other formatters out there take a pretty-printing approach where the AST is parsed, and then a pretty-printer inspects and formats the AST back to code without taking spaces and newlines into account. The advantage is that it's initially easier to implement. The output is very strict and the same AST will always give the same output. One disadvantage is that the pretty-printer needs to handle all the possible combination of Nix code to make them look good.

With nixpkgs-fmt the output will depend on how the code was formatted initially. The developer still has some input on how they want to format their code. If there is no rule for a complicated case, the code will be left alone. For nixpkgs this approach will be preferable since it minimizes the diff.

Well done for reading all of this, I hope this clarifies a bit why nixpkgs-fmt exists and what role it can play.

Usage

nixpkgs-fmt 1.2.0
Format Nix code

USAGE:
    nixpkgs-fmt [FLAGS] [OPTIONS] [FILE]...

FLAGS:
        --check      Only test if the formatter would produce differences
        --explain    Show which rules are violated
    -h, --help       Prints help information
        --parse      Show syntax tree instead of reformatting
    -V, --version    Prints version information

OPTIONS:
        --output-format <FORMAT>    Set output format of --parse [default: rnix]  [possible values: rnix, json]

ARGS:
    <FILE>...    File to reformat in place. If no file is passed, read from stdin.

Tree traversal

When nixpkgs-fmt is given a folder as a file argument, it will traverse that using the same ignore crate as ripgrep, using 8 parallel threads.

By default it will automatically ignore files reading .ignore, .gitignore, and .git/info/exclude files in that order. If additional files need to be ignored, it is also possible to add --exclude <glob> to the call.

Installation

nixpkgs-fmt is available in nixpkgs master. nix-env -i nixpkgs-fmt.

It's also possible to install it directly from this repository:

nix-env -f https://github.com/nix-community/nixpkgs-fmt/archive/master.tar.gz -iA nixpkgs-fmt

VSCode extensions

There are a few VSCode extensions that make using nixpkgs-fmt convenient. Check out:

pre-commit hook

This project can also be installed as a pre-commit hook.

Add to your project's .pre-commit-config.yaml:

-   repo: https://github.com/nix-community/nixpkgs-fmt
    rev: master
    hooks:
    -   id: nixpkgs-fmt

Make sure to have rust available in your environment.

Then run pre-commit install-hooks

Development

Install Rust and Cargo or run nix-shell to load the project dependencies.

Install pre-commit and run pre-commit install to setup the git hooks on the repository. This will allow to keep the code nicely formatted over time.

Then use cargo run to build and run the software.

Running Fuzzer

$ cargo install cargo-fuzz
$ mkdir -p ./fuzz/corpus/fmt
$ cp test_data/**.nix ./fuzz/corpus/fmt
$ rustup run nightly -- cargo fuzz run fmt

or with nix:

$ nix-shell --run "cargo fuzz run fmt"

fmt is the name of the target in ./fuzz/Cargo.toml

Fuzzer will run indefinitely or until it finds a crash. The crashing input is written to fuzz/artifacts directory. Commit this crash- file, and it will be automatically tested by a unit-test.

Documentation

Related projects

Feel free to submit your project!

Using nixpkgs-fmt

Emacs integration, including minor mode for format-on-save
rnix-lsp - A Lambda Server for Nix

Formatters

alejandra - Another rnix based formatter, using a rule based engine.
canonix - Nix formatter prototype written in Haskell using the tree-sitter-nix grammar.
format-nix - A nix formatter using tree-sitter-nix.
nix-format - Emacs-based Nix formatter.
nix-lsp - Nix language server using rnix.
nixfmt - A nix formatter written in Haskell.

Linters

nix-linter

Parsers

hnix - Haskell implementation of Nix including a parser. The parser is not comment-preserving.
rnix - Rust Nix parser based on rowan
tree-sitter-nix - Tree Sitter is a forgiving parser used by Atom for on-the-fly syntax highlighting and others. This is a implementation for Nix.

Discussions

nixpkgs-fmt's People

Contributors

Stargazers

Watchers

nixpkgs-fmt's Issues

RuntimeError when the first line is whitespace

Input

Note there is a before pkgs:

 pkgs

Output

RuntimeError: unreachable executed

NUR package?

I currently have added it to my namespace:

$ nix-shell -p nur.repos.mic92.nixpkgs-fmt
$ nixpkgs-fmt --help
nixpkgs-fmt 0.1
Format Nix code

USAGE:
    nixpkgs-fmt [FLAGS] [OPTIONS] [FILE]

FLAGS:
    -h, --help        Prints help information
    -i, --in-place    Overwrite FILE in place
        --parse       Show syntax tree instead of reformatting
    -V, --version     Prints version information

OPTIONS:
    -o, --output <file>    Place the output into <file>

ARGS:
    <FILE>    File to reformat

However it might be useful to have it might useful to have it a namespace
managed by one of the maintainers. Otherwise you can always make a pull request to my repository ...

WASM Demo: show diff

I tested it with a package i created, but see no diff. So maybe my code is perfect (i hope :D), but i might not see the diff. So it would be helpful to highlight the diff.

default.nix

I'd love to use it if I can nix-build :)

Fuzzing crash 654e6ed06597b86f23483b793664777da9b24c04

Describe the bug
Fuzzing crash: fuzz/artifacts/fmt/crash-654e6ed06597b86f23483b793664777da9b24c04

To Reproduce
Steps to reproduce the behavior:

Run echo ".." > foo.nix
Run `nixpkgs-fmt foo.nix'

thread 'main' panicked at 'assertion failed: prev.is_none()', src/engine/fmt_model.rs:269:9

Provide 'validate' option

Is your feature request related to a problem? Please describe.
I usually use formaters / linters in pre-commit hooks to prevent mal-formatted commits but I dislike for the formatter to manipulate the git index by formatting changed files. Instead, I prefer running in linter mode that can fail the hook with exit code != 0 in case of formatting issues. The point of this is that I prefer to review the change before committing it. nixpkgs-fmt lacks such a mode.

Describe the solution you'd like
A command option nixpkgs-fmt --validate [srcs] that takes a list of files and fails if one or more files are not properly formatted.

Describe alternatives you've considered
I suppose you can achieve this using bash scripting, eg (pseudo-code)

for file in $(find . -name "*.nix"); do
  cat "$file" | nixpkgs-fmt --explain >  "$file.explain"
  if diff "$file" "$file.explain"; then
    exit 1
  fi 
done
exit 0

This does feel like a hack though.

sample: Weirdly closed brace in attribute set default in function arguments

Input

{
  pkgs ? import ./nix/nixpkgs.nix { },
  src ? builtins.fetchGit {
    url = ./.;
    ref = "HEAD";
  }
}:
pkgs.example rec {
}

Output

{ pkgs ? import ./nix/nixpkgs.nix {}
, src ? builtins.fetchGit {
  url = ./.;
  ref = "HEAD";
} # <- it seems this } and the two prior lines should be indented
}:
pkgs.example rec {
}

sample: indented let body

whenever a let..in block is being added, it re-indents the whole body, creating a lot of noise in the diff.

Maybe it's worth special-casing at least for the top-level to not indent the body?

Input

{ stdenv, fetchFromGitHub }:
let
pname = "hello";
version = "1.2.3";
in
stdenv.mkDerivation {
  inherit pname version;
  src = fetchFromGitHub {
    owner = "xxx";
    repo = pname;
    rev = version;
    sha256 = "...";
  };
}

Output

{ stdenv, fetchFromGitHub }:
let
  pname = "hello";
  version = "1.2.3";
in
  stdenv.mkDerivation {
    inherit pname version;
    src = fetchFromGitHub {
      owner = "xxx";
      repo = pname;
      rev = version;
      sha256 = "...";
    };
  }

Desired output

{ stdenv, fetchFromGitHub }:
let
  pname = "hello";
  version = "1.2.3";
in
stdenv.mkDerivation {
  inherit pname version;
  src = fetchFromGitHub {
    owner = "xxx";
    repo = pname;
    rev = version;
    sha256 = "...";
  };
}

Debug mode

It would be cool if the engine could should all the matching rules with the diff. The more rule we add, the harder it becomes to understand how they interact with each-other.

Example output:

<rule name> matched <element>
<diff>

What do you think @matklad ?

sample:

Input

{
      inherit (import ./pep425.nix {
        inherit lib python;
        inherit (pkgs) stdenv;
      }) selectWheel;
}

Output

{
  inherit (import ./pep425.nix {
    inherit lib python;
    inherit (pkgs) stdenv;
  }) selectWheel
    ;
}

Desired output

{
  inherit (import ./pep425.nix {
    inherit lib python;
    inherit (pkgs) stdenv;
  }) selectWheel;
}

The ; should not be on a separate line.

think about how to integrate with nixpkgs

https://github.com/nrwl/precise-commits#why-precise-commits looks interesting for a gradual transition

sample:

Input

{
  inherit (builtins)
    # comment
    toString
    ;
}

Output

{
  inherit (builtins) # comment toString;
}

Expected output

{
  inherit (builtins)
    # comment
    toString
    ;
}

remove line break after expanded let ... in

# One line, everyone's happy
let self = { }; in self

#65 (comment) suggests that we could consider let ... in just as an assert ...; or a with ... ; statement that does not indent the following body. In that case example 2 is inconsistent.

# input
# This should be okay if `let` is considered like a `with` or `assert` 
let self = { }; in
self

# nix-fmt output
# But it is expanded :-(
let
  self = {};
in
self

Overall, it seems that the let ... in body is expanded (split across multiple lines as long as it is not written on one line. I think that expanding the let ... in part should be independent from expanding the body itself and from decoupling the in from the body.

The above example is made even worse by the fact that we do not indent toplevel let bodies (#108). The self really looks weird and could be on the same line as the in.

May I propose the following patterns ?

let foo = bar; in foo

let foo = bar; in
foo

let
  foo = bar;
in {
  bar = foo;
}

The implicit rule is that when the let+in construct is one one line, it behaves like a with or assert, and when it is on several lines (expanded form), no line break is enforced after the in (which means that line breaks are only enforced for a multi-line body after a single line let+in).

Build failure on Darwin

On Darwin "cargo build" fails with the following error message:

  = note: ld: framework not found Security
          clang-7: error: linker command failed with exit code 1 (use -v to see invocation)

I think that darwin.apple_sdk.frameworks.Security should be added to the buildInputs.

sample: normalize indentation to tabs or spaces

Input

let
  pkgs = ((import (fetchTarball https://github.com/NixOS/nixpkgs/archive/staging.tar.gz))  {});
in
pkgs.buildEnv {
  name = "innovu-httpd";
  paths = [ pkgs.apacheHttpd pkgs.modsecurity_standalone ];
  postBuild = ''
    # no tabs
		# tabs
  '';
}

Output

let
  pkgs = ((import (fetchTarball https://github.com/NixOS/nixpkgs/archive/staging.tar.gz)) {});
in
pkgs.buildEnv {
  name = "innovu-httpd";
  paths = [ pkgs.apacheHttpd pkgs.modsecurity_standalone ];
  postBuild = ''
        # no tabs
    		# tabs
  '';
}

Desired output

Ideally the output should confirm to whatever the nix language specifies or the conventions of nixpkgs. I don't really care whether it is tabs or spaces. The below uses spaces just to show the indentation consistency in GitHub's markdown code block:

let
  pkgs = ((import (fetchTarball https://github.com/NixOS/nixpkgs/archive/staging.tar.gz))  {});
in
pkgs.buildEnv {
  name = "innovu-httpd";
  paths = [ pkgs.apacheHttpd pkgs.modsecurity_standalone ];
  postBuild = ''
    # no tabs
    # tabs
  '';
}

sample: balanced let block

It would better if the body was balanced with the set entries when the let block is not on the root of the file:

Input

{
  myAttr = foo:
    let
 hello = 3; in hello;
}

Output

{
  myAttr = foo:
    let
      hello = 3;
    in
    hello;
}

Desired output

{
  myAttr = foo:
    let
      hello = 3;
    in
      hello;
}

Create VSCode extension

nixpkgs-fmt is already available as a WASM module, and you can use WASM in VSCode extensions as described at microsoft/vscode#65559 (comment), so it looks like it would be possible to make a formatter extension for VSCode.

sample: function argument comments

src/rules.rs apply_arg should probably also look if the node is a comment:

Input

{
  foo =
    # describe bar
    bar:
    # describe baz
    baz:
      fnbody;
}

Output

{
  foo =
  # describe bar
      bar:
      # describe baz
          baz:
            fnbody;
}

Desired output

See input

Fuzzing and propery-based testing?

It would be nice to run the code through more extensive tests.

From #70 (comment)

We use libfuzzer-sys in rust-analyzer:

https://github.com/rust-analyzer/rust-analyzer/tree/master/crates/ra_syntax/fuzz

https://github.com/rust-fuzz/libfuzzer-sys

Another approach is to use property-based testing + randomized text edits.

sample: text block is not formatted correctly

Input

{ lib, ... }:

with lib;

{
  nix.extraOptions = ''
    builders-use-substitutes = true
  '';
}

Output

{ lib, ... }:

with lib;

  {
    nix.extraOptions = ''
    builders-use-substitutes = true
  '';
  }

Expected

    nix.extraOptions = ''
    	builders-use-substitutes = true
  	'';

URLs and paths

We should rewrite URLs like http://example.com to string "http://example.com"

Should we do the same for paths like ./some/dir? Or are paths and urls are completely different things? rnix uses TOKEN_PATH for both of these, and I wonder if that should be fixed.

Avoid opening a scope for parentheses and for single-element lists.

Looking at the diff produced by @zimbatm on discourse, the main differences that I see are on using compact forms to avoid extra lines.
For example:

diff --git a/maintainers/maintainer-list.nix b/maintainers/maintainer-list.nix
index 7ff22fb61fa..87d2ca3aa43 100644
--- a/maintainers/maintainer-list.nix
+++ b/maintainers/maintainer-list.nix
@@ -51,10 +51,12 @@
     name = "Joachim Ernst";
     github = "0x4A6F";
     githubId = 9675338;
-    keys = [{
-      longkeyid = "rsa8192/0x87027528B006D66D";
-      fingerprint = "F466 A548 AD3F C1F1 8C88  4576 8702 7528 B006 D66D";
-    }];
+    keys = [
+      {
+        longkeyid = "rsa8192/0x87027528B006D66D";
+        fingerprint = "F466 A548 AD3F C1F1 8C88  4576 8702 7528 B006 D66D";
+      }
+    ];
   };
   "1000101" = {
     email = "[email protected]";

and this occurs in many places, like for fetch* calls

diff --git a/pkgs/development/python-modules/pyscard/default.nix b/pkgs/development/python-modules/pyscard/default.nix
index bab8e502b9c..e601f23ada9 100644
--- a/pkgs/development/python-modules/pyscard/default.nix
+++ b/pkgs/development/python-modules/pyscard/default.nix
@@ -19,10 +19,12 @@ buildPythonPackage rec {
   patches = [
     # Fixes darwin tests
     # See: https://github.com/LudovicRousseau/pyscard/issues/77
-    (fetchpatch {
-      url = "https://github.com/LudovicRousseau/pyscard/commit/62e675028086c75656444cc21d563d9f08ebf8e7.patch";
-      sha256 = "1lr55npcpc8j750vf7vaisqyk18d5f00l7nii2lvawg4sssjaaf7";
-    })
+    (
+      fetchpatch {
+        url = "https://github.com/LudovicRousseau/pyscard/commit/62e675028086c75656444cc21d563d9f08ebf8e7.patch";
+        sha256 = "1lr55npcpc8j750vf7vaisqyk18d5f00l7nii2lvawg4sssjaaf7";
+      }
+    )
   ];

There seems to be an implicit rule that collapsing delimiters (parentheses, square brackets and such) is desirable if there is only one expression inside. Furthermore, the collapsing results in a single indent, instead of two (or more).

In short:

Input

{ 
  testA= [(fetchurl {
  url = "bla";
})];

 testB = [( fetchurl{url="bla";} )];
}

Output

{
  testA = [
    (
      fetchurl {
        url = "bla";
      }
    )
  ];

  testB = [ (fetchurl { url = "bla"; }) ];
}

Desired output

{
  testA = [ 
    (fetchurl {
      url = "bla";
    })
  ];

  testB = [ (fetchurl { url = "bla"; }) ];
}

{
  testA = [ (fetchurl {
    url = "bla";
  }) ];

  testB = [ (fetchurl { url = "bla"; }) ];
}

or (mind the space changes)

{
  testA = [(fetchurl {
    url = "bla";
  })];

  testB = [ (fetchurl { url = "bla"; }) ];
}

Of course, when the list constains mutiple elements, we should switch to a safer form.

Input

{ 
  testA= [(fetchurl {
  url = "bla";
})(fetchurl {
  url = "foo";
})];

 testB = [( fetchurl{url="bla";} )( fetchurl{url="foo";} )];
}

Output

{
  testA = [
    (
      fetchurl {
        url = "bla";
      }
    )
    (
      fetchurl {
        url = "foo";
      }
    )
  ];

  testB = [ (fetchurl { url = "bla"; }) (fetchurl { url = "foo"; }) ];
}

Desired output

{
  testA = [
    (fetchurl {
      url = "bla";
    })
    (fetchurl {
      url = "foo";
    })
  ];

  testB = [ (fetchurl { url = "bla"; }) (fetchurl { url = "foo"; }) ];
}

The simplest way to achieve this would be to make parentheses special. When they are expanded (split on several lines) do not force a line break after them. that would still keep the following desired output impossible, but get all the other options above right.

{
  testA = [(fetchurl {
    url = "bla";
  })];

  testB = [ (fetchurl { url = "bla"; }) ];
}

sample: if statement after key

Input

{
  userHome = if home != null
    then home
    else config.users.extraUsers.${user}.home;
}

Output

{
  userHome = if home != null
  then home
  else config.users.extraUsers.${user}.home;
}

Desired output

{
  userHome = if home != null
    then home
    else config.users.extraUsers.${user}.home;
}

or maybe

{
  userHome =
    if home != null
    then home
    else config.users.extraUsers.${user}.home;
}

sample: `++` folding

nixpkgs-fmt should respect \n and never fold lines back into one

Input

foo ++
bar

Output

foo ++ bar

Desired output

foo
++ bar

attset indentation issue

Another variation of #20. This is the diff I am getting on a correctly-formatted code:

   catAttrs = builtins.catAttrs or
-    (attr: l: concatLists (map (s: if s ? ${attr} then [s.${attr}] else []) l));
+  (attr: l: concatLists (map (s: if s ? ${attr} then [ s.${attr} ] else []) l));

Still fighting the same conflict we have with keeping the balanced trees.

sample: multi-line parenthesis

When a parenthesis is multi-line it should indent the inner bits.

Input

{
  xxx = listToAttrs (
  concatMap (name:
    let a = 4; in 5
));
}

Output

{
  xxx = listToAttrs (
  concatMap (name:
    let a = 4; in 5
  ));
}

Desired output

{
  xxx = listToAttrs (
    concatMap (name:
      let a = 4; in 5
    )
  );
}

trace: assert failed: hash mismatch for file '/nix/store/7jaw5xi1f5nbhbyfhzasvkzgaws03f9q-master.tar.gz/Cargo.lock':

Describe the bug

nix-env -f https://github.com/nix-community/nixpkgs-fmt/archive/master.tar.gz -i
trace: assert failed: hash mismatch for file '/nix/store/7jaw5xi1f5nbhbyfhzasvkzgaws03f9q-master.tar.gz/Cargo.lock':
  wanted: sha256:d6e3edd1c5e96ea40c8ffe5fedf110ad7e903a478d4ae999186023819388f8cb
  got:    sha256:36f58d1ddf4d56bc75aa1d18bb7c4e894eb56c2a0f4a81d83c3f24aec2e98926

System information

system: "x86_64-linux"
host os: Linux 4.19.66, NixOS, 19.09.git.4557b9f (Loris)
multi-user?: yes
sandbox: yes
version: nix-env (Nix) 2.2.2
channels(root): "nixos-19.09pre188498.4557b9f1f50"
channels(bbigras2): "home-manager"
nixpkgs: /nix/var/nix/profiles/per-user/root/channels/nixos

sample: wrap if statements

Input

if true==1231234123412341234123412341234123412341234132412341234123412341234123412341234123412341234 then foo else bar

Output

if true == 1231234123412341234123412341234123412341234132412341234123412341234123412341234123412341234 then foo else bar

Desired output

if
  true == 1231234123412341234123412341234123412341234132412341234123412341234123412341234123412341234
then foo
else bar

if statements that are too long should be wrapped

sample: if then else indent

Input

 if
 foo
 then
 bar
 else
 baz

Output

if
foo
then
bar
else
baz

Desired output

if
  foo
then
  bar
else
  baz

RuntimeError: unreachable executed

Describe the bug

Formatting this code produces a runtime error:

{ } =

To Reproduce
Steps to reproduce the behavior:

Run echo "{} =" > sample.nix
Run `nixpkgs-fmt ./sample.nix'

thread 'main' panicked at 'assertion failed: prev.is_none()', src/engine/fmt_model.rs:267:9
note: Run with `RUST_BACKTRACE=1` environment variable to display a backtrace.

Expected behavior
No runtime error

System information

nixpkgs-fmt version: master 81e2341

sample: output converges after 2 tries

This sample takes two tries to stabilize

Input

{
  buildInputs = [
  ] ++ stdenv.lib.optionals enableGui (with qt5; [ qtbase qtwebkit ])
    ++ stdenv.lib.optionals enableJupyter [ boost jsoncpp openssl zmqpp ]
  ;
}

Output

{
  buildInputs = []
  ++ stdenv.lib.optionals enableGui (with qt5; [ qtbase qtwebkit ])
  ++ stdenv.lib.optionals enableJupyter [ boost jsoncpp openssl zmqpp ]
  ;
}

After feeding the output back as an input:

{
  buildInputs = []
    ++ stdenv.lib.optionals enableGui (with qt5; [ qtbase qtwebkit ])
    ++ stdenv.lib.optionals enableJupyter [ boost jsoncpp openssl zmqpp ]
    ;
}

sample: Multiline comments cause an inconsistent amount of items per line

Just played around in the WASM example a little.

Input

[ one /*two*/ three four five six seven eight nine
]

Output

[
  one /*two*/ three
  four
  five
  six
  seven
  eight
  nine
]

Desired output

[
  one /*two*/
  three
  four
  five
  six
  seven
  eight
  nine
]

or maybe even the comment should be on its own line, not sure.

sample: demo for @nbp

Input

# NixOS module handling.

let lib = import ./default.nix; in

with { inherit (builtins) head tail; };
with import ./trivial.nix;
with import ./lists.nix;
with import ./misc.nix;
with import ./attrsets.nix;
with import ./properties.nix;

rec {

  # Unfortunately this can also be a string.
  isPath = x: !(
     builtins.isFunction x
  || builtins.isAttrs x
  || builtins.isInt x
  || builtins.isBool x
  || builtins.isList x
  );

  importIfPath = path:
    if isPath path then
      import path
    else
      path;

  applyIfFunction = f: arg:
    if builtins.isFunction f then
      f arg
    else
      f;

  moduleClosure = initModules: args:
    let
      moduleImport = m:
        (applyIfFunction (importIfPath m) args) // {
          # used by generic closure to avoid duplicated imports.
          key = m;
        };

      removeKeys = list: map (m: removeAttrs m ["key"]) list;

      getImports = m:
        if m ? config || m ? options then
          attrByPath ["imports"] [] m
        else
          toList (rmProperties (attrByPath ["require"] [] (delayProperties m)));

      getImportedPaths = m: filter isPath (getImports m);
      getImportedSets = m: filter (x: !isPath x) (getImports m);

      inlineImportedSets = list:
        lib.concatMap (m:[m] ++ map moduleImport (getImportedSets m)) list;
    in
      removeKeys (inlineImportedSets (lazyGenericClosure {
        startSet = map moduleImport initModules;
        operator = m: map moduleImport (getImportedPaths m);
      }));

  selectDeclsAndDefs = modules:
    lib.concatMap (m:
      if m ? config || m ? options then
         attrByPath ["options"] [] m
      ++ attrByPath ["config"] [] m
      else
        [ m ]
    ) modules;

}

Output

# NixOS module handling.

let
  lib = import ./default.nix;
in

with { inherit (builtins) head tail; };
with import ./trivial.nix;
with import ./lists.nix;
with import ./misc.nix;
with import ./attrsets.nix;
with import ./properties.nix;

rec {

  # Unfortunately this can also be a string.
  isPath = x: !(
    builtins.isFunction x
    || builtins.isAttrs x
    || builtins.isInt x
    || builtins.isBool x
    || builtins.isList x
  );

  importIfPath = path:
    if isPath path then
      import path
    else
      path;

  applyIfFunction = f: arg:
    if builtins.isFunction f then
      f arg
    else
      f;

  moduleClosure = initModules: args:
    let
      moduleImport = m:
        (applyIfFunction (importIfPath m) args)
        // {
             # used by generic closure to avoid duplicated imports.
             key = m;
           };

      removeKeys = list: map (m: removeAttrs m [ "key" ]) list;

      getImports = m:
        if m ? config || m ? options then
          attrByPath [ "imports" ] [] m
        else
          toList (rmProperties (attrByPath [ "require" ] [] (delayProperties m)));

      getImportedPaths = m: filter isPath (getImports m);
      getImportedSets = m: filter (x: !isPath x) (getImports m);

      inlineImportedSets = list:
        lib.concatMap (m:[ m ] ++ map moduleImport (getImportedSets m)) list;
    in
    removeKeys (
      inlineImportedSets (
        lazyGenericClosure {
          startSet = map moduleImport initModules;
          operator = m: map moduleImport (getImportedPaths m);
        }
      )
    );

  selectDeclsAndDefs = modules:
    lib.concatMap (
      m:
        if m ? config || m ? options then
          attrByPath [ "options" ] [] m
          ++ attrByPath [ "config" ] [] m
        else
          [ m ]
    ) modules;

}

Desired output

slight formatting issue: brace is left alone in args

Input

{foo,
bar}:
              foo+bar

Output

{
  foo
, bar
}:
foo + bar

Desired output

This seems more canonical in nixpkgs:

{ foo
, bar
}:
foo + bar

sample: // forced to next line

Coming form https://discourse.nixos.org/t/nixpkgs-fmt-beta-0-3-1/3812/2?u=zimbatm

/cc @lilyball

Input

{
  meta = (oldAttrs.meta or {}) // {
    platforms = pkgs.rustc.meta.platforms;
  };
}

Output

{
  meta = (oldAttrs.meta or {})
    // {
         platforms = pkgs.rustc.meta.platforms;
       }
    ;
}

Desired output

see input?

multi-line string is indented wrong in paranthese

(let
  a = ''
    foo
  '';
 in a)

results in

(
  let
    a = ''
      foo
      '';
  in
    a
)

instead of

(
  let
    a = ''
      foo
    '';
  in
    a
)

a more practical example would be:

wpa_supplicant.overrideAttrs(old: {
  postBuild = ''
    echo foo
  '';
})

wpa_supplicant.overrideAttrs (
  old: {
    postBuild = ''
      echo foo
      '';
  }
)

indentation of buildInputs is very odd

Not sure what the proper way to do this would be, but the nested indentation is not right.

Input

{ mkDerivation }:

mkDerivation rec {

  buildInputs = [
    alsaLib
  ]  ++ lib.optionals stdenv.isLinux [
     utillinux
  ] ++ (with gst_all_1; [
     gstreamer
  ]);
}

Output

{ mkDerivation }:

mkDerivation rec {

  buildInputs = [
    alsaLib
  ]
    ++ lib.optionals stdenv.isLinux [
         utillinux
       ]
    ++ (
         with gst_all_1; [
           gstreamer
         ]
       )
    ;
}

Desired output

{ mkDerivation }:

mkDerivation rec {

  buildInputs = [
    alsaLib
  ]
  ++ lib.optionals stdenv.isLinux [
    utillinux
  ]
  ++ (with gst_all_1; [
    gstreamer
  ])
  ;
}

sample: RuntimeError: unreachable executed

Input

{
  traceIf =
    # predicate to check
    pred:

Output

RuntimeError: unreachable executed

Design the CLI

There are essentially two modes of operation for the CLI:

on a buffer: takes a buffer in with some nix code, outputs a formatted nix code
on a dir: scan all the nix files in a folder and format them. with options to ignore some folders like .git

Github Pages version of the WASM formatter doesn't load a needed JS file

I am so excited for this. I tried out the existing version on master at https://nix-community.github.io/nixpkgs-fmt/ and it fails to load a javascript file.

The script from “https://nix-community.github.io/nixpkgs-fmt/pkg/nixpkgs_fmt_wasm.js” was loaded even though its MIME type (“text/html”) is not a valid JavaScript MIME type.[Learn More] nixpkgs-fmt
Loading failed for the module with source “https://nix-community.github.io/nixpkgs-fmt/pkg/nixpkgs_fmt_wasm.js”. nixpkgs-fmt:54:1

at this time the file 404s.

Way to go on the WASM idea. This is incredible.

sample: unindented `++`s after a new line

Input

pkgs.example rec {
  buildInputs = [ pkgs.foo  ] ++
    pkgs.stdenv.lib.optionals pkgs.stdenv.isDarwin [
      pkgs.bar
    ];
}

Output

pkgs.example rec {
  buildInputs = [ pkgs.foo ]
  ++ pkgs.stdenv.lib.optionals pkgs.stdenv.isDarwin [ # <- I'm surprised this isn't indentend two spaces
    pkgs.bar
  ]
  ;
}

Not so sure about this one, though.

sample: `builtins.fold' (` causes funky indentation for following arguments

Input

    (
      builtins.foldl' (
        acc: v: let
          isOperator = builtins.typeOf v == "list";
          operator = if isOperator then (builtins.elemAt v 0) else acc.operator;
        in
          if isOperator then (acc // { inherit operator; }) else {
            inherit operator;
            state = operators."${operator}" acc.state (satisfiesSemver pythonVersion v);
          }
      )
        {
          operator = ",";
          state = true;
        }
        tokens
    ).state;

Output

(
  builtins.foldl' (
    acc: v: let
      isOperator = builtins.typeOf v == "list";
      operator = if isOperator then (builtins.elemAt v 0) else acc.operator;
    in
      if isOperator then (acc // { inherit operator; }) else {
        inherit operator;
        state = operators."${operator}" acc.state (satisfiesSemver pythonVersion v);
      }
  )
    {
      operator = ",";
      state = true;
    }
    tokens
).state;

Desired output

(
  builtins.foldl'
    (
      acc: v: let
        isOperator = builtins.typeOf v == "list";
        operator = if isOperator then (builtins.elemAt v 0) else acc.operator;
      in
        if isOperator then (acc // { inherit operator; }) else {
          inherit operator;
          state = operators."${operator}" acc.state (satisfiesSemver pythonVersion v);
        }
    )
    {
      operator = ",";
      state = true;
    }
    tokens
).state;

sample: function body indent

I think the function body shouldn't be indented if the arguments are on the same level.

Input

let
  aaa =
 {foo,bar}:
              foo+bar
;
bbb = {foo,bar}:
              foo+bar
;
in
  null

Output

let
  aaa =
    { foo, bar }:
      foo + bar
  ;
  bbb = { foo, bar }:
    foo + bar
  ;
in
null

Desired output

let
  aaa =
    { foo, bar }:
    foo + bar
    ;
  bbb = { foo, bar }:
    foo + bar
    ;
in
null

sample: comments are not aligned with function body

Input

let
id =
x:
# comment
x;
in
id 1

Output

let
  id =
    x:
    # comment
      x;
in
id 1

Desired output

let
  id =
    x:
      # comment
      x;
in
id 1

sample:

Input

{config,pkgs, lib      =wfp}: {}

Output

{ config, pkgs, lib      =wfp }: {}

Desired output

{ config, pkgs, lib = wfp }: {}

sample: multi-line strings should be balanced

The multi-line string should behave more like the list [ and ]

Input

{
  string =          ''
hello
'';
  list = [
hello
];
}

Output

{
  string = ''
    hello
    '';
  list = [
    hello
  ];
}

Desired output

{
  string = ''
    hello
  '';
  list = [
    hello
  ];
}

Add a way for users to submit code samples

As a user I want an easy way to report formatting issues.

A possible implementation would be:

Go to a webpage
Paste my code. The code is automatically formatted using a WebAssembly build of the project.
Change the formatted code to my liking
Click "Create issue" and be brought to a pre-filled issue entry on GitHub. ( eg: https://github.com/nix-community/nixpkgs-fmt/issues/new?title=foo&body=bar )

Haskell comma in function parameters

I see two different patterns which are used to format function arguments:

Haskell comma:

{ pkgs ? import ./nix/nixpkgs.nix {}
, src ? builtins.fetchGit {
    url = ./.;
    ref = "HEAD";
  }
}:
pkgs.example rec {}

More traditional, with comma after the pattern:

  toINI = {
    mkSectionName ? (name: libStr.escape [ "[" "]" ] name),
    mkKeyValue    ? mkKeyValueDefault {} "="
  }: attrsOfAttrs:
    let ...

which one we should use?

Additionally, is it a good idea to to keep first parameter on the same line, as in { pkgs ? import ./nix/nixpkgs.nix {} ?

Scientific notation is corrupted

Describe the bug
nixpkgs-fmt changes the meaning of scientific notation.

To Reproduce

Format the following expression:

3.9e-3

It will format this as

3.9 e-3

which is not the same expression.

Expected behavior

The file is unchanged

System information

nixpkgs-fmt --version

nixpkgs-fmt 0.6.0

Additional context
Add any other context about the problem here.

WASM Demo: Style diff output for better readability

Right now it looks like this:

It would be more readable when using the usual colors red and green for the changed lines.

See for example GitHub: https://github.com/nix-community/nixpkgs-fmt/pull/110/files

green: #e6ffed
red: #ffeef0

nix-community / nixpkgs-fmt Goto Github PK

nixpkgs-fmt's Introduction

nixpkgs-fmt - Nix code formatter for nixpkgs

Demo

Design decisions

Usage

Tree traversal

Installation

VSCode extensions

pre-commit hook

Development

Running Fuzzer

Documentation

Related projects

Using nixpkgs-fmt

Formatters

Linters

Parsers

Discussions

Sponsors

nixpkgs-fmt's People

Contributors

Stargazers

Watchers

Forkers

nixpkgs-fmt's Issues

Input

Output

Input

Output

Input

Output

Desired output

Input

Output

Desired output

Input

Output

Expected output

Input

Output

Desired output

Input

Output

Desired output

Input

Output

Desired output

Input

Output

Input

Output

Desired output

Input

Output

Desired output

Input

Output

Desired output

Input

Output

Desired output

Input

Output

Desired output

Input

Output

Desired output

Input

Output

Desired output

Input

Output

Input

Output

Desired output

Input

Output

Desired output

Input