Code Monkey home page Code Monkey logo

spago's Introduction

spago

npm Latest release build Maintainer: f-f

(IPA: /หˆspaษกo/)

PureScript package manager and build tool.

Spago logo - a 3d box containing a blob of spaghetti

Installation

Important

This documentation concerns the new PureScript rewrite of Spago. If you are looking for the Haskell codebase (versions up to 0.21.x), please head over to the spago-legacy repo.

Warning

This new Spago is still in alpha, so while most of it works well, there will be some rough edges here and there. Please report them if you find any!

The recommended installation method for Windows, Linux and macOS is npm (see the latest releases on npm here):

npm install -g spago@next

Other installation methods available:

General notes:

  • The assumption is that you already installed the PureScript compiler. If not, get it with npm install -g purescript, or the recommended method for your OS.
  • You might have issues with npm and Docker (e.g. getting the message "Downloading the spago binary failed.." etc) You have two options:
    • either do not run npm as root, because it doesn't work well with binaries. Use it as a nonprivileged user.
    • or use --unsafe-perm: npm install -g --unsafe-perm spago@next

Super quick tutorial

Let's set up a new project!

$ mkdir purescript-pasta
$ cd purescript-pasta
$ spago init

This last command will create a few files:

.
โ”œโ”€โ”€ spago.yaml
โ”œโ”€โ”€ src
โ”‚ย ย  โ””โ”€โ”€ Main.purs
โ””โ”€โ”€ test
    โ””โ”€โ”€ Test
        โ””โ”€โ”€ Main.purs

If you have a look at the spago.yaml file, you'll see that it contains two sections:

  • the workspace section, which details the configuration for the dependencies of the project as a whole (which can be a monorepo, and contain more than one package), and other general configuration settings. In this sample project, the only configuration needed is the package set version from which all the dependencies will be chosen. See here for more info about how to query the package sets.
  • the package section, that is about the configuration of the package at hand, such as its name, dependencies, and so on.

For more info about all the various configuration settings, visit the section about the configuration format.

To build and run your project, use:

$ spago run

This will:

  • download and compile the necessary dependencies (equivalent to spago install)
  • compile this sample project in the output/ directory (equivalent to spago build).
    You can take a look at the content of output/Main/index.js to see what kind of JavaScript has been generated from your new Main.purs file
  • run the generated JS, which is roughly equivalent to running
    $ node -e "import('./output/Main/index').then(m => m.main())"
    The above code imports the JS file you just looked at, and runs its main with Node.

You can also bundle the project in a single file with an entry point, so it can be run directly (useful for CLI apps):

$ spago bundle --bundle-type app --platform node
$ node .

Great! If you read unitl here you should be set to go write some PureScript without worrying too much about the build ๐Ÿ˜Š

Where to go from here? There are a few places you should check out:

Table of contents

Design goals and reasons

Our main design goals are:

  • Great UX: a good build system just does what's most expected and gets out of the way so you can focus on actually thinking about the software itself, instead of spending your time configuring the build.
  • Minimal dependencies: users should not be expected to install a myriad of tools on their system to support various workflows. Spago only expects git and purs to be installed.
  • Reproducible builds: we exploit package sets and lock files to make your build reproducible, so that if your project builds today it will also build tomorrow and every day after that.

Some tools that inspired spago are: Rust's Cargo, Haskell's Stack, psc-package, pulp and bazel.

Developing and contributing

We'd love your help, and welcome PRs and contributions!

Some ideas for getting started:

For more details see the CONTRIBUTING.md

How do I...

This section contains a collection of mini-recipes you might want to follow in order to get things done with Spago.

Migrate from spago.dhall to spago.yaml

You'll need to use spago-legacy for this.

# Install spago-legacy
npm install -g spago-legacy
# You can then create a `spago.yaml` file with `migrate`
spago-legacy migrate

# Ready to remove the dhall files and move to the new spago
npm install -g spago@next
rm spago.dhall packages.dhall

Note

The spago-legacy NPM package has exactly the same content as the [email protected] release, which is the last one released from the legacy codebase, and the only release to contain the migrate command. The new codebase's releases start from [email protected].

Some packages might not be found or have the wrong version, in which case you'll have to carefully:

In all cases, you'll want to switch to the new Registry package sets, so replace something like this:

workspace:
  packageSet:
    url: https://raw.githubusercontent.com/purescript/package-sets/psc-0.15.10-20230919/packages.json

...with this:

workspace:
  packageSet:
    registry: 41.2.0

This is because the legacy package set format - while being supported - is pointing at git repositories, so Spago will fetch them using git, which can get quite slow and error-prone.

The new package sets are instead pointing at the Registry, and can fetch compressed archives from our CDN, which is much faster and more reliable.

To figure out what package set you're supposed to be using, see the section about querying package sets.

You might also want to check out the section about differences from legacy spago.

Migrate from bower

Same as above, but with an additional spago init command just after you install spago-legacy, so that the bower.json file is converted into a spago.dhall file.

See what commands and flags are supported

For an overview of the available commands, run:

$ spago --help

You will see several subcommands (e.g. build, test); you can ask for help about them by invoking the command with --help, e.g.:

$ spago build --help

This will give a detailed view of the command, and list any flags you can use with that command.

Setup a new project using a specific package set

Since spago init does not necessarily use the latest package set. Fortunately, you can specify which package set to use via the --package-set flag:

$ spago init --package-set 41.2.0

See here for how to ask Spago which sets are available for use.

Setup a new project using the solver

Package sets are the default experience to ensure that you always get a buildable project out of the box, but one does not necessarily have to use them.

If you'd like to set up a project that uses the Registry solver to figure out a build plan, you can use:

$ spago init --use-solver

When using the solver (and when publishing a package), it's important to specify the version bounds for your dependencies, so that the solver can figure out a build plan.

You can ask Spago to come up with a good set of bounds for you by running:

$ spago install --ensure-ranges

Install a direct dependency

To add a dependency to your project you can run:

# E.g. installing Halogen
$ spago install halogen

# This also supports multiple packages
$ spago install foreign aff

If you are using the Registry solver then the package must be available in the Registry, while if you are using package sets it needs to be contained in the set. See here to know more about adding more packages to the local set.

Download my dependencies locally

$ spago install

This will download and compile all the transitive dependencies of your project (i.e. the direct dependencies, i.e. the ones listed in the dependencies key of spago.yaml, plus all their dependencies, recursively) to the local .spago folder.

However, running this directly is usually not necessary, as all commands that need the dependencies to be installed will run this for you.

Running spago fetch is equivalent, but skips the compilation step.

Build and run my project

You can build the project and its dependencies by running:

$ spago build

This is mostly just a thin layer above the PureScript compiler command purs compile.

Note: by default the build command will try to install any dependencies that haven't been fetched yet - if you wish to disable this behaviour, you can pass the --no-install flag.

The build will produce very many JavaScript files in the output/ folder. These are ES modules, and you can just import them e.g. on Node.

Note

The wrapper on the compiler is so thin that you can pass options to purs. E.g. if you wish to ask purs to emit errors in JSON format, you can run

$ spago build --purs-args "--json-errors"

However, some purs flags are covered by Spago ones, e.g. to change the location of the output folder:

$ spago build --output myOutput

To run a command before a build you can use the --before flag, eg to post a notification that a build has started:

$ spago build --before "notify-send 'Building'"

To run a command after the build, use --then for successful builds, or --else for unsuccessful builds:

$ spago build --then "notify-send 'Built successfully'" --else "notify-send 'Build failed'"

Multiple commands are possible - they will be run in the order specified:

$ spago build --before clear --before "notify-send 'Building'"

If you want to run the program, just use run:

$ spago run -p package-name -m Module.Containing.Main

# We can pass arguments through to `purs compile`
$ spago run -p package-name  -m Module.Containing.Main --purs-args "--verbose-errors"

# We can pass arguments through to the program being run
$ spago run -p package-name  -m Module.Containing.Main -- arg1 arg2

Oof! That's a lot of typing. Fortunately it's possible to configure most of these parameters in the package.run section of your configuration file, so you don't have to supply them at the command line.

See here for more info about this, but it allows us to instead write:

# The main module can be defined in the configuration file, but
# purs flags still need to be supplied at the command line
spago run -p package-name --purs-args "--verbose-errors"

# It's possible to even pass arguments from the config, which would look like this:
#
# package:
#   run:
#       main: Main
#       execArgs:
#         - "arg1"
#         - "arg2"
$ spago run -p package-name

Lastly, if you only have a single package defined in the workspace with these parameters defined in the config file, you can just run

spago run

Test my project

You can also test your project with spago:

# Test.Main is the default here, but you can override it as usual
$ spago test --main Test.Main
Build succeeded.
You should add some tests.
Tests succeeded.

As with the run command, it's possible to configure the tests using the spago.yaml - most importantly to separate test dependencies from the dependencies of your application/library.

Please see the section about the configuration format for more info, but in the meantime note that it's possible to install test dependencies by running:

$ spago install --test-deps spec

Run a repl

You can start a repl with the following command:

$ spago repl

Run a standalone PureScript file as a script

You can run a standalone PureScript file as a script via spago script. Note: The module name must be Main, and it must export a function main :: Effect Unit.

By default, the following dependencies are installed: effect, console, prelude.

You can run a script via the following, optionally specifying a package set to use, and additional dependencies to pull from there:

$ spago script --package-set 41.2.0 -d node-fs path/to/script.purs

List available packages

It is sometimes useful to know which packages are contained in our package set (e.g. to see which version we're using, or to search for packages).

You can get a complete list of the packages provided by your workspace (together with their versions, locations, and license) by running:

$ spago ls packages

By using the ls deps command instead you can restrict the list to direct or transitive dependencies:

# Direct dependencies, i.e. only the ones listed in spago.dhall
$ spago ls deps

# Transitive dependencies, i.e. all the dependencies of your dependencies
$ spago ls deps --transitive

You can provide the --json flag for a more machine-friendly output.

Install all the packages in the set

There might be cases where you'd like your project to depend on all the packages that are contained in the package set (this is sometimes called "acme build").

If you have jq installed, you can accomplish this in relatively few characters:

$ spago ls packages --json | jq -r 'keys[]' | xargs spago install

Override a package in the package set with a local one

Let's say I'm a user of the popular aff package. Now, let's say I stumble upon a bug in there, but thankfully I figure how to fix it. So I clone it locally and add my fix.

Now if I want to test this version in my current project, how can I tell spago to do it?

There's a section of the spago.yaml file just for that, called extraPackages.

In this case we override the package with its local copy, which should have a spago.yaml - our workspace will look something like this:

workspace:
    registry: 41.2.0
  extraPackages:
    aff:
      path: ../my-purescript-aff

Now if we run spago ls packages, we'll see that it is now included as a local package:

$ spago ls packages
+----------------------------------+------------------------------------------+------------------------------------------------+
| Package                          | Version                                  | Location                                       |
+----------------------------------+------------------------------------------+------------------------------------------------+
| abc-parser                       | 2.0.0                                    | -                                              |
| ace                              | 9.1.0                                    | -                                              |
| aff                              | local                                    | ../my-purescript-aff                           |
| aff-bus                          | 6.0.0                                    | -                                              |
| aff-coroutines                   | 9.0.0                                    | -                                              |
| aff-promise                      | 4.0.0                                    | -                                              |
...

Override a package in the package set with a remote one

Let's now say that we test that our fix from above works, and we are ready to Pull Request the fix.

So we push our fork and open the PR, but we want to already use the fix in our build, while we wait for it to land upstream and then on the next package set.

In this case, we can just change the override to point to some commit of our fork, like this:

workspace:
    registry: 41.2.0
  extraPackages:
    aff:
      git: https://github.com/my-user/purescript-aff.git
      ref: aaa0aca7a77af368caa221a2a06d6be2079d32da

Warning

You can use a "branch", a "tag" or a "commit hash" as a version. It's strongly recommended to avoid using branches, because if you push new commits to a branch, spago won't pick them up unless you delete the .spago/packages/aff/your-branch folder.

Add a package to the package set

Important

You still need to spago install my-new-package after adding it to the package set, or Spago will not know that you want to use it as a dependency!

If a package is not in the upstream package set, you can add it exactly in the same way, by adding it to extraPackages.

E.g. if we want to add the facebook package:

workspace:
    registry: 41.2.0
  extraPackages:
    facebook:
      git: https://github.com/Unisay/purescript-facebook.git
      ref: v0.3.0 # branch, tag, or commit hash

Note

If the upstream library that you are adding has a spago.yaml file, then Spago will just pick up the dependencies from there. If that's not the case, then you'll have the provide the dependencies yourself, adding a dependencies field.

As you might expect, this works also in the case of adding local packages:

workspace:
    registry: 41.2.0
  extraPackages:
    facebook:
      path: ../my-purescript-facebook

Querying package sets

Since the versioning scheme for package sets does not tell anything about the compiler version or when they were published, you might want to have a look at the list of all the available ones. You can do that with:

$ spago registry package-sets

This will print a list of all the package sets ever releases, which could be overwhelming, as you'd likely only be interested in the latest one.

This is how you would ask for the latest package sets for each compiler version:

$ spago registry package-sets --latest
+---------+------------+----------+
| VERSION | DATE       | COMPILER |
+---------+------------+----------+
| 10.0.0  | 2023-01-05 | 0.15.4   |
| 20.0.3  | 2023-04-08 | 0.15.7   |
| 27.2.0  | 2023-06-17 | 0.15.8   |
| 29.1.0  | 2023-07-18 | 0.15.9   |
| 42.1.0  | 2023-09-26 | 0.15.10  |
+---------+------------+----------+

Upgrading packages and the package set

If your project is using the Registry solver (i.e. no package set and only version bounds), then running spago upgrade will try to put together a new build plan with the latest package versions published on the Registry, given that they are still compatible with your current compiler.

If instead you are using package sets, then spago upgrade will bump your package set version to the latest package set available for your compiler version.

You can pass the --package-set flag if you'd rather upgrade to a specific package set version. You can of course just edit the workspace.packageSet field in the spago.yaml file.

Custom package sets

Spago supports fetching custom package sets from URLs and paths, so you can build your own package set if you'd like - this is useful for example if you want to put together a custom package set for your company, or if you are using an alternate backend.

Spago will be happy to use a package set from a local path:

workspace:
  packageSet:
    path: ../my-custom-package-set.json

Otherwise you can point Spago to any URL on the internet:

workspace:
  packageSet:
    url: https://raw.githubusercontent.com/purescript/package-sets/psc-0.15.7-20230207/packages.json

...and it will try to fetch the content, parse it as JSON and conform it to one of the possible package set schemas.

The first one is what Spago calls a RemotePackageSet, which contains some metadata, and a map of packages in the shapes (2), (3) and (4) described for extraPackages in the configuration format section.

This package set could look something like this:

{
  compiler: "0.15.10",
  version: "0.0.1",
  packages: {
    "some-registry-package": "1.0.2",
    "some-package-from-git-with-a-spago-yaml": {
      "git": "https://github.com/purescript/registry-dev.git",
      "ref": "68dddd9351f256980454bc2c1d0aea20e4d53fa9"
    },
    "legacy-package-style": {
      "repo": "https://github.com/purescript/purescript-prelude.git",
      "version": "v6.0.1",
      "dependencies": ["prelude", "effect", "console"]
    }
  }
}

The second format possible is what Spago calls a LegacyPackageSet, and it's simply a map from package names to the location of the package, described as the (4) option for how to specify extraPackages in the configuration format section.

Something like this:

{
  "legacy-package-style": {
    "repo": "https://github.com/purescript/purescript-prelude.git",
    "version": "v6.0.1",
    "dependencies": ["prelude", "effect", "console"]
  }
  "metadata": {
    "repo": "https://github.com/purescript/metadata.git",
    "version": "v0.15.10",
    "dependencies": []
  }
}

This is supported to allow for just using legacy package sets, and be able to automatically migrate spago.dhall files to spago.yaml files.

It is not recommended to craft your own package set in the legacy format - please use the RemotePackageSet format instead - but if you do just be aware that you'll need to include a package called metadata that has a version that matches the compiler version that the set is supposed to support.

Graph the project modules and dependencies

You can use the graph command to generate a graph of the modules and their dependencies:

$ spago graph modules

The same goes for packages:

$ spago graph packages

The command accepts the --json and --dot flags to output the graph in JSON or DOT format respectively.

This means that you can pipe the output to other tools, such as [graphviz][graphviz] to generate a visual representation of the graph:

$ spago graph packages --dot | dot -Tpng > graph.png

...which will generate something like this:

packages-graph

Finally, the graph command is also able to return a topological sorting of the modules or packages, with the --topo flag:

$ spago graph modules --topo

Monorepo support

Spago supports "monorepos" (see here as well for more monorepo goodness), allowing you to split a pile of code into different "compilation units" that might have different dependencies, deliverables, etc, but still compile together.

The vast majority of Spago projects will contain only one package, defined in the package section of the same spago.yaml that contains its workspace. It is however possible to define multiple packages in the same repository!

The basic rules are:

  • a package is defined by a spago.yaml file containing a package section.
  • there can be only one workspace section in the whole repository, which defines the "root" of the current Spago Workspace. This defines your package set/build plan.
  • Spago will autodetect all the packages inside the workspace
  • ...except for spago.yaml files with a workspace section, which will be ignored (together with their subfolders, as they establish a the boundary of another "workspace")

For more info about the concept of Spago Workspaces, see the dedicated section.

Since this might sound a little abstract, let's try to picture the case where you might want to have the packages lib1, lib2 and app1.

Then your file tree might look like this:

.
โ”œโ”€โ”€ app1
โ”‚   โ”œโ”€โ”€ spago.yaml
โ”‚   โ”œโ”€โ”€ src
โ”‚   โ”‚   โ””โ”€โ”€ Main.purs
โ”‚   โ””โ”€โ”€ test
โ”‚       โ””โ”€โ”€ Main.purs
โ”œโ”€โ”€ lib1
โ”‚   โ”œโ”€โ”€ spago.yaml
โ”‚   โ””โ”€โ”€ src
โ”‚       โ””โ”€โ”€ Main.purs
โ”œโ”€โ”€ lib2
โ”‚   โ”œโ”€โ”€ spago.yaml
โ”‚   โ”œโ”€โ”€ src
โ”‚   โ”‚   โ””โ”€โ”€ Main.purs
โ”‚   โ””โ”€โ”€ test
โ”‚       โ””โ”€โ”€ Main.purs
โ””โ”€โ”€ spago.yaml

Where:

  • the top level spago.yaml could look like this:

    workspace:
      packageSet:
        registry: 41.2.0
  • and the lib1/spago.yaml would look something like this:

    package:
      name: lib1
      dependencies:
        - effect
        - console
        - prelude
  • then, assuming lib2 depends on lib1, lib2/spago.yaml might look like this:

    package:
      name: lib2
      dependencies:
        - effect
        - console
        - prelude
        - lib1 # <------ Note the dependency here
      test:
        main: Test.Lib2.Main
        dependencies:
          - spec
  • and then app1/spago.yaml would look something like this:

    package:
      name: app1
      # Note that the app does not include all the dependencies that the lib included
      dependencies:
        - prelude
        - aff # This dep was not used by the library
        - lib2 # And we have `lib2` as a dependency

Given this setup, Spago will figure out that there are three separate packages in the repository.

You can select a package to perform operations on it by using the --package flag, e.g. the follow will install the maybe package in the lib1/spago.yaml:

spago install -p lib1 maybe

The --package flag is also available for many more commands, such as build, run, test, bundle and so on.

An important property of this "monorepo setup" is that the output folder will be shared between all the packages: they will share the same build package set (or build plan when using the solver) and they will be all build together.

Polyrepo support

There might be cases where you want to have multiple loosely-connected codebases in the same repository that do not necessarily build together all the time. This is sometimes called a "polyrepo".

One such example of this could be a project that has a frontend and a backend, and they are both written in PureScript, but run on different backends: the frontend runs in the browser (so in JavaScript), and the backend runs on Erlang.

Let's say you might also want to share some code between the two (that was the point of using the same language, no?), so you might have a common package that is used by both.

You can achieve all of this with Spago, by having multiple workspaces - let's try to visualise this.

The file tree might look like this:

.
โ”œโ”€โ”€ client
โ”‚ย ย  โ”œโ”€โ”€ spago.yaml
โ”‚ย ย  โ”œโ”€โ”€ src
โ”‚ย ย  โ”‚ย ย  โ””โ”€โ”€ Main.purs
โ”‚ย ย  โ””โ”€โ”€ test
โ”‚ย ย      โ””โ”€โ”€ Test
โ”‚ย ย          โ””โ”€โ”€ Main.purs
โ”œโ”€โ”€ common
โ”‚ย ย  โ”œโ”€โ”€ spago.yaml
โ”‚ย ย  โ”œโ”€โ”€ src
โ”‚ย ย  โ”‚ย ย  โ””โ”€โ”€ Main.purs
โ”‚ย ย  โ””โ”€โ”€ test
โ”‚ย ย      โ””โ”€โ”€ Test
โ”‚ย ย          โ””โ”€โ”€ Main.purs
โ””โ”€โ”€ server
    โ”œโ”€โ”€ spago.yaml
    โ”œโ”€โ”€ src
    โ”‚ย ย  โ””โ”€โ”€ Main.purs
    โ””โ”€โ”€ test
        โ””โ”€โ”€ Test
            โ””โ”€โ”€ Main.purs

Where the common/spago.yaml is just a package with no workspace defined, as it's going to support both the JS and the Erlang backend:

package:
  name: common
  dependencies:
    - effect
    - console
    - prelude

Then the client/spago.yaml might look like this:

workspace:
  packageSet:
    registry: 41.2.0
  extraPackages:
    common:
      path: ../common
package:
  name: client
  dependencies:
    - prelude
    - common
    - halogen

And the server/spago.yaml might look like this:

workspace:
  packageSet:
    url: https://raw.githubusercontent.com/purerl/package-sets/erl-0.15.3-20220629/packages.json
  backend:
    cmd: purerl
  extraPackages:
    common:
      path: ../common
package:
  name: server
  dependencies:
    - prelude
    - common
    - erl-process

This all means that:

  • there is a Spago Workspace in the client folder, another one in the server folder, but none in the common folder
  • the common package is shared between the two workspaces, note that it's included as a local package in both
  • the client workspace uses the default JS package set, and the server workspace uses a Purerl package set
  • to use each workspace you would need to cd into its folder, so that Spago can detect the workspace

Test dependencies

Like this:

package:
  name: mypackage
  dependencies:
    - effect
    - console
    - prelude
  test:
    main: Test.Main
    dependencies:
      - spec

You can add more with spago install --test-deps some-new-package.

Bundle a project into a single JS file

Use spago bundle.

This is a good-defaults wrapper into esbuild, and it's meant to be used for bundling small projects. Once your project grows in size, you might want to look into configuring esbuild (or parcel, or webpack) directly.

See the esbuild getting started for installation instructions.

This command supports a few options, and the most important ones are:

  • the --bundle-type flag, which can be either app or module
  • the --platform flag, which can be either browser or node

See the help message for more flags, and the configuration format section for how to configure these options in the spago.yaml file.

When bundling an app, the output will be a single executable file:

$ spago bundle --to index.js --bundle-type app --platform node

# It is then possible to run it with node:
$ node index.js

Note

Spago will bundle your project in the esbuild bundle format IIFE.

When bundling a module instead, the output will be a single JS module that you can import from JavaScript:

# You can specify the main module and the target file, or these defaults will be used
$ spago bundle --bundle-type module --main Main --outfile index.js

Can now import it in your Node project:

$ node -e "import('./index.js').then(m => console.log(m.main))"
[Function]

Spago does not wrap the entirety of the bundler's API (esbuild for JS builds), so it's possible to pass arguments through to it. E.g. to exclude an NPM package from the bundle you can pass the --external flag to esbuild:

  • either through the command line, with the --bundler-args flag, i.e. --bundler-args "--external:better-sqlite3".
  • or by adding it to the configuration file:
    package:
      bundle:
        extra_args:
          - "--external:better-sqlite3"

Enable source maps

When bundling, you can include --source-maps to generate a final source map for your bundle.

Example:

spago bundle -p my-project --source-maps --minify --outfile=bundle.js

will generate a minified bundle: bundle.js, and a source map: bundle.js.map.

Node

If your target platform is node, then you need to ensure your node version is >= 12.2.0 and enable source maps when executing your script:

spago bundle -p my-project --platform node --source-maps --minify --outfile=bundle.js
node --enable-source-maps bundle.js

Browsers

If you are targeting browsers, then you will need to ensure your server is configured to serve the source map from the same directory as your bundle.

So for example if your server is configured to serve files from public/, you might run:

spago bundle -p my-project --platform browser --source-maps --minify --outfile=public/bundle.js

Skipping the "build" step

When running spago bundle, Spago will first try to build your project, since bundling requires the project to be compiled first.

If you already compiled your project and want to skip this step you can pass the --no-build flag.

Generated build info/metadata

Spago will include some metadata in the build, such as the version of the compiler used, the version of Spago, and the versions of the package itself.

This is so that you can access all these things from your application, e.g. to power a --version command in your CLI app.

This info will be available in the Spago.Generated.BuildInfo module, which you can import in your project.

The file itself is stored in the .spago folder if you'd like to have a look at it.

Generate documentation for my project

To build documentation for your project and its dependencies (i.e. a "project-local Pursuit"), you can use the docs command:

$ spago docs

This will generate all the documentation in the ./generated-docs folder of your project. You might then want to open the index.html file in there.

If you wish for the documentation to be opened in browser when generated, you can pass an open flag:

$ spago docs --open

You can customize the output to other formats beyond html. Supported formats include ctags, etags, and markdown. For example to generate ctags for use in your editor:

$ spago docs --format ctags

Sometimes you'd like to pull up docs for dependencies even when you have compilation errors in your project. This is a good use case for the --deps-only flag:

$ spago docs --deps-only`

Alternate backends

Spago supports compiling with alternate purescript backends, such as purerl.

To use an alternate backend, include the workspace.backend section in your workspace's spago.yaml. See the configuration format section for more info.

Publish my library

To publish your library to the PureScript Registry, you can run:

$ spago publish

...and follow the instructions ๐Ÿ™‚

Know which purs commands are run under the hood

The -v flag will print out all the purs commands that spago invokes during its operations, plus a lot of diagnostic info, so you might want to use it to troubleshoot weird behaviours and/or crashes.

Install autocompletions for bash

You can just add this to your .bashrc:

source <(spago --bash-completion-script `which spago`)

or alternatively if you don't want to edit your ~/.bashrc:

spago --bash-completion-script $(which spago) >> ~/.bash_completion

Note

If you installed Spago not with NPM, but with PNPM or some other package manager, this package manager might have bundled your installation and your package name in the script may end up being incorrect. For example, when installed with PNPM, the resulting script will reference incorrect package bundle.js instead of spago. If you're using something other than NPM, verify the referenced package name in the completions script.

Install autocompletions for zsh

Autocompletions for zsh need to be somewhere in the fpath - you can see the folders included in your by running echo $fpath.

You can also make a new folder - e.g. ~/.my-completions - and add it to the fpath by just adding this to your ~/.zshrc:

fpath=(~/.my-completions $fpath)

Then you can obtain the completion definition for zsh and put it in a file called _spago (yes it needs to be called like that):

spago --zsh-completion-script $(which spago) > ~/.my-completions/_spago

Then, reload completions with:

compinit

Note

You might need to call this multiple times for it to work.

Note

See the note in the Bash section above when installing Spago with a package manager other than NPM.

Concepts and explanations

This section details some of the concepts that are useful to know when using Spago. You don't have to read through this all at once, it's meant to be a reference for when you need it.

What's a "package"?

Spago considers a "package" any folder that contains:

  • a spago.yaml file with a valid package section
  • a src subfolder with PureScript source files

That's all there is to it! You can have many of these in your repository if you'd like, and build them all together. See the monorepo section for more details.

The above holds for "workspace packages", i.e. the packages for which you have the source locally, and inside your repository. These are the packages "in your project".

Packages on which your project depends on can come from a few different sources:

  • the Registry
  • local packages - i.e. packages that are on your filesystem but external to your repository
  • remote packages - i.e. packages that are not on your filesystem, but somewhere on the internet

The bulk of the packages in your build will come from the Registry (often via a package set), but you are able to add local and remote packages to your build as well, by adding them to the workspace.extraPackages section of your spago.yaml file.

See here and here for more info about how to add these "extra packages".

Packages have "dependencies", which are other packages that are required for them to build. These dependencies are listed in the dependencies field of the package section of the spago.yaml file. See here for more info about the structure of a package configuration.

What's a "package set"?

The most generic way of defining a "package set" is "a collection of package versions that are known to build together". The point of a package set is to provide a "stable" set of packages that you can use to build your project, without worrying about version conflicts.

In practice, it looks something like this:

{
  "version": "41.2.0",
  "compiler": "0.15.10",
  "published": "2023-09-15",
  "packages": {
    "abc-parser": "2.0.1",
    "ace": "9.1.0",
    "aff": "7.1.0",
    "aff-bus": "6.0.0",
    "aff-coroutines": "9.0.0",
    "aff-promise": "4.0.0",
    "aff-retry": "2.0.0",
    "affjax": "13.0.0",
    "affjax-node": "1.0.0",
    "affjax-web": "1.0.0",
    "ansi": "7.0.0",
    "argonaut": "9.0.0",
    ...
  }
}

The Registry maintains an "official" package set such as the above, which is used by default by Spago, and only contains packages that are contained in the Registry.

Whenever anyone publishes a new package version to the Registry, the pipeline will try to build this package together with the existing set, and if the build succeeds then the new version will be added to this official set.

However, Spago also supports using custom package sets, which can contain packages that are not in the Registry, and can be used to override existing packages with local or remote versions. See here for more info.

The workspace

For any software project, it's usually possible to find a clear line between "the project" and "the dependencies of the project": we "own" our sources, while the dependencies only establish some sort of substrate over which our project lives and thrives.

Following this line of reasoning, Spago - taking inspiration from other tools such as Bazel - uses the concept of of a "workspace" to characterise the sum of all the project packages and their dependencies (including only "potential" ones).

A very succint introduction to this idea can be found in Bazel's documentation:

A workspace is a directory tree on your filesystem that contains the source files for the software you want to build.
Each workspace has a text file named WORKSPACE which may be empty, or may contain references to external dependencies required to build the outputs.
Directories containing a file called WORKSPACE are considered the root of a workspace.
Therefore, Bazel ignores any directory trees in a workspace rooted at a subdirectory containing a WORKSPACE file, as they form another workspace.

Spago goes by these same rules, with the difference that we do not use a separate WORKSPACE file, but instead use the workspace section of the spago.yaml file to define what the set of our external dependencies are, and where they come from.

This can be as simple as:

workspace: {}

...which means that "this is now a workspace, and all the dependencies are going to be fetched from the Registry".

Or it can be more complex, e.g.:

workspace:
  packageSet:
    url: https://raw.githubusercontent.com/some-user/custom-package-sets/some-release/packages.json
  extraPackages:
    aff:
      path: ../my-purescript-aff

...which means that "this is now a workspace, and all the dependencies are going to be fetched using instructions from this custom package set (which could point to the Registry packages or somewhere else), except for the aff package, which is going to be fetched from the local folder ../my-purescript-aff".

As described in the Bazel docs quoted above, the presence of a workspace section will denote the current folder as the root of a workspace, and Spago will recurse into its subfolders to find all the packages that are part of it - by looking for spago.yaml files with a package section - but ignore the subdirectory trees that are themselves workspaces - i.e. containing spago.yaml files with a workspace section.

The configuration file

This section documents all the possible fields that can be present in the spago.yaml file, and their meaning.

# The workspace section is one of the two sections that can be present
# at the top level. As described above, it defines where all of the
# dependencies of the project come from.
# It's optional, as it will be found only in the root of the project
# (defining the workspace), and not in any sub-package configurations,
# which will only contain the `package` section.
workspace:

  # The packageSet field defines where to fetch the package set from.
  # It's optional - not defining this field will make Spago use the
  # Registry solver instead, to come up with a build plan.
  packageSet:
    # It could either be a pointer to the official registry sets that
    # live at https://github.com/purescript/registry/tree/main/package-sets
    registry: 11.10.0
    # Or it could just point to a URL of a custom package set
    # See the "Custom package sets" section for more info on making one.
    url: https://raw.githubusercontent.com/purescript/package-sets/psc-0.15.7-20230207/packages.json
    # It is also possible to point to a local package set instead:
    path: ./my-custom-package-set.json

  # This section defines any other packages that you'd like to include
  # in the build. It's optional, in case you just want to use the ones
  # coming from the Registry/package set.
  extraPackages:
    # Packages are always specified as a mapping from "package name" to
    # "where to find them", and there are quite a few ways to define
    # these locations:
    # 1) local package - it's on your filesystem but not in the workspace
    some-local-package:
      path: ../some-local-package
    # 2) remote package from the Registry
    #    Just specify the version you want to use. You can run
    #    `spago registry info` on a package to see its versions.
    #    This is useful for packages that are not in the package set,
    #    and useless when using the Registry solver.
    some-registry-package: 1.0.2
    # 3) remote package from Git
    #    The `git` and `ref` fields are required (`ref` can be a branch,
    #    a tag, or a commit hash).
    #    The `subdir` field is optional and necessary if you'd like to use
    #    a package that is not located in the root of the repo.
    #    The `dependencies` field is optional and necessary if the package
    #    does not have a `spago.yaml` file. In that case Spago will figure
    #    out the dependencies automatically.
    some-package-from-git:
        git: https://github.com/purescript/registry-dev.git
        ref: 68dddd9351f256980454bc2c1d0aea20e4d53fa9
        subdir: lib
        dependencies:
          - foo
    # 4) remote package from Git, legacy style (as the old package sets)
    #    Works like the above, but all fields are mandatory.
    legacy-package-style:
        repo: "https://github.com/purescript/purescript-prelude.git"
        version: "v6.0.1"
        dependencies:
          - prelude
          - effect
          - console

  # This section is optional, and you should specify it only if you'd like
  # to build with a custom backend, such as `purerl`.
  # Please see the "Alternate backends" section for more info.
  backend:
    # The command to run to build with this backend - required.
    cmd: "node"
    # Optional list of arguments to pass to the backend when building.
    args:
      - "arg1"
      - "arg2"
      - "arg3"

  # Optional setting to enable the "lockfile". Enabling it will generate
  # a `spago.lock` file with a cache of the build plan.
  # It's disabled by default when using package sets (because we already
  # get a stable build plan from there) and enabled by default when using
  # the Registry solver.
  # See "The lock file" section for more details.
  lock: false

  # Optional section to further customise the build.
  buildOpts:
    # Directory for the compiler products - optional, defaults to `output`.
    output: "output"
    # Specify whether to censor warnings coming from the compiler
    # for files in the `.spago` directory`.
    # Optional and can be one of two possible values
    censorLibraryWarnings:
      # Value 1: "all" - All warnings are censored
      all

      # Value 2: `NonEmptyArray (Either String { byPrefix :: String })`
      # - String values:
      #      censor warnings if the code matches this code
      # - { byPrefix } values:
      #      censor warnings if the warning's message
      #      starts with the given text
      - CodeName
      # Note: when using `byPrefix`, use the `>` for block-string:
      # see https://yaml-multiline.info/
      - byPrefix: >
        "Data.Map"'s `Semigroup instance`

    # Specify whether to show statistics at the end of the compilation,
    # and how verbose they should be.
    # Can be 'no-stats', 'compact-stats' (default), or 'verbose-stats',
    # which breaks down the statistics by warning code.
    statVerbosity: "compact-stats"

# This is the only other section that can be present at the top level.
# It specifies the configuration for a package in the current folder,
# and it's optional, as one could just have a `workspace` section.
package:

  # The package name and the `dependencies` fields are the only required
  # ones, everything else is optional.
  name: my-package-name
  dependencies:
    # Dependencies can be specified in a few ways:
    # 1) just with a package name
    #    Then Spago will use the version specified in the package set,
    #    or assume the widest possible range if using the Registry solver.
    - some-package
    # 2) explicitly specify the widest range
    #    Same as above, but explicit.
    - package-with-star-range: *
    # 3) specify a defined version range
    #    The registry will then check if the package version is included
    #    in this range.
    - package-with-range: ">=1.1.1 <2.0.0"

  # Optional description for the package
  description: "a useful package"

  # Optional section to further customise the build for this package.
  build:
    # Fail the build if this package's `dependencies` field has redundant/underspecified packages.
    # Optional boolean that defaults to `false`.
    pedanticPackages: false

    # Specify whether to censor warnings coming from the compiler
    # for files from this package.
    # Optional and can be one of two possible values
    censorProjectWarnings:
      # Value 1: "all" - All warnings are censored
      all

      # Value 2: `NonEmptyArray (Either String { byPrefix :: String })`
      # - String values:
      #      censor warnings if the code matches this code
      # - { byPrefix } values:
      #      censor warnings if the warning's message
      #      starts with the given text
      - CodeName
      # Note: when using `byPrefix`, use the `>` for block-string:
      # see https://yaml-multiline.info/
      - byPrefix: >
        "Data.Map"'s `Semigroup instance`
    # Convert compiler warnings for files in this package's src code
    # into errors that can fail the build.
    # Optional and defaults to false
    strict:
      true

  # Optional section to specify the configuration options for bundling
  # The following options are all optional, and will default to the values
  # shown below.
  bundle:
    minify: false
    # Entrypoint for the bundle
    module: Main
    # The path of the bundle file
    outfile: "index.js"
    # Possible values are 'node' or 'browser'
    platform: browser
    # Possible values are 'app' or 'module'
    type: "app"
    # Any other flags that should be passed to the bundler.
    # You can use this to e.g. pass `--external` flags to esbuild:
    extra_args:
      - "--external:ssh2"

  # Optional section to configure the behaviour of `spago run`.
  # All the fields are optional.
  run:
    # The entrypoint for the program
    main: Main
    # List of arguments to pass to the program
    execArgs:
      - "--cli-arg"
      - "foo"

  # Optional section to configure `spago test`
  # The `main` and `dependencies` fields are required.
  test:
    main: Test.Main
    # This works like `package.dependencies`
    dependencies:
    - foo
    # Optional list of arguments to pass to the test program
    execArgs:
      - "--cli-arg"
      - "foo"

    # Fail the build if this package's test's `dependencies` field has redundant/underspecified packages.
    # Optional boolean that defaults to `false`.
    pedanticPackages: false

    # Specify whether to censor warnings coming from the compiler
    # for files from this package's test code.
    # Optional and can be one of two possible values
    censorTestWarnings:
      # Value 1: "all" - All warnings are censored
      all

      # Value 2: `NonEmptyArray (Either String { byPrefix :: String })`
      # - String values:
      #      censor warnings if the code matches this code
      # - { byPrefix } values:
      #      censor warnings if the warning's message
      #      starts with the given text
      - CodeName
      # Note: when using `byPrefix`, use the `>` for block-string:
      # see https://yaml-multiline.info/
      - byPrefix: >
        "Data.Map"'s `Semigroup instance`
    # Convert compiler warnings for files from this package's test code
    # into errors that can fail the build.
    # Optional and defaults to false
    strict:
      true

  # Optional section for configuring the `spago publish` command.
  # If you intend to publish your package, this section becomes mandatory.
  publish:
    # The version of your package. This follows semver rules, but with no
    # prereleases - so only major.minor.patch.
    version: 1.0.0
    # The license for your source, in SPDX format: https://spdx.dev/
    license: BSD-3-Clause
    # Optional list of globs to include in the published archive, in
    # addition to the list of files that the Registry includes by default:
    # https://github.com/purescript/registry-dev/blob/master/SPEC.md#always-included-files
    include:
    - "test/**/*.purs"
    # Optional list of globs to exclude from the published archive, in
    # addition to the list of files that the Registry includes by default:
    # https://github.com/purescript/registry-dev/blob/master/SPEC.md#always-excluded-files
    # Note that the Registry will first apply the `include` list, then
    # the `exclude` one, as detailed in the specification:
    # https://github.com/purescript/registry-dev/blob/master/SPEC.md#33-manifest
    exclude:
    - "test/graphs/**/*"
    # The place where the Registry will fetch the source from.
    # This is optional since you might want to release the code without
    # publishing to the Registry, which is what this is needed for.
    location:
      # There are a few supported locations:
      # 1) Github: no URL needed, just username and the name of the repo
      #    The `subdir` field is optional, and only necessary if your
      #    package is not in the root of the repo.
      githubOwner: owners-username
      githubRepo: repo-name
      subdir: lib
      # 2) Git: any git server should work with this
      #    The `subdir` is optional as above
      url: git://someurl...
      subdir: lib

The lock file

The lock file is a file that Spago can generate to cache the build plan, so that it can be reused in subsequent builds.

When using package sets it is disabled by default - since we already get a stable build plan from there - while it's enabled by default when using the Registry solver.

You can enable it manually by adding a lock: true field to the workspace section of your spago.yaml file, and that will keep it on regardless of which solving mode you're using.

File System Paths used in Spago

Run spago ls paths to see all paths used by Spago. But in general, Spago utilizes two main directories for every project:

  • the local cache directory
  • the global cache directory

The local cache directory is located at <project-directory>/.spago and its location cannot be changed.

The global cache directory's location depends on your OS. Its location can be changed by configuring the corresponding environment variable, if it is used:

  • Mac: ~/Library/Caches/spago-nodejs. The location cannot be changed.
  • Linux: ${XDG_CACHE_HOME}/spago-nodejs, or if XDG_CACHE_HOME is not set, ~/.cache/spago-nodejs. See XDG_CACHE_HOME.
  • Windows: %LOCALAPPDATA%\spago-nodejs\Cache, or if $LOCALAPPDATA% is not set, C:\Users\USERNAME\AppData\Local\spago-nodejs\Cache.
  • NixOS: ${XDG_RUNTIME_DIR}/spago-nodejs. See XDG_RUNTIME_DIR.

FAQ

Why can't spago also install my npm dependencies?

A common scenario is that you'd like to use things like react-basic, or want to depend on JS libraries like ThreeJS. In any case, you end up depending on some NPM package.

And it would be really nice if spago would take care of installing all of these dependencies, so we don't have to worry about running npm besides it, right?

While these scenarios are common, they are also really hard to support. In fact, it might be that a certain NPM package in your transitive dependencies would only support the browser, or only node. Should spago warn about that? And if yes, where should we get all of this info?

Another big problem is that the JS backend is not the only backend around. For example, PureScript has an fairly active Erlang backend among the others.

These backends are going to use different package managers for their native dependencies, and while it's feasible for spago to support the backends themselves, also supporting all the possible native package managers (and doing things like building package-sets for their dependencies' versions) is not a scalable approach (though we might do this in the future if there's enough demand).

So this is the reason why if you or one of your dependencies need to depend on some "native" packages, you should run the appropriate package-manager for that (e.g. npm).

Differences from legacy spago

Watch mode

Spago dropped support for the --watch flag in spago build and spago test.

VSCode users are recommended to use the Purescript IDE extension for seamless experiences with automatic rebuilds.

Users of other editors, e.g. vim, emacs, etc., can make use of the underlying LSP plugin.

If you want a very simple drop in replacement for spago test --watch, you can use a general purpose tool such as watchexec:

watchexec -e purs,js,yaml -- spago test

sources in the configuration file

The sources field in the configuration file does not exist anymore.

Instead, Spago will look for a src folder in the package root, and will use that as the source folder, and similarly for the test folder, using that for the test sources.

spago's People

Contributors

benjmhart avatar blugatroff avatar btrepp avatar cdepillabout avatar charlestaylor7 avatar cofad avatar dariooddenino avatar dependabot[bot] avatar dretch avatar eahlberg avatar elliotdavies avatar f-f avatar flip111 avatar imsaravana369 avatar jhrcek avatar jordanmartinez avatar justinwoo avatar klntsky avatar milesfrain avatar monoidmusician avatar nwolverson avatar pacchettibotti avatar peterbecich avatar postsolar avatar sigma-andex avatar stkb avatar thomashoneyman avatar toastal avatar vapaj avatar waddlaw avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

spago's Issues

Module not found when running `spago test`

I'm getting Module Chat.Data.User was not found when trying to run tests with spago test command.

Here is my test directory structure:

.
โ”œโ”€โ”€ Main.purs
โ””โ”€โ”€ Spec
    โ”œโ”€โ”€ Capability
    โ”œโ”€โ”€ Capability.purs
    โ”œโ”€โ”€ Data
    โ”‚ย ย  โ””โ”€โ”€ Message.purs
    โ”œโ”€โ”€ Data.purs
    โ””โ”€โ”€ Utils.purs

Actually, my playground project is here https://github.com/vyorkin-play/chat-api/ (I'm following the purescript-halogen-realworld, trying to build a simple halogen app).

Previously I used psc-package and the following "scripts":

    "test": "run-s compile:test test:run",
    "test:run": "node -e 'require(\"./output/Test.Main\").main()'",
    "compile": "psc-package build",
    "compile:test": "psc-package build -- 'test/**/*.purs'",

Local dependencies in the package-set

Coming from using bower I'm missing bower link, spago is making it easier to consume others libraries but using bower it's damn handy to be able to link against local versions of packages when developing libraries/verifying changes etc.

Two options I can think of thus far.

  1. Dropping a symlink to .spago/package/local/
  2. Playing with packages .dhall to link in the other package.

The first is simple, but if your package has other deps it could be a fun little dependency hell.

For the latter, dhall is powerful enough that you can do things like

let tiled = ../tiled/spago.dhall

let mkPackage =
      https://raw.githubusercontent.com/spacchetti/spacchetti/20190105/src/mkPackage.dhall sha256:90974a8e07650e49a4197d4ce5b59e82740fd8469c8c1b9793939845ca8faad9

let upstream =
      https://raw.githubusercontent.com/spacchetti/spacchetti/20190105/src/packages.dhall sha256:04da9c924e05cb6b43447ffd13408d01992f3172eb80d5d94e7f1b14c76a5123

let overrides = {=}

let additions = 
    { tiled  = 
        mkPackage 
          tiled.dependencies
          "..\\..\\..\\..\\tiled"
          "master"
    }

let tiledset = tiled.packages    

in  upstream โซฝ overrides โซฝ additions // tiledset

This works, with the caveat that if you update the repo's master, you need to delete .spago\tiled\master\ to get spago install to copy everything again.

I can also appreciate that it's better to guide everyone to contribute more to the package sets, but this is more of a situation where having my own forked spachetti set/ overrides seems a tad overkill, especially as I'm still experimenting with what the 'package' should look like, so there's little purposing in publishing any of it.. yet.

Sources

Need a sources command that will get the source globs

Question on additions

Hi and thank you for this great repo. I'm thinking about additions. Let's say I try to install the purescript-graphql library. This is not in the default package-set so I add it

{
    graphql =
    mkPackage
     [ "aff"
     , "argonaut-core"
     , "httpure"
     , "argonaut-codecs"
     ]
     "https://github.com/hendrikniemann/purescript-graphql"
     "master"
}

I then run spago install and it installs graphql correctly. Now when running spago build I get errors on the dependencies graphql needs (Node.Stream for example). So I add purescript-node-streams etc. Do I do this for every dependency the addition relies on?

It makes sense for me that I would do this seeing that spago cannot know how to install dependencies from other additional dependencies. Just wanted to be 100% I'm not missing anything.

Build Windows binaries in CI

Followup from #47 (comment): we should build binaries for Windows in CI (with AppVeyor or Azure Pipelines), and upload them to GitHub releases.

(Or if we want to have it even more awesome: add a way to install them with npm)

Usage with psc-ide-emacs

To use spago with psc-ide-emacs, will either need to specify some user globs yourself or make a PR to psc-ide-emacs: https://github.com/epost/psc-ide-emacs

In particular, see this block for how source globs are prepared and inserted:

https://github.com/epost/psc-ide-emacs/blob/01a158b77210fec9c1bfc0caffaf08fccc0412ac/psc-ide.el#L263-L274

(defun psc-ide--server-start-globs ()
  "Detects bower and psc-package projects and determines sensible
  source globs"

  (when (and (file-exists-p "psc-package.json") (file-exists-p "bower.json"))
    (message "Detected both a \"psc-package.json\" and a \"bower.json\" file."))

  (let ((server-globs psc-ide-source-globs))

    ;; see this section specifically, which handles psc-package globs
    (if (file-exists-p "psc-package.json")                        ; first, we check for psc-package.json
        (let* ((results "*PSC-PACKAGE SOURCES*")                  ; we let stdout go to a buffer named *PSC-PACKAGE SOURCES*
               (errors "*PSC-PACKAGE ERRORS*"))                   ; stderr goes ot the errors buffer

          (shell-command "psc-package sources" results errors)    ; run the psc-package sources command

          (if (get-buffer errors)                                 ; if there's an error, display it
              (switch-to-buffer-other-window errors)
            (progn                                                ; otherwise, we can take the results and kill the results buffer
              (with-current-buffer results
                (let* ((globs (split-string (buffer-string))))
                  (setq server-globs (append server-globs globs)) ; then we set the server globs with our globs being added to it
                  (kill-buffer results)))
              (message "Set source globs from psc-package. Starting server..."))))
              ;; end of psc-package glob handling


      (if (file-exists-p "bower.json")
          (setq server-globs (append server-globs '("bower_components/purescript-*/src/**/*.purs")))
        (message "Couldn't find psc-package.json nor bower.json files, using just the user specified globs.")))
server-globs))

`devDependencies` option

What do you think about adding devDependencies option (to both Package and Config)? The motivation here is that most test frameworks are pretty big, with lots of transitive dependencies and different libs can use different test frameworks, wich adds even more packages that I will have to download, but most of the time will not use. Downloading only those packages that are required for library to work should make installation much faster.

`spago publish` command

We should have a spago publish command, that publishes the docs for the current package to Pursuit (as pulp publish does)

Automate updating the spacchetti version

We should update automatically the spacchetti version and sha in packages.dhall every time spacchetti releases. This should be done in the release step of spacchetti Travis.

Steps for automating this:

  • sed the new release name in the urls in packages.dhall, more sed to remove the sha265:..
  • run dhall freeze --inplace templates/packages.dhall

Add a way to specify an output folder

With psc-package you can do something like this:

psc-package build -- -o path/to/folder

and with pulp you can use --build-path or -o flags. It would be nice if spago had something like this.

`spago upstream` command

It is usual to add packages that are not in the package-set to the packages.dhall (I mean, this is basically our main feature). It would be nice to have an easy way to have them contributed to the upstream package-set, e.g. with something like spago upstream whatever-package.

What this command would do is:

  • check that the package is not in the upstream package-set (if it is, prompt the user to upgrade)
  • if it's not, open a pull request to spacchetti (e.g. with this) (note: we'll need to think of a way to handle auth. I guess passing a github token with a flag could be ok?)

Add ability to build command to watch for file changes

Hi there, and thanks for Spago!

As I was creating a fork of a react-basic-starter to use Spago, I wanted to recommend a way to have the project be auto-built when making changes.
Right now there is in a workaround for people using IDE PureScript, but it would be nice to have an alternative for people who don't use the plugin.

I realize this is a little bit more complicated since Spago uses purs under the hood, and unlike pulp it doesn't support the watch flag.

As an aside, the project seems to work like a treat combined with Parcel ๐ŸŽ‰

Add searchbox to generated documentation

We should be able to generate documentation for all the libraries in the package set (and/or just the libraries used in the project), in a similar way to what stack haddock does for Haskell projects.

The use-cases for this are multiple and useful:

  • get specialized documentation for your project without having to rely on Pursuit (because you might have dependencies not from the package set and/or on Pursuit)
  • more generally, allow an easy way to have an offline Pursuit
  • generate a "static Pursuit" for every package set in Spacchetti and host that alongside the package set, which is effectively the Stackage approach.
    Note that this would obsolete #27, as it would entirely remove the "where do I upload my library" problem โ†’ you would just add it to a Spacchetti package-set, and have it "published" in the next Spacchetti release

How to achieve this:

  • it is possible to generate html docs for the whole package-set by running a purs docs command.
    Justin had a good start with acme-spago. He included a spago project in there, but if this is integrated in spago itself, we wouldn't need it (as we would just generate that info on the fly)
  • then we'd need a way to search in the existing docs. Since the whole thing needs to be static files (i.e. no backend), we should have a frontend-side search). A simple way to do this would be to generate markdown docs and then index them with something like Lunr (@justinwoo suggested scraping the externs JSON files, but I'm not sure what that would entail, so probably worth detailing more)

`spago verify` command

Need a verify command that will attempt to install and build a given dependency. This can be done in almost any way, but a full set verify should install one package while verifying at a time.

libtinfo.so.5: cannot open shared object file

After the latest software update on my Pop!_OS (which is basically Ubuntu), spago started to throw an error:

spago: error while loading shared libraries: libtinfo.so.5: cannot open shared object file: No such file or directory

After some investigation I found out that I have libtinfo.so.6 not libtinfo.so.5 and the way to fix this problem is to create this symlink:

sudo ln -s /path/to/libtinfo.so.6 /path/to/libtinfo.so.5

I understand that it's not exactly a spago issue, but I still want to bring this up

  • to save people that are facing this issue some googling time
  • to find out if this fix is a correct way solve this problem

Dhall error doing `local-setup --force`

We discussed on twitter this might be related to old dhall, but I just installed it straight from git and that didn't fix it.

$ spacchetti local-setup --force
spacchetti: 
Error: Invalid input

(stdin):114:1:
    |
114 | let upstream =
    | ^^
unexpected "le"
expecting "!=", "&&", "++", "->", "//", "//\\", "/\", "==", "as", "in", "||", "โ†’", "โˆง", "โฉ“", "โซฝ", '#', '*', '+', ':', '?', or whitespace

RFC: depend on the PureScript compiler

Following up from this Discourse thread, a big problem seems to be that when we have breaking changes in the compiler there are "ecosystem splits", and picking the right packages is hard (especially with bower) if the people just does npm install purescript (from personal experience, the 0.12 transition was indeed quite bad).

Spago is currently not immune to this problem: we're wrapping whatever compiler the user has installed, so if we have a post-0.12 package set, it's not gonna work if the user has purs-0.11 installed. In general, the "compiler version the package-set is compatible with" is not checked against "compiler version currently installed", and this is not right! (as it's easy to diverge)

As usual, the "Stack solution" for this is really good:

  • every LTS has a compiler version associated with it
  • stack installs a sandboxed version of the compiler for every project

The downside of this is that the tooling (read: editor integrations) will have to talk to spago instead of the compiler for things like purs ide.

In our case we would not download a version of the compiler, but we could:

  • we'd depend on the compiler library, and run things through it instead of calling an external executable
  • this means that a certain Spago version would have an associated Purs version (i.e. not dependent on the package-set)
  • editor plugins should call stuff like spago ide instead of purs ide

`spago clean` command

It would be nice to have a command that could remove all artifacts, produced by other commands (like build, repl and so on), except for those, that were produced by init.
Sometimes when something goes wrong it's easier to just clean and rebuild :)

Packages listed but not installed

Hi. I'm adding purescript-express as an addition in packages.dhall like so

let additions = 
  { express  = 
      mkPackage 
      [ "foreign"
      , "foreign-generic"
      , "node-http"
      , "test-unit"
      , "aff"
      ]
      "https://github.com/nkly/purescript-express"
      "master"
  }

Then I do spago install. Now when doing

spago list-packages | grep express

I get

express (master, https://github.com/nkly/purescript-express)

But when I do spago build I get

Error 1 of 4:

  in module Server
  at src/Server.purs line 6, column 1 - line 6, column 61

    Module Node.Express.App was not found.
    Make sure the source file exists, and that it has been provided as an input to the compiler.

also when verifying the location of the package: spago sources | grep express
i get nothing.

does this have something to do with the verification of a package? and if it fails it will do so silently?

TODO: update "docs"

There are various places where spacchetti-cli and such are referenced, so they should at least be updated to reference this project after releases.

TODO when breaking compatibility

The goal of this issue is to list all the things that we might want to do the next time we are going to break compatibility (probably when 1.0 will happen). The reason for this is that bundling breaking changes is always a good thing to do, since people will have to make changes anyways, and it's best to just do it once than to repeatedly have to do it.

Up to date list of things we might want to do:

  • rename packages.dhall to package-set.dhall: the name packages is confusing because we have to explain it contains the package-set update: the ship for this has probably sailed, as now spacchetti has merged into package-sets, so everything is just called packages, so let's keep it like that for consistency
  • stop publishing under purescript-spago on NPM, and only publish under spago: the fact that we publish both might be confusing
  • start failing when package-sets don't include the minimum compiler version update: this cannot be done by design, as package sets might come from a hash as well. All we can do then is to log a warning "spago assumes that you know what you're doing since we cannot parse the version from the package-set tag"

Rudimentary windows support

The issue you were already expecting :). I'll open this and document trying to get it running on windows. Maintainers: don't feel compelled to respond yet, I'll try to do as much as I can myself.

Running in cmd.exe, can't even get to running the install commands. Looks like the install script assumes bash, curl/wget and tar to be present.

UPDATE: Looks like npm install would never work, that script downloads binaries that have been prebuilt. Next steps for me getting this to work would be getting stack+haskell and trying to build it myself.

UPDATE2: stack install fails due to win32/turtle version mismatchs. Stack is implying to add Win32-2.5.4.1@sha256:e623a1058bd8134ec14d62759f76cac52eee3576711cb2c4981f398f1ec44b85 to extra deps. Will see what we get from that. (looks like Gabriella439/turtle#323 is the upstream issue)

UPDATE3: This built with the above change. Though stack install did have to be ran multiple times, but I do have a spago.exe :)

Trying yarn

yarn global add purescript-spago
yarn global v1.9.4
[1/4] Resolving packages...
[2/4] Fetching packages...
[------------------------------------------------------------------------------------------------------------------------------------------------] 0/501(node:11552) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead.
info [email protected]: The platform "win32" is incompatible with this module.
info "[email protected]" is an optional dependency and failed compatibility check. Excluding it from installation.
[3/4] Linking dependencies...
[4/4] Building fresh packages...
error C:\Users\beau.trepp\AppData\Local\Yarn\Data\global\node_modules\purescript-spago: Command failed.
Exit code: 1
Command: .\install
Arguments:
Directory: C:\Users\beau.trepp\AppData\Local\Yarn\Data\global\node_modules\purescript-spago
Output:
'.\install' is not recognized as an internal or external command,
operable program or batch file.
info Visit https://yarnpkg.com/en/docs/cli/global for documentation about this command.

Trying npm

npm install -g purescript-spago
C:\Users\beau.trepp\AppData\Roaming\npm\spago -> C:\Users\beau.trepp\AppData\Roaming\npm\node_modules\purescript-spago\spago

[email protected] install C:\Users\beau.trepp\AppData\Roaming\npm\node_modules\purescript-spago
./install

'.' is not recognized as an internal or external command,
operable program or batch file.
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] install: ./install
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR! C:\Users\beau.trepp\AppData\Roaming\npm-cache_logs\2018-12-18T03_17_01_744Z-debug.log

Trying stack install

Error: While constructing the build plan, the following exceptions were encountered:

In the dependencies for turtle-1.5.12:
Win32-2.6.1.0 from stack configuration does not match >=2.2.0.1 && <2.6 (latest matching
version is 2.5.4.1)
needed due to spago-0.6.0.0 -> turtle-1.5.12

Some different approaches to resolving this:

  • Set 'allow-newer: true' in C:\sr\config.yaml to ignore all version constraints and build anyway.

  • Consider trying 'stack solver', which uses the cabal-install solver to attempt to find some
    working build configuration. This can be convenient when dealing with many complicated
    constraint errors, but results may be unpredictable.

  • Recommended action: try adding the following to your extra-deps
    in C:\VisualStudio_Projects\purescript\spago\stack.yaml:

Win32-2.5.4.1@sha256:e623a1058bd8134ec14d62759f76cac52eee3576711cb2c4981f398f1ec44b85

Plan construction failed.

`test` command doesn't work correctly

When I run test command, I end up with a bunch of Module Foo.Bar was not found errors. It just doesn't see modules in src folder. So I guess the answer to this question is "yes" :)

Allow integration with atom-ide-purescript

I use Atom to write Purescript source code.

When I converted my project from Pulp + Bower to Spago the nice features of Atom (show-type-on-hover, go-to-definition, auto-import, etc) stopped working.

It would be nice to get this working again. I am not sure entirely what is involved, or whether changes are required to https://github.com/nwolverson/atom-ide-purescript itself. One thing that does appear to be required is for spago build to pass through arguments to purs, like pulp build -- --json-errors (once this is done then atom-ide can at least be configured to build with Spago).

New name proposal

As we discussed some time ago with @justinwoo, the name of this repo/program has a bit of a name clash with the spacchetti package set repo, and this is super confusing.

So I'd like to propose a new name for this repo and tool: spago
(IPA: /หˆspaษกo/)

Why?

  • It means "twine" in Italian, and it would symbolize the new role of this tool (well at least after #21), to "tie everything together"
  • It is different than spacchetti
  • It kind of reminds of it at the same time (they have the same root)
  • It is actually shorter to type at the command line, but unique enough

Sample image to visualize it:
spago

Spago testing fails on windows

Running the python tests seems to fail on windows.

I'll spend some time later investigating it further.

python ..\spago-test.py
Expecting success from: "spago init"
Expecting failure from: "spago init"
Expecting success from: "rm spago.dhall packages.dhall"
Expecting success from: "spago init"
Expecting success from: "rm src/Main.purs"
Expecting success from: "spago init -f"
FAILURE: Spago should always succeed in doing init with force
Program output:
spago: .\packages.dhall: openFile: permission denied (Permission denied)

`spago spacchetti-upgrade` command

It is common that every once in a while you might want to sync to the upstream package-set.

We could have a spago spacchetti-upgrade command, that would:

  • check that we are using a package set from a spacchetti release
  • check if we're on the latest release
  • if we are not:
    • get the latest release
    • fix the import URL
    • and recompute the Dhall hash for the new import

Related to #39

Init/spacchetti.dhall

Minimum needs two things:

  1. Import packages.dhall from the local directory
  2. Define a list of primary dependencies

Add tests

There should be some tests so that we have some kind of assurance that we don't break stuff.

Anything is good to start, even just a bash script that installs the binary, runs some commands and checks that the return codes are all right.
This should prevent simple breakage, like what happened when 4e9c20e lead to #29

`spago build --path` option to specify source paths

In cases of more complex setups, it is useful to specify the source paths from which we build.
You can do this with psc-package build "your/source/path/here" "another/source/path", but this is a clumsy interface.
I propose we should have this looking like spago build --path "some/source" --path "some/other-source"

Hashes in existing projects break with new Dhall version

I setup a project with the currently released Spago version:

$ spago version
0.6.3.0

$ spago init
Set up a local Spago project.
Try running `spago install`

Then I upgrade Spago to the latest master, and if I try to install, it fails:

$ spago version
0.6.4.0

$ spago install
spago:
โ†ณ ./packages.dhall
  โ†ณ https://raw.githubusercontent.com/spacchetti/spacchetti/20181209/src/mkPackage.dhall sha256:8e1c6636f8a089f972b21cde0cef4b33fa36a2e503ad4c77928aabf92d2d4ec9
This decoded version is not supported: 4.0.0

What happened here?
In this commit we upgraded to the latest Dhall version (1.19 โ†’ 1.20), which also upgraded the Dhall language version (4.0.0 โ†’ 5.0.0), rendering the hashes in the project file non-compatible with the new version, which cannot decode them.

There is already an issue open about a related problem: dhall-lang/dhall-lang#335

Once the fix will be implemented in Dhall it will be easy to "upgrade the clients" if the hash of the expression is the same (but the version unsupported), I'll investigate if in the meanwhile we can patch it nicely and "autoupgrade" the projects

NPM release

This is currently done here, but we should now do it from this repo

override command

We should be able to be able to override packages from the cli.

If having it in a sane way is too complex, we should at least allow to override the package version (which I think already covers 99% of the cases)

Check that purs version is at least what the package-set specifies

As detailed in #98 (comment), we should run purs --version every time we want to install packages, to check that we're running at least the compiler version specified by the package-set (see spacchetti/spacchetti#63)

If the version isn't there we should skip the check (see #94 for when we could fail instead), if the version is there we should fail if it's not SemVer-compatible (e.g. 0.11.x is not compatible with 0.12.x, but 0.12.x is compatible with 0.12.y, if x > y and the compiler has x), so that the user can install the right version of the compiler.

Module Prim.Boolean was not found

I'm not sure exactly what brought in typelevel-prelude (I was trying to use Halogen), but I am getting this error.

in module Type.Data.Boolean
at .spago/typelevel-prelude/v4.0.0/src/Type/Data/Boolean.purs line 17, column 1 - line 17, column 48

  Module Prim.Boolean was not found.
  Make sure the source file exists, and that it has been provided as an input to the compiler.

I tried from a fresh project, spago init, spago upgrade-spacchetti, spago install typelevel-prelude, spago test and still go the error. I tried the latest release of spago and building from source.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.