purescript / spago Goto Github PK

I use Atom to write Purescript source code.

When I converted my project from Pulp + Bower to Spago the nice features of Atom (show-type-on-hover, go-to-definition, auto-import, etc) stopped working.

It would be nice to get this working again. I am not sure entirely what is involved, or whether changes are required to https://github.com/nwolverson/atom-ide-purescript itself. One thing that does appear to be required is for spago build to pass through arguments to purs, like pulp build -- --json-errors (once this is done then atom-ide can at least be configured to build with Spago).

It is common that every once in a while you might want to sync to the upstream package-set.

We could have a spago spacchetti-upgrade command, that would:

• check that we are using a package set from a spacchetti release
• check if we're on the latest release
• if we are not:
  • get the latest release
  • fix the import URL
  • and recompute the Dhall hash for the new import

Related to #39

Needs API_KEY in Travis also

I'm not sure exactly what brought in typelevel-prelude (I was trying to use Halogen), but I am getting this error.

in module Type.Data.Boolean
at .spago/typelevel-prelude/v4.0.0/src/Type/Data/Boolean.purs line 17, column 1 - line 17, column 48

  Module Prim.Boolean was not found.
  Make sure the source file exists, and that it has been provided as an input to the compiler.

I tried from a fresh project, spago init, spago upgrade-spacchetti, spago install typelevel-prelude, spago test and still go the error. I tried the latest release of spago and building from source.

We should be able to be able to override packages from the cli.

If having it in a sane way is too complex, we should at least allow to override the package version (which I think already covers 99% of the cases)

The goal of this issue is to list all the things that we might want to do the next time we are going to break compatibility (probably when 1.0 will happen). The reason for this is that bundling breaking changes is always a good thing to do, since people will have to make changes anyways, and it's best to just do it once than to repeatedly have to do it.

Up to date list of things we might want to do:

• rename packages.dhall to package-set.dhall: the name packages is confusing because we have to explain it contains the package-set update: the ship for this has probably sailed, as now spacchetti has merged into package-sets, so everything is just called packages, so let's keep it like that for consistency
• stop publishing under purescript-spago on NPM, and only publish under spago: the fact that we publish both might be confusing
• start failing when package-sets don't include the minimum compiler version update: this cannot be done by design, as package sets might come from a hash as well. All we can do then is to log a warning "spago assumes that you know what you're doing since we cannot parse the version from the package-set tag"

As detailed in #98 (comment), we should run purs --version every time we want to install packages, to check that we're running at least the compiler version specified by the package-set (see spacchetti/spacchetti#63)

If the version isn't there we should skip the check (see #94 for when we could fail instead), if the version is there we should fail if it's not SemVer-compatible (e.g. 0.11.x is not compatible with 0.12.x, but 0.12.x is compatible with 0.12.y, if x > y and the compiler has x), so that the user can install the right version of the compiler.

This guy doesn't even use this name anymore: https://www.npmjs.com/package/spago (https://github.com/FormBucket/xander#readme)

We could ask him to give the name

Minimum needs two things:

1. Import packages.dhall from the local directory
2. Define a list of primary dependencies

I setup a project with the currently released Spago version:

$ spago version
0.6.3.0

$ spago init
Set up a local Spago project.
Try running `spago install`

Then I upgrade Spago to the latest master, and if I try to install, it fails:

$ spago version
0.6.4.0

$ spago install
spago:
↳ ./packages.dhall
  ↳ https://raw.githubusercontent.com/spacchetti/spacchetti/20181209/src/mkPackage.dhall sha256:8e1c6636f8a089f972b21cde0cef4b33fa36a2e503ad4c77928aabf92d2d4ec9
This decoded version is not supported: 4.0.0

What happened here?
In this commit we upgraded to the latest Dhall version (1.19 → 1.20), which also upgraded the Dhall language version (4.0.0 → 5.0.0), rendering the hashes in the project file non-compatible with the new version, which cannot decode them.

There is already an issue open about a related problem: dhall-lang/dhall-lang#335

Once the fix will be implemented in Dhall it will be easy to "upgrade the clients" if the hash of the expression is the same (but the version unsupported), I'll investigate if in the meanwhile we can patch it nicely and "autoupgrade" the projects

Coming from using bower I'm missing bower link, spago is making it easier to consume others libraries but using bower it's damn handy to be able to link against local versions of packages when developing libraries/verifying changes etc.

Two options I can think of thus far.

1. Dropping a symlink to .spago/package/local/
2. Playing with packages .dhall to link in the other package.

The first is simple, but if your package has other deps it could be a fun little dependency hell.

For the latter, dhall is powerful enough that you can do things like

let tiled = ../tiled/spago.dhall

let mkPackage =
      https://raw.githubusercontent.com/spacchetti/spacchetti/20190105/src/mkPackage.dhall sha256:90974a8e07650e49a4197d4ce5b59e82740fd8469c8c1b9793939845ca8faad9

let upstream =
      https://raw.githubusercontent.com/spacchetti/spacchetti/20190105/src/packages.dhall sha256:04da9c924e05cb6b43447ffd13408d01992f3172eb80d5d94e7f1b14c76a5123

let overrides = {=}

let additions = 
    { tiled  = 
        mkPackage 
          tiled.dependencies
          "..\\..\\..\\..\\tiled"
          "master"
    }

let tiledset = tiled.packages    

in  upstream ⫽ overrides ⫽ additions // tiledset

This works, with the caveat that if you update the repo's master, you need to delete .spago\tiled\master\ to get spago install to copy everything again.

I can also appreciate that it's better to guide everyone to contribute more to the package sets, but this is more of a situation where having my own forked spachetti set/ overrides seems a tad overkill, especially as I'm still experimenting with what the 'package' should look like, so there's little purposing in publishing any of it.. yet.

What do you think about adding devDependencies option (to both Package and Config)? The motivation here is that most test frameworks are pretty big, with lots of transitive dependencies and different libs can use different test frameworks, wich adds even more packages that I will have to download, but most of the time will not use. Downloading only those packages that are required for library to work should make installation much faster.

Hi and thank you for this great repo. I'm thinking about additions. Let's say I try to install the purescript-graphql library. This is not in the default package-set so I add it

{
    graphql =
    mkPackage
     [ "aff"
     , "argonaut-core"
     , "httpure"
     , "argonaut-codecs"
     ]
     "https://github.com/hendrikniemann/purescript-graphql"
     "master"
}

I then run spago install and it installs graphql correctly. Now when running spago build I get errors on the dependencies graphql needs (Node.Stream for example). So I add purescript-node-streams etc. Do I do this for every dependency the addition relies on?

It makes sense for me that I would do this seeing that spago cannot know how to install dependencies from other additional dependencies. Just wanted to be 100% I'm not missing anything.

For now, it just outputs:

spago: Installation failed.

I think it might be useful to output more contextual info about the installation error.

Followup from #47 (comment): we should build binaries for Windows in CI (with AppVeyor or Azure Pipelines), and upload them to GitHub releases.

(Or if we want to have it even more awesome: add a way to install them with npm)

To use spago with psc-ide-emacs, will either need to specify some user globs yourself or make a PR to psc-ide-emacs: https://github.com/epost/psc-ide-emacs

In particular, see this block for how source globs are prepared and inserted:

https://github.com/epost/psc-ide-emacs/blob/01a158b77210fec9c1bfc0caffaf08fccc0412ac/psc-ide.el#L263-L274

(defun psc-ide--server-start-globs ()
  "Detects bower and psc-package projects and determines sensible
  source globs"

  (when (and (file-exists-p "psc-package.json") (file-exists-p "bower.json"))
    (message "Detected both a \"psc-package.json\" and a \"bower.json\" file."))

  (let ((server-globs psc-ide-source-globs))

    ;; see this section specifically, which handles psc-package globs
    (if (file-exists-p "psc-package.json")                        ; first, we check for psc-package.json
        (let* ((results "*PSC-PACKAGE SOURCES*")                  ; we let stdout go to a buffer named *PSC-PACKAGE SOURCES*
               (errors "*PSC-PACKAGE ERRORS*"))                   ; stderr goes ot the errors buffer

          (shell-command "psc-package sources" results errors)    ; run the psc-package sources command

          (if (get-buffer errors)                                 ; if there's an error, display it
              (switch-to-buffer-other-window errors)
            (progn                                                ; otherwise, we can take the results and kill the results buffer
              (with-current-buffer results
                (let* ((globs (split-string (buffer-string))))
                  (setq server-globs (append server-globs globs)) ; then we set the server globs with our globs being added to it
                  (kill-buffer results)))
              (message "Set source globs from psc-package. Starting server..."))))
              ;; end of psc-package glob handling


      (if (file-exists-p "bower.json")
          (setq server-globs (append server-globs '("bower_components/purescript-*/src/**/*.purs")))
        (message "Couldn't find psc-package.json nor bower.json files, using just the user specified globs.")))
server-globs))

It might be useful to be able to start a REPL from spago

We discussed on twitter this might be related to old dhall, but I just installed it straight from git and that didn't fix it.

$ spacchetti local-setup --force
spacchetti: 
Error: Invalid input

(stdin):114:1:
    |
114 | let upstream =
    | ^^
unexpected "le"
expecting "!=", "&&", "++", "->", "//", "//\\", "/\", "==", "as", "in", "||", "→", "∧", "⩓", "⫽", '#', '*', '+', ':', '?', or whitespace

See spacchetti/spacchetti#16

I'm getting a compilation error saying that Main has been defined multiple times and it's pointing at the test directory of my local dependencies.

This is currently done here, but we should now do it from this repo

If you try to run build before install, the build will fail, as purs won't be able to find the modules.

I think we should just run install before build in any case? (psc-package does this)

Need a sources command that will get the source globs

I'm getting Module Chat.Data.User was not found when trying to run tests with spago test command.

Here is my test directory structure:

.
├── Main.purs
└── Spec
    ├── Capability
    ├── Capability.purs
    ├── Data
    │   └── Message.purs
    ├── Data.purs
    └── Utils.purs

Actually, my playground project is here https://github.com/vyorkin-play/chat-api/ (I'm following the purescript-halogen-realworld, trying to build a simple halogen app).

Previously I used psc-package and the following "scripts":

    "test": "run-s compile:test test:run",
    "test:run": "node -e 'require(\"./output/Test.Main\").main()'",
    "compile": "psc-package build",
    "compile:test": "psc-package build -- 'test/**/*.purs'",

There are various places where spacchetti-cli and such are referenced, so they should at least be updated to reference this project after releases.

Need a verify command that will attempt to install and build a given dependency. This can be done in almost any way, but a full set verify should install one package while verifying at a time.

We should be able to generate documentation for all the libraries in the package set (and/or just the libraries used in the project), in a similar way to what stack haddock does for Haskell projects.

The use-cases for this are multiple and useful:

• get specialized documentation for your project without having to rely on Pursuit (because you might have dependencies not from the package set and/or on Pursuit)
• more generally, allow an easy way to have an offline Pursuit
• generate a "static Pursuit" for every package set in Spacchetti and host that alongside the package set, which is effectively the Stackage approach.
  Note that this would obsolete #27, as it would entirely remove the "where do I upload my library" problem → you would just add it to a Spacchetti package-set, and have it "published" in the next Spacchetti release

How to achieve this:

• it is possible to generate html docs for the whole package-set by running a purs docs command.
  Justin had a good start with acme-spago. He included a spago project in there, but if this is integrated in spago itself, we wouldn't need it (as we would just generate that info on the fly)
• then we'd need a way to search in the existing docs. Since the whole thing needs to be static files (i.e. no backend), we should have a frontend-side search). A simple way to do this would be to generate markdown docs and then index them with something like Lunr (@justinwoo suggested scraping the externs JSON files, but I'm not sure what that would entail, so probably worth detailing more)

Hi. I'm adding purescript-express as an addition in packages.dhall like so

let additions = 
  { express  = 
      mkPackage 
      [ "foreign"
      , "foreign-generic"
      , "node-http"
      , "test-unit"
      , "aff"
      ]
      "https://github.com/nkly/purescript-express"
      "master"
  }

Then I do spago install. Now when doing

spago list-packages | grep express

I get

express (master, https://github.com/nkly/purescript-express)

But when I do spago build I get

Error 1 of 4:

  in module Server
  at src/Server.purs line 6, column 1 - line 6, column 61

    Module Node.Express.App was not found.
    Make sure the source file exists, and that it has been provided as an input to the compiler.

also when verifying the location of the package: spago sources | grep express
i get nothing.

does this have something to do with the verification of a package? and if it fails it will do so silently?

It would be nice to have a command that could remove all artifacts, produced by other commands (like build, repl and so on), except for those, that were produced by init.
Sometimes when something goes wrong it's easier to just clean and rebuild :)

I guess by Travis or something

https://github.com/spacchetti/spago/releases/tag/0.6.0.0

These release notes shoudl also include psc-package-* commands sometime

We should have a spago publish command, that publishes the docs for the current package to Pursuit (as pulp publish does)

We should parallelize the fetching of git repos on install, as psc-package already does

Running the python tests seems to fail on windows.

I'll spend some time later investigating it further.

python ..\spago-test.py
Expecting success from: "spago init"
Expecting failure from: "spago init"
Expecting success from: "rm spago.dhall packages.dhall"
Expecting success from: "spago init"
Expecting success from: "rm src/Main.purs"
Expecting success from: "spago init -f"
FAILURE: Spago should always succeed in doing init with force
Program output:
spago: .\packages.dhall: openFile: permission denied (Permission denied)

The issue you were already expecting :). I'll open this and document trying to get it running on windows. Maintainers: don't feel compelled to respond yet, I'll try to do as much as I can myself.

Running in cmd.exe, can't even get to running the install commands. Looks like the install script assumes bash, curl/wget and tar to be present.

UPDATE: Looks like npm install would never work, that script downloads binaries that have been prebuilt. Next steps for me getting this to work would be getting stack+haskell and trying to build it myself.

UPDATE2: stack install fails due to win32/turtle version mismatchs. Stack is implying to add Win32-2.5.4.1@sha256:e623a1058bd8134ec14d62759f76cac52eee3576711cb2c4981f398f1ec44b85 to extra deps. Will see what we get from that. (looks like Gabriella439/turtle#323 is the upstream issue)

UPDATE3: This built with the above change. Though stack install did have to be ran multiple times, but I do have a spago.exe :)

Trying yarn

yarn global add purescript-spago
yarn global v1.9.4
[1/4] Resolving packages...
[2/4] Fetching packages...
[------------------------------------------------------------------------------------------------------------------------------------------------] 0/501(node:11552) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead.
info [email protected]: The platform "win32" is incompatible with this module.
info "[email protected]" is an optional dependency and failed compatibility check. Excluding it from installation.
[3/4] Linking dependencies...
[4/4] Building fresh packages...
error C:\Users\beau.trepp\AppData\Local\Yarn\Data\global\node_modules\purescript-spago: Command failed.
Exit code: 1
Command: .\install
Arguments:
Directory: C:\Users\beau.trepp\AppData\Local\Yarn\Data\global\node_modules\purescript-spago
Output:
'.\install' is not recognized as an internal or external command,
operable program or batch file.
info Visit https://yarnpkg.com/en/docs/cli/global for documentation about this command.

Trying npm

npm install -g purescript-spago
C:\Users\beau.trepp\AppData\Roaming\npm\spago -> C:\Users\beau.trepp\AppData\Roaming\npm\node_modules\purescript-spago\spago

[email protected] install C:\Users\beau.trepp\AppData\Roaming\npm\node_modules\purescript-spago
./install

'.' is not recognized as an internal or external command,
operable program or batch file.
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] install: ./install
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR! C:\Users\beau.trepp\AppData\Roaming\npm-cache_logs\2018-12-18T03_17_01_744Z-debug.log

Trying stack install

Error: While constructing the build plan, the following exceptions were encountered:

In the dependencies for turtle-1.5.12:
Win32-2.6.1.0 from stack configuration does not match >=2.2.0.1 && <2.6 (latest matching
version is 2.5.4.1)
needed due to spago-0.6.0.0 -> turtle-1.5.12

Some different approaches to resolving this:

• Set 'allow-newer: true' in C:\sr\config.yaml to ignore all version constraints and build anyway.

• Consider trying 'stack solver', which uses the cabal-install solver to attempt to find some
  working build configuration. This can be convenient when dealing with many complicated
  constraint errors, but results may be unpredictable.

• Recommended action: try adding the following to your extra-deps
  in C:\VisualStudio_Projects\purescript\spago\stack.yaml:

Win32-2.5.4.1@sha256:e623a1058bd8134ec14d62759f76cac52eee3576711cb2c4981f398f1ec44b85

Plan construction failed.

There should be some tests so that we have some kind of assurance that we don't break stuff.

Anything is good to start, even just a bash script that installs the binary, runs some commands and checks that the return codes are all right.
This should prevent simple breakage, like what happened when 4e9c20e lead to #29

After #21 the feature set of this overlaps quite a lot with purp, so we might as well just port that in (since it's 100 lines)

In cases of more complex setups, it is useful to specify the source paths from which we build.
You can do this with psc-package build "your/source/path/here" "another/source/path", but this is a clumsy interface.
I propose we should have this looking like spago build --path "some/source" --path "some/other-source"

Following up from this Discourse thread, a big problem seems to be that when we have breaking changes in the compiler there are "ecosystem splits", and picking the right packages is hard (especially with bower) if the people just does npm install purescript (from personal experience, the 0.12 transition was indeed quite bad).

Spago is currently not immune to this problem: we're wrapping whatever compiler the user has installed, so if we have a post-0.12 package set, it's not gonna work if the user has purs-0.11 installed. In general, the "compiler version the package-set is compatible with" is not checked against "compiler version currently installed", and this is not right! (as it's easy to diverge)

As usual, the "Stack solution" for this is really good:

• every LTS has a compiler version associated with it
• stack installs a sandboxed version of the compiler for every project

The downside of this is that the tooling (read: editor integrations) will have to talk to spago instead of the compiler for things like purs ide.

In our case we would not download a version of the compiler, but we could:

• we'd depend on the compiler library, and run things through it instead of calling an external executable
• this means that a certain Spago version would have an associated Purs version (i.e. not dependent on the package-set)
• editor plugins should call stuff like spago ide instead of purs ide

Hi there, and thanks for Spago!

As I was creating a fork of a react-basic-starter to use Spago, I wanted to recommend a way to have the project be auto-built when making changes.
Right now there is in a workaround for people using IDE PureScript, but it would be nice to have an alternative for people who don't use the plugin.

I realize this is a little bit more complicated since Spago uses purs under the hood, and unlike pulp it doesn't support the watch flag.

As an aside, the project seems to work like a treat combined with Parcel 🎉

It is usual to add packages that are not in the package-set to the packages.dhall (I mean, this is basically our main feature). It would be nice to have an easy way to have them contributed to the upstream package-set, e.g. with something like spago upstream whatever-package.

What this command would do is:

• check that the package is not in the upstream package-set (if it is, prompt the user to upgrade)
• if it's not, open a pull request to spacchetti (e.g. with this) (note: we'll need to think of a way to handle auth. I guess passing a github token with a flag could be ok?)

After the latest software update on my Pop!_OS (which is basically Ubuntu), spago started to throw an error:

spago: error while loading shared libraries: libtinfo.so.5: cannot open shared object file: No such file or directory

After some investigation I found out that I have libtinfo.so.6 not libtinfo.so.5 and the way to fix this problem is to create this symlink:

sudo ln -s /path/to/libtinfo.so.6 /path/to/libtinfo.so.5

I understand that it's not exactly a spago issue, but I still want to bring this up

• to save people that are facing this issue some googling time
• to find out if this fix is a correct way solve this problem

With psc-package you can do something like this:

psc-package build -- -o path/to/folder

and with pulp you can use --build-path or -o flags. It would be nice if spago had something like this.

From #29

People who don't use releases will run into random broken things

We should update automatically the spacchetti version and sha in packages.dhall every time spacchetti releases. This should be done in the release step of spacchetti Travis.

Steps for automating this:

• sed the new release name in the urls in packages.dhall, more sed to remove the sha265:..
• run dhall freeze --inplace templates/packages.dhall

Needs a spec of spacchetti.dhall

Right now adding a package to the set is a somewhat tedious manual process.

Getting inspiration from here: https://github.com/justinwoo/spacchetti/blob/master/scripts/from-bower.pl

It would be cool to be able to do spacchetti from-bower simple-json and having it appear as an addition in the packages.dhall file.

4e9c20e

When I run test command, I end up with a bunch of Module Foo.Bar was not found errors. It just doesn't see modules in src folder. So I guess the answer to this question is "yes" :)

As we discussed some time ago with @justinwoo, the name of this repo/program has a bit of a name clash with the spacchetti package set repo, and this is super confusing.

So I'd like to propose a new name for this repo and tool: spago
(IPA: /ˈspaɡo/)

Why?

• It means "twine" in Italian, and it would symbolize the new role of this tool (well at least after #21), to "tie everything together"
• It is different than spacchetti
• It kind of reminds of it at the same time (they have the same root)
• It is actually shorter to type at the command line, but unique enough

Sample image to visualize it: