ndmitchell / weeder Goto Github PK

Detect dead exports or package imports

License: BSD 3-Clause "New" or "Revised" License

Haskell 97.90% Shell 0.45% PowerShell 0.45% Logos 0.23% Batchfile 0.96%

weeder's Introduction

Weeder

Weeder has moved!

Weeder 2.0 is being developed at https://github.com/ocharles/weeder on different foundations. This repo is for historical reference only.

Most projects accumulate code over time. Weeder detects unused Haskell exports, allowing dead code to be removed (pulling up the weeds). Weeder piggy-backs off files generated by stack, so first obtain stack, then:

Install weeder by running stack install weeder --resolver=nightly.
Ensure your project has a stack.yaml file. If you don't normally build with stack then run stack init to generate one.
Make sure you have ghc-options: {"$locals": -ddump-to-file -ddump-hi} in your stack.yaml.
Run weeder . --build, which builds your project with stack and reports any weeds.

What does Weeder detect?

Weeder detects a bunch of weeds, including:

You export a function helper from module Foo.Bar, but nothing else in your package uses helper, and Foo.Bar is not an exposed-module. Therefore, the export of helper is a weed. Note that helper itself may or may not be a weed - once it is no longer exported -fwarn-unused-binds will tell you if it is entirely redundant.
Your package depends on another package but doesn't use anything from it - the dependency should usually be deleted. This functionality is quite like packunused, but implemented quite differently.
Your package has entries in the other-modules field that are either unused (and thus should be deleted), or are missing (and thus should be added). The stack tool warns about the latter already.
A source file is used between two different sections in a .cabal file - e.g. in both the library and the executable. Usually it's better to arrange for the executable to depend on the library, but sometimes that would unnecessarily pollute the interface. Useful to be aware of, and sometimes worth fixing, but not always.
A file has not been compiled despite being mentioned in the .cabal file. This situation can be because the file is unused, or the stack compilation was incomplete. I recommend compiling both benchmarks and tests to avoid this warning where possible - running weeder . --build will use a suitable command line.

Beware of conditional compilation (e.g. CPP and the Cabal flag mechanism), as these may mean that something is currently a weed, but in different configurations it is not.

I recommend fixing the warnings relating to other-modules and files not being compiled first, as these may cause other warnings to disappear.

What else should I use?

Weeder detects dead exports, which can be deleted. To get the most code deleted from removing an export, use:

GHC with -fwarn-unused-binds -fwarn-unused-imports, which finds unused definitions and unused imports in a module.
HLint, looking for "Redundant extension" hints, which finds unused extensions.
Unused, which works at the level of ctags information, so can be used if you don't want to use stack, can't compile your code, or want to detect unused code between projects.

Ignoring weeds

If you want your package to be detected as "weed free", but it has some weeds you know about but don't consider important, you can add a .weeder.yaml file adjacent to the stack.yaml with a list of exclusions. To generate an initial list of exclusions run weeder . --yaml > .weeder.yaml.

You may wish to generalise/simplify the .weeder.yaml by removing anything above or below the interesting part. As an example of the .weeder.yaml file from ghcid:

- message: Module reused between components
- message:
  - name: Weeds exported
  - identifier: withWaiterPoll

This configuration declares that I am not interested in the message about modules being reused between components (that's the way ghcid works, and I am aware of it). It also says that I am not concerned about withWaiterPoll being a weed - it's a simplified method of file change detection I use for debugging, so even though it's dead now, I sometimes do switch to it.

Running with Continuous Integration

Before running Weeder on your continuous integration (CI) server, you should first ensure there are no existing weeds. One way to achieve that is to ignore existing hints by running weeder . --yaml > .weeder.yaml and checking in the resulting .weeder.yaml.

On the CI you should then run weeder . (or weeder . --build to compile as well). To avoid the cost of compilation you may wish to fetch the latest Weeder binary release.

For the CI systems Travis, Appveyor and Azure Pipelines add the line:

curl -sSL https://raw.github.com/ndmitchell/weeder/master/misc/run.sh | sh -s .

The arguments after -s are passed to weeder, so modify the final . if you want other arguments. This command works on Windows, Mac and Linux.

What about Cabal users?

Weeder requires the textual .hi file for each source file in the project. Stack generates that already, so it was easy to integrate in to. There's no reason that information couldn't be extracted by either passing flags to Cabal, or converting the .hi files afterwards. I welcome patches to do that integration.

What about false positives?

Weeder strives to avoid incorrectly warning about something that is required, if you find such an instance please report it on the issue tracker. Unfortunately there are some cases where there are still false positives, as GHC doesn't put enough information in the .hi files:

Data.Coerce If you use Data.Coerce.coerce the constructors for the data type must be in scope, but if they aren't used anywhere other than automatically by coerce then Weeder will report unused imports. You can ignore such warnings by adding - message: Unused import to your .weeder.yaml file.

Declaration QuasiQuotes If you use a declaration-level quasi-quote then weeder won't see the use of the quoting function, potentially leading to an unused import warning, and marking the quoting function as a weed. The only solution is to ignore the entries with a .weeder.yaml file.

Stack extra-deps Packages marked extra-deps in your stack.yaml will be weeded, due to a bug in stack. The only solution is to ignore the packages with a .weeder.yaml file.

Linking to C functions If a library provides C functions, and these are used directly from another library/executable, the library providing these functions may be marked as a redundant build-depends, see more details.

weeder's People

Contributors

Stargazers

Watchers

Forkers

bergmark geigerzaehler fintanh sirlensalot brianwitte odr yazimai kindaro

weeder's Issues

Packages with 'extra-dep: true' should not get weeded

Technically you might call this a bug in 'stack query', which lists packages as 'local' when it probably shouldn't. But 'stack query' is marked as experimental. ;)

Integrate with code climate

I would like to automatically monitor the quality of my haskell code.
Could weeder output be connected to code climate?
See https://codeclimate.com/

Weeder does not detect redundant build dependecies in common stanzas

If we have a common stanza in our cabal file that includes a build-depends component with one or more dependencies, and each of the build targets that imports that common stanza does not require one or more of the dependencies listed in the build-depends of the common stanza, then weeder does not report these as redundant dependencies.

Consider the following .cabal file:

name: my-package

common deps

  build-depends: libA, libB

library

  import deps

  [... more stuff here ...]

executable exe

  import deps

  [... more stuff here ...]

If neither the executable exe nor the package library my-package require libA or libB, it would be very helpful for weeder to report libA and libB are redundant dependencies.

This issue is perhaps speculative, as I don't know if it is feasible for weeder to do this.

Support cabal projects

This is a very nice tool, and I wish I could use it without stack.

Wildcards in .weeder.yaml file

Two examples where it would be useful:

ignore modules, matching wildcard (protobuffer-generated modules)
ignore symbols, matching wildcard (groundhog-generated Fields, Selectors, Contructors, ...)

Weeder does not properly parse dependencies with version `-any`

If your dependencies have the -any flag for their version, weeder gives the following output:

= Package ats-pkg =
== Section exe:atspkg ==
Redundant build-depends entry
* ats-pkg -any
* base -any
* composition-prelude -any
* directory -any
* lens -any
* optparse-applicative -any
* shake -any
* shake-ats -any
* temporary -any
* text -any
== Section library ==
Redundant build-depends entry
* ansi-wl-pprint -any
* binary -any
* bytestring -any
* bzlib -any
* containers -any
* dependency -any
* dhall -any
* directory -any
* file-embed -any
* http-client -any
* http-client-tls -any
* lens -any
* lzma -any
* parallel-io -any
* process -any
* shake -any
* tar -any
* temporary -any
* text -any
* unix -any
* zip-archive -any
* zlib -any
= Package hs2ats =
No warnings
= Package dependency =
No warnings
= Package shake-ats =
No warnings

cabal file:

cabal-version: 1.18
name: ats-pkg
version: 2.5.0.3
license: BSD3
license-file: LICENSE
copyright: Copyright: (c) 2018 Vanessa McHale
maintainer: [email protected]
author: Vanessa McHale
homepage: https://github.com/vmchale/atspkg#readme
synopsis: A build tool for ATS
description:
    A collection of scripts to simplify building ATS projects.
category: Development, ATS
build-type: Custom
extra-source-files:
    stack.yaml
    man/atspkg.1
    config.dhall
extra-doc-files: README.md
                 docs/manual.tex

source-repository head
    type: git
    location: [email protected]:vmchale/atspkg.git

custom-setup
    setup-depends: base -any,
                   Cabal >=2.0,
                   cli-setup >=0.2.0.1

flag development
    description:
        Enable `-Werror`
    default: False
    manual: True

flag no-executable
    description:
        Enable `-Werror`
    default: False

library
    exposed-modules:
        Language.ATS.Package
    build-tools: cpphs -any
    hs-source-dirs: src
    other-modules:
        Paths_ats_pkg
        Language.ATS.Package.Error
        Language.ATS.Package.Type
        Language.ATS.Package.Dependency
        Language.ATS.Package.Compiler
        Language.ATS.Package.Build
        Language.ATS.Package.Build.IO
        Language.ATS.Package.Upgrade
        Language.ATS.Package.Config
        Language.ATS.Package.PackageSet
        Language.ATS.Package.Dhall
        Quaalude
    default-language: Haskell2010
    ghc-options: -Wall -Wincomplete-uni-patterns
                 -Wincomplete-record-updates -Wcompat
    build-depends:
        base >=4.7 && <5,
        http-client -any,
        bytestring -any,
        file-embed -any,
        shake -any,
        bzlib -any,
        lzma -any,
        tar -any,
        zlib -any,
        http-client-tls -any,
        text -any,
        directory -any,
        process -any,
        containers -any,
        parallel-io -any,
        unix -any,
        lens -any,
        dhall -any,
        ansi-wl-pprint -any,
        shake-ats >=1.3.0.0,
        shake-ext >=2.3.0.0,
        composition-prelude >=1.3.0.0,
        zip-archive -any,
        temporary -any,
        ansi-wl-pprint -any,
        binary -any,
        dependency -any,
        ats-setup >=0.3.1.1
    
    if flag(development)
        ghc-options: -Werror

executable atspkg
    main-is: Main.hs
    hs-source-dirs: app
    other-modules:
        Paths_ats_pkg
    default-language: Haskell2010
    ghc-options: -threaded -rtsopts -with-rtsopts=-N -Wall
                 -Wincomplete-uni-patterns -Wincomplete-record-updates -Wcompat
    build-depends:
        base -any,
        ats-pkg -any,
        optparse-applicative -any,
        lens -any,
        shake-ats -any,
        temporary -any,
        directory -any,
        composition-prelude -any,
        text -any,
        shake -any
    
    if flag(no-executable)
        buildable: False
    
    if flag(development)
        ghc-options: -Werror

When the versions are removed it is back to normal:

= Package ats-pkg =
No warnings
= Package hs2ats =
No warnings
= Package dependency =
No warnings
= Package shake-ats =
No warnings

cabal file:

cabal-version: 1.18
name: ats-pkg
version: 2.5.0.3
license: BSD3
license-file: LICENSE
copyright: Copyright: (c) 2018 Vanessa McHale
maintainer: [email protected]
author: Vanessa McHale
homepage: https://github.com/vmchale/atspkg#readme
synopsis: A build tool for ATS
description:
    A collection of scripts to simplify building ATS projects.
category: Development, ATS
build-type: Custom
extra-source-files:
    stack.yaml
    man/atspkg.1
    config.dhall
extra-doc-files: README.md
                 docs/manual.tex

source-repository head
    type: git
    location: [email protected]:vmchale/atspkg.git

custom-setup
    setup-depends: base,
                   Cabal >=2.0,
                   cli-setup >=0.2.0.1

flag development
    description:
        Enable `-Werror`
    default: False
    manual: True

flag no-executable
    description:
        Enable `-Werror`
    default: False

library
    exposed-modules:
        Language.ATS.Package
    build-tools: cpphs
    hs-source-dirs: src
    other-modules:
        Paths_ats_pkg
        Language.ATS.Package.Error
        Language.ATS.Package.Type
        Language.ATS.Package.Dependency
        Language.ATS.Package.Compiler
        Language.ATS.Package.Build
        Language.ATS.Package.Build.IO
        Language.ATS.Package.Upgrade
        Language.ATS.Package.Config
        Language.ATS.Package.PackageSet
        Language.ATS.Package.Dhall
        Quaalude
    default-language: Haskell2010
    ghc-options: -Wall -Wincomplete-uni-patterns
                 -Wincomplete-record-updates -Wcompat
    build-depends:
        base >=4.7 && <5,
        http-client,
        bytestring,
        file-embed,
        shake,
        bzlib,
        lzma,
        tar,
        zlib,
        http-client-tls,
        text,
        directory,
        process,
        containers,
        parallel-io,
        unix,
        lens,
        dhall,
        ansi-wl-pprint,
        shake-ats >=1.3.0.0,
        shake-ext >=2.3.0.0,
        composition-prelude >=1.3.0.0,
        zip-archive,
        ansi-wl-pprint,
        binary,
        dependency,
        ats-setup >=0.3.1.1
    
    if flag(development)
        ghc-options: -Werror

executable atspkg
    main-is: Main.hs
    hs-source-dirs: app
    other-modules:
        Paths_ats_pkg
    default-language: Haskell2010
    ghc-options: -threaded -rtsopts -with-rtsopts=-N -Wall
                 -Wincomplete-uni-patterns -Wincomplete-record-updates -Wcompat
    build-depends:
        base,
        ats-pkg,
        optparse-applicative,
        lens,
        shake-ats,
        temporary,
        directory,
        composition-prelude,
        text,
        shake
    
    if flag(no-executable)
        buildable: False
    
    if flag(development)
        ghc-options: -Werror

I think this is independent of stack because it depends only on the .cabal file (that is, I can get weeder to give distinct outputs without re-running stack).

Thanks :)

hspec-discover

Says the main function is a weed when that is required in all *Spec.hs files

Doesn't flag unnecessary transitive dependencies

If package a depends on b, and your package depends on a and b despite using nothing directly in b, then weeder doesn't spot it as redundant. In practice, everything in Windows depends on Win32, so adding Win32 to a package is a horrible dependency (breaks cross-platformness), but weeder doesn't spot it. Either see if we can get the necessary info out of the hi files, or document it as a known weakness.

CC @pepeiborra

Feature request: output styled as compiler errors

This would allow editor integration, quickly jumping to trouble spots.

Output format

Thanks for weeder! It's super useful keeping our codebase in check.

Something that's been a bit hard for us as we ramp up more people is interpreting the output from weeder. In our particular case, we have a bunch of exceptions to the rules (NoImplicitPrelude means base is sometimes marked as unused but the code won't compile otherwise). Those exceptions mean that when there's a weeder error, we get a wall of text to read through and find the fix. We tend to run with weeder --match . so it hits all of the code in the repo.

Is there another way to format the output that might make understanding failures easier? If not, would you be open to a PR to reformat the output to make understanding failures easier?

Executables with same name as a submodule tree are skipped

I've encountered a behaviour where if a project has an executable with the same name as a submodule tree it gets ignored. The following .cabal file reproduces the problem

name:           weeder-bug
version:        0.1.0.0
build-type:     Simple
cabal-version:  >= 1.10

library
  hs-source-dirs: src
  build-depends: base >=4.7 && <5
  exposed-modules: Myexe
  other-modules: Myexe.Int
  default-language: Haskell2010

executable myexe
  main-is: Main.hs
  other-modules: MainWeed
  hs-source-dirs: app
  ghc-options: -threaded -rtsopts -with-rtsopts=-N
  build-depends: base >=4.7 && <5
               , weeder-bug
  default-language: Haskell2010

Where src/ contains

src
├── Myexe
│   └── Int.hs
└── Myexe.hs

and app/ contains

app
├── Main.hs
└── MainWeed.hs

where MainWeed.hs contains a weed. Running weeder -b . produces the following:

== Section exe:myexe ==
Module not compiled
* Main.hs
* MainWeed
Redundant build-depends entry
* base
* weeder-bug

When I remove the Myexe.Int submodule from the package, weeder -b . correctly reports:

== Section exe:myexe ==
Weeds exported
* MainWeed
  - weed

I've attached the example:
weeder-bug.zip

HSC executables are marked as weed

HSC executables (*.hsc) and all their dependencies are marked as weed.

main executable is marked as a weed

After running weeder on my project (Ampersand), it warns that the main executable is redundant. I firmly believe it is not 😁

This is the output:

PS C:\Users\hjo20125\Git\Ampersand> weeder . --build
weeder : ampersand-3.9.1: unregistering (components added: exe:ampersand, test:ampersand-test, test:regression-test)
At line:1 char:1
+ weeder . --build
+ ~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (ampersand-3.9.1...egression-test):String) [], RemoteException
    + FullyQualifiedErrorId : NativeCommandError
 
ampersand-3.9.1: configure (lib + exe + test)
Configuring ampersand-3.9.1...
ampersand-3.9.1: build (lib + exe + test)
Static files unchanged, no need to update src/Ampersand/Prototype/StaticFiles_Generated.hs
Preprocessing library ampersand-3.9.1...
[  9 of 112] Compiling Ampersand.Basics.BuildInfo_Generated ( src\Ampersand\Basics\BuildInfo_Generated.hs, .stack-work\dist\ca59d0ab\build\Ampersand\Basics\BuildInfo_Generated.o )
Preprocessing executable 'ampersand' for ampersand-3.9.1...
Linking .stack-work\dist\ca59d0ab\build\ampersand\ampersand.exe ...
Preprocessing test suite 'ampersand-test' for ampersand-3.9.1...
Linking .stack-work\dist\ca59d0ab\build\ampersand-test\ampersand-test.exe ...
Preprocessing test suite 'regression-test' for ampersand-3.9.1...
Linking .stack-work\dist\ca59d0ab\build\regression-test\regression-test.exe ...
ampersand-3.9.1: copy/register
Installing library in
C:\Users\hjo20125\Git\Ampersand\.stack-work\install\4938d489\lib\x86_64-windows-ghc-8.0.2\ampersand-3.9.1-9OW4IxuilJN65tElRslO2z
Installing executable(s) in
C:\Users\hjo20125\Git\Ampersand\.stack-work\install\4938d489\bin
Registering ampersand-3.9.1...
ampersand-3.9.1: Test running disabled by --no-run-tests flag.
Completed 2 action(s).
= Package ampersand =

== Section exe:ampersand ==
Module not compiled
* Main.hs
Redundant build-depends entry
* ampersand
* base


PS C:\Users\hjo20125\Git\Ampersand>

Weeder doesn't work with stack 2.*

The new version of Stack doesn't generate .dump-hi files anymore (commercialhaskell/stack#4804).

The workaround is to tell stack to generate them: --ghc-options "-ddump-to-file -ddump-hi".

Are dump-hi files mandatory for weeder? Is possible to use .hi files only?

UPD. I guess the solution is to use http://hackage.haskell.org/package/hi-file-parser which is used by stack, but that's a lot of things to do.
UPD2. https://www.haskell.org/ghc/blog/20190626-HIEFiles.html

Promote weeder

Weeder doesn't really exist until I announce it to people. I should announce it once it's useful and is free of all the obvious simple defects. @bergmark - any thoughts as to whether we've reached that milestone? Anything else that is annoying you about weeder? What else needs doing before https://github.com/bos/aeson/tree/weeder-instances is merged? (if that is indeed your plan)

Non-exhaustive match error on invalid .weeder.yaml

Here's my incorrect yaml, I don't need help finding the error :)

package:
  - name: aeson
  - section:
    - name: library
    - message:
      - name: Weeds exported
      - module:
        - name: Data.Aeson.Types.Generic
        - identifier:
        - And

Weeder doesn't respect `STACK_YAML` environment variable

I mostly work on monorepos that have one stack.yaml file at the top level of a multi-package git repository. Accordingly, most of my Makefiles look something like:

stack_yaml = STACK_YAML="../stack.yaml"
stack = $(stack_yaml) stack
package = my-pkg
version = v0.0.5.9

What I'd like to be able to do is to use STACK_YAML="../stack.yaml" but it seems to unconditionally look for the file at ./ which makes running it on a single package impossible for the moment.

How do I install weeder?

I'm using Stack, but am not yet very familiar with it. I assume that I should add weeder as a package to my project, and then e.g. issue stack install weeder?

Or do I need to install weeder independent of my project? However, given that Stack even installs ghc as part of a project, that would seem strange.

Always show library warnings in order

The warnings should be in the same order as the sections in the .cabal file, which in practice means libary warnings should always come first. Currently I don't think it obeys that rule.

weeder seems to misparse build-depends

weeder-1.0.3 reports the following for threepenny-gui:

== Section library ==
Redundant build-depends entry
* aeson(

I suspect that this is due to the version bounds on that dependency:

                    ,aeson                  (>= 0.7 && < 0.10) || == 0.11.* || (>= 1.0 && < 1.4)

Running weeder on IHaskell uses all my RAM and swap

Does this indicate a memory leak or is it just because IHaskell is a large project?

$ git clone https://github.com/gibiansky/IHaskell
$ cd IHaskell
$ weeder
(watch as weeder fills all 16GB of my RAM and 8GB of swap before being killed)

I'm using weeder v0.1.6, which is the latest available in LTS-9.0.

Doesn't seem to understand generated modules

I seem to get incorrect output running weeder on a project that uses alex and happy. I have another project that uses hsc2hs with the same problem.

I don't know if it's possible to do anything about this, but it would be neat if it could be supported.

Example project: https://hackage.haskell.org/package/config-value

= Package config-value =

== Section library ==
Excessive other-modules entry
* Config.LexerUtils
Missing other-modules entry
* Config.Lexer
* Config.Parser
Module not compiled
* Config.Lexer
* Config.Parser
Weeds exported
* Config.LexerUtils
  - Action
  - AlexInput
  - InComment
  - InCommentString
  - InNormal
  - InString
  - LexerMode
  - alexGetByte
  - byteForChar
  - endMode
  - eofAction
  - errorAction
  - floating
  - move
  - nestMode
  - number
  - park
  - section
  - startPos
  - startString
  - token
  - token_
  - untermString

Change the .hi file search mechanism

It should find all the .hi files, not load any of them (or perhaps do so lazy), and then assign heuristics to each one by how likely it is to represent a given module. The current logic is complex and incomplete, so heuristics are much more likely to get somewhere. Concretely:

temp/Foo/Bar.hi-dump should be guessed at being Foo.Bar, but with an outside possibility of Temp.Foo.Bar (you might be on a case insensitive file system) or Bar (Foo might be the name of the exe). Using these techniques hopefully the hacks for lexer files will go away, and Paths_foo can be dealt with properly rather than never being found. Also case insensitivity is a property of file system, not OS, as I currently pretend in #42 .

Warnings about orphan imports

Can be seen on this aeson branch https://github.com/bos/aeson/tree/weeder-instances

Weeder warns on these test suite modules:

Excessive other-modules entry
* DataFamilies.Instances
* Functions
* Instances

But Instances is definitely used, the import is import Instances ().

How about not warning on these imports, or do you think it's better to manually ignore them?

Edit: Functions actually exports identifiers but is only used by Instances so that makes things slightly more inconvenient ;-)

False negative on executable's redundant build-depends used in library

File test-weeder-example.cabal:

name:                test-weeder-example
version:             0.1.0.0
cabal-version:       >=1.10

library
  hs-source-dirs:      src
  default-language:    Haskell2010
  exposed-modules:  Foo
  build-depends:
      base >=4.9 && < 5
    , containers

executable bar
  hs-source-dirs:      bar
  main-is:             Main.hs
  default-language:    Haskell2010
  build-depends:
      base >= 4.9 && < 5
    , test-weeder-example
    , containers

File src/Foo.hs:

module Foo (foo) where

import qualified Data.Map as Map

fooData = Map.singleton "foo" True

foo :: Maybe Bool
foo = Map.lookup "foo" fooData

File bar/Main.hs:

module Main where

import Foo (foo)

main = print foo

weeder does not detect that the containers dep on the executable is redundant; I assume this is because it's actually being used by the library, and weeder's somehow getting confused about the transitive dependency between the local components.

Detect unused exports even when exposed

I would love a way to tell weeder to flag functions that are exported, even from exposed modules, but not used by anything.

The use-case is an internal application. It has a library and exposed module(s), but it's a closed world -- nothing outside of this application will ever use these modules. So even though a function is exported from an exposed module, it is still a weed if unused anywhere else in the package.

Cannot run weeder using --resolver nightly

I'm running into some issues on CI when building on Stackage nightly. Here is the full case: https://app.circleci.com/jobs/github/pbrisbin/bugsnag-haskell/852.

Since nightly moves, I use the following stack-nightly.yaml on CI:

# overriden by --resolver on CI
resolver: nightly-2019-11-29

# Fix weeder with stack-2.0
ghc-options: { "$locals": -ddump-to-file -ddump-hi }

set via STACK_YAML. Since you can't put resolver: nightly in stack.yaml, I pass --resolver nightly on CI, as noted. This drift is causing weeder to not function.

In the Lint step of the build, I see

% stack exec --no-terminal --resolver nightly weeder .
Getting project config file from STACK_YAML environment
Selected resolver: nightly-2020-02-26
Stack has not been tested with GHC versions above 8.6, and using 8.8.2, this may fail
Stack has not been tested with Cabal versions above 2.4, but version 3.0.1.0 was found, this may fail

Notice resolver nightly-2020-02-26, ghc 8.8.2, and Cabal 3.0.1.0, which is all correct. But then (I assume) weeder is going to re-execute stack. It loses the --resolver argument and is using the resolver in the bare stack.yaml.

So, I see this output:

Getting project config file from STACK_YAML environment
Stack has not been tested with GHC versions above 8.6, and using 8.8.1, this may fail
Preparing to install GHC to an isolated location.
This will not interfere with any system-level installation.
Preparing to download ghc-8.8.1 ...
ghc-8.8.1: download has begun
ghc-8.8.1:   40.06 MiB / 186.83 MiB ( 21.44%) downloaded...
ghc-8.8.1:   81.84 MiB / 186.83 MiB ( 43.81%) downloaded...
ghc-8.8.1:  121.86 MiB / 186.83 MiB ( 65.22%) downloaded...
ghc-8.8.1:  158.23 MiB / 186.83 MiB ( 84.69%) downloaded...
ghc-8.8.1:  186.83 MiB / 186.83 MiB (100.00%) downloaded...
Downloaded ghc-8.8.1.
Unpacking GHC into /root/.stack/programs/x86_64-linux/ghc-8.8.1.temp/ ...
Configuring GHC ...
Installing GHC ...
Installed GHC.
Stack has not been tested with Cabal versions above 2.4, but version 3.0.0.0 was found, this may fail
Getting project config file from STACK_YAML environment

We've now got ghc 8.8.1 and Cabal 3.0.0.0. As you might expect, weeder fails:

weeder: /root/project/.stack-work/dist/x86_64-linux/Cabal-3.0.0.0: getDirectoryContents:openDirStream: does not exist (No such file or directory)

Is there some way to make this function? To get weeder to pass --resolver nightly along when it re-executes Stack?

My current workaround is easy: don't lint the Nightly CI job, but I thought this was an interesting enough edge case to report.

Non-exhaustive patterns in function fromString

Running this on https://github.com/vaibhavsagar/suppandi gives me the following error:

weeder: src/Stack.hs:34:9-42: Non-exhaustive patterns in function fromString

Fails on minimal stack.yaml

Fails on a stack.yaml that only consists of a one line Windows file containing just:
resolver: nightly-2017-04-24

Error is:

> weeder .
weeder: Failed to parse stack file
CallStack (from HasCallStack):
  error, called at src\Stack.hs:19:15 in main:Stack

Forcing a new line at the end of the file didn't help.

Executable's redundant depend suggestion breaks compilation

Given this project:

package.yaml

name: weeder-repro

default-extensions:
  - NoImplicitPrelude

dependencies:
  - base

library:
  source-dirs: src
  dependencies:
    - text

executables:
  example:
    main: Main.hs
    source-dirs: app
    dependencies:
      - weeder-repro

src/Lib.hs

module Lib (main) where

import Prelude

import Data.Text (pack)
import qualified Data.Text.IO as T

main :: IO ()
main = T.putStrLn $ pack "hi"

app/Main.hs

module Main
    ( module Lib
    )
where

import Lib (main)

Build the project:

stack init
stack setup
stack install --copy-compiler-tool weeder
stack build --test --no-run-tests --bench --no-run-benchmarks

Run weeder, observe the following error:

% stack exec weeder 
= Package weeder-repro =

== Section exe:example ==
Redundant build-depends entry
* base

This seems reasonable. The exectuable does not in fact use anything from base.

Apply the suggestion:

name: weeder-repro

default-extensions:
  - NoImplicitPrelude

library:
  source-dirs: src
  dependencies:
    - base
    - text

executables:
  example:
    main: Main.hs
    source-dirs: app
    dependencies:
      - weeder-repro

Build again, and observe it breaks compilation:

% stack build --test --no-run-tests --bench --no-run-benchmarks
weeder-repro-0.0.0: unregistering (local file changes: package.yaml weeder-repro.cabal)
weeder-repro-0.0.0: configure (lib + exe)
Configuring weeder-repro-0.0.0...
weeder-repro-0.0.0: build (lib + exe)
Preprocessing library for weeder-repro-0.0.0..
Building library for weeder-repro-0.0.0..
Preprocessing executable 'example' for weeder-repro-0.0.0..
Building executable 'example' for weeder-repro-0.0.0..
[2 of 2] Compiling Paths_weeder_repro ( .stack-work/dist/x86_64-linux/Cabal-2.4.0.1/build/example/autogen/Paths_weeder_repro.hs, .stack-work/dist/x86_64-linux/Cabal-2.4.0.1/build/example/example-tmp/Paths_weeder_repro.o ) [Prelude changed]

/home/patrick/code/pbrisbin/weeder-repro/.stack-work/dist/x86_64-linux/Cabal-2.4.0.1/build/example/autogen/Paths_weeder_repro.hs:10:1: error:
    Could not load module ‘Control.Exception’
    It is a member of the hidden package ‘base-4.12.0.0’.
    Perhaps you need to add ‘base’ to the build-depends in your .cabal file.
    Use -v to see a list of the files searched for.
   |
10 | import qualified Control.Exception as Exception
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

/home/patrick/code/pbrisbin/weeder-repro/.stack-work/dist/x86_64-linux/Cabal-2.4.0.1/build/example/autogen/Paths_weeder_repro.hs:11:1: error:
    Could not load module ‘Data.Version’
    It is a member of the hidden package ‘base-4.12.0.0’.
    Perhaps you need to add ‘base’ to the build-depends in your .cabal file.
    Use -v to see a list of the files searched for.
   |
11 | import Data.Version (Version(..))
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

/home/patrick/code/pbrisbin/weeder-repro/.stack-work/dist/x86_64-linux/Cabal-2.4.0.1/build/example/autogen/Paths_weeder_repro.hs:12:1: error:
    Could not load module ‘System.Environment’
    It is a member of the hidden package ‘base-4.12.0.0’.
    Perhaps you need to add ‘base’ to the build-depends in your .cabal file.
    Use -v to see a list of the files searched for.
   |
12 | import System.Environment (getEnv)
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

/home/patrick/code/pbrisbin/weeder-repro/.stack-work/dist/x86_64-linux/Cabal-2.4.0.1/build/example/autogen/Paths_weeder_repro.hs:13:1: error:
    Could not load module ‘Prelude’
    It is a member of the hidden package ‘base-4.12.0.0’.
    Perhaps you need to add ‘base’ to the build-depends in your .cabal file.
    Use -v to see a list of the files searched for.
   |
13 | import Prelude
   | ^^^^^^^^^^^^^^


--  While building package weeder-repro-0.0.0 using:
      /home/patrick/.stack/setup-exe-cache/x86_64-linux/Cabal-simple_mPHDZzAJ_2.4.0.1_ghc-8.6.3 --builddir=.stack-work/dist/x86_64-linux/Cabal-2.4.0.1 build lib:weeder-repro exe:example --ghc-options " -ddump-hi -ddump-to-file -fdiagnostics-color=always"
    Process exited with code: ExitFailure 1

I'm not sure where the fault lies here. Maybe it's unreasonable to expect weeder to be aware of how these Paths_ modules will be built in a naive hpack-based package?

Confusing handling of Cabal internal libraries

Our project uses multiple packages as sub-libraries, one of which has multiple internal libraries that are re-exported.

When we run weeder on the package with multiple internal libraries, we get the following output:

= Package pcg-core =

== Section library ==
Module not compiled
* Analysis.Parsimony.Additive
* Analysis.Parsimony.Additive.Internal
* Analysis.Parsimony.Dynamic.DirectOptimization
* Analysis.Parsimony.Dynamic.DirectOptimization.Internal
* Analysis.Parsimony.Dynamic.DirectOptimization.Pairwise
* Analysis.Parsimony.Dynamic.DirectOptimization.Pairwise.FFI
* Analysis.Parsimony.Dynamic.DirectOptimization.Pairwise.Internal
* Analysis.Parsimony.Dynamic.DirectOptimization.Pairwise.NeedlemanWunsch
* Analysis.Parsimony.Dynamic.DirectOptimization.Pairwise.Ukkonen
* Analysis.Parsimony.Dynamic.DirectOptimization.Pairwise.Ukkonen.Internal
* Analysis.Parsimony.Dynamic.DirectOptimization.Pairwise.Ukkonen.Ribbon
* Analysis.Parsimony.Fitch
* Analysis.Parsimony.Fitch.Internal
* Analysis.Parsimony.Sankoff
* Analysis.Parsimony.Sankoff.Internal
* Analysis.Scoring
* Analysis.TotalEdgeCost
* Bio.Character
* Bio.Character.Decoration.Additive
* Bio.Character.Decoration.Additive.Class
* Bio.Character.Decoration.Additive.Internal
* Bio.Character.Decoration.Continuous
* Bio.Character.Decoration.Continuous.Class
* Bio.Character.Decoration.Continuous.Internal
* Bio.Character.Decoration.Discrete
* Bio.Character.Decoration.Dynamic
* Bio.Character.Decoration.Dynamic.Class
* Bio.Character.Decoration.Dynamic.Internal
* Bio.Character.Decoration.Fitch
* Bio.Character.Decoration.Fitch.Class
* Bio.Character.Decoration.Fitch.Internal
* Bio.Character.Decoration.Metric
* Bio.Character.Decoration.Metric.Class
* Bio.Character.Decoration.Metric.Internal
* Bio.Character.Decoration.NonMetric
* Bio.Character.Decoration.NonMetric.Class
* Bio.Character.Decoration.NonMetric.Internal
* Bio.Character.Decoration.Shared
* Bio.Character.Encodable
* Bio.Character.Encodable.Continuous
* Bio.Character.Encodable.Continuous.Class
* Bio.Character.Encodable.Continuous.Internal
* Bio.Character.Encodable.Dynamic
* Bio.Character.Encodable.Dynamic.Class
* Bio.Character.Encodable.Dynamic.Internal
* Bio.Character.Encodable.Internal
* Bio.Character.Encodable.Static
* Bio.Character.Encodable.Static.Class
* Bio.Character.Encodable.Static.Internal
* Bio.Character.Encodable.Stream
* Bio.Character.Type
* Bio.Graph
* Bio.Graph.BinaryRenderingTree
* Bio.Graph.Component
* Bio.Graph.Constructions
* Bio.Graph.Forest
* Bio.Graph.LeafSet
* Bio.Graph.Node
* Bio.Graph.Node.Context
* Bio.Graph.Node.Internal
* Bio.Graph.PhylogeneticDAG
* Bio.Graph.PhylogeneticDAG.Class
* Bio.Graph.PhylogeneticDAG.DynamicCharacterRerooting
* Bio.Graph.PhylogeneticDAG.Internal
* Bio.Graph.PhylogeneticDAG.NetworkEdgeQuantification
* Bio.Graph.PhylogeneticDAG.Postorder
* Bio.Graph.PhylogeneticDAG.Preorder
* Bio.Graph.PhylogeneticDAG.Reification
* Bio.Graph.ReferenceDAG
* Bio.Graph.ReferenceDAG.Internal
* Bio.Graph.ReferenceDAG.Network
* Bio.Graph.ReferenceDAG.Traversal
* Bio.Graph.ReferenceDAG.Utility
* Bio.Graph.Solution
* Bio.Graph.ZipperDAG
* Bio.Graph.ZipperDAG.Internal
* Bio.Metadata
* Bio.Metadata.CharacterName
* Bio.Metadata.Continuous
* Bio.Metadata.Discrete
* Bio.Metadata.Discrete.Class
* Bio.Metadata.Discrete.Internal
* Bio.Metadata.DiscreteWithTCM
* Bio.Metadata.DiscreteWithTCM.Class
* Bio.Metadata.DiscreteWithTCM.Internal
* Bio.Metadata.Dynamic
* Bio.Metadata.Dynamic.Class
* Bio.Metadata.Dynamic.Internal
* Bio.Metadata.General
* Bio.Metadata.General.Class
* Bio.Metadata.General.Internal
* Bio.Sequence
* Bio.Sequence.Block
* Bio.Sequence.Block.Builder
* Bio.Sequence.Block.Character
* Bio.Sequence.Block.Internal
* Bio.Sequence.Block.Metadata
* Bio.Sequence.Character
* Bio.Sequence.Internal
* Bio.Sequence.Metadata
* Data.EdgeLength
* Data.EdgeSet
* Data.MetricRepresentation
* Data.NodeLabel
* Data.TCM
* Data.TCM.Dense
* Data.TCM.Dense.FFI
* Data.TCM.Internal
* Data.TopologyRepresentation
* Test.Custom.NucleotideSequence
Redundant build-depends entry
* 
* QuickCheck
* bimap
* binary
* bv-little
* cassava
* compact
* containers
* data-default
* deepseq
* dlist
* foldl
* graphviz
* hashable
* keys
* lens
* monad-loops
* mtl
* parallel
* pcg-alphabet
* pcg-analysis
* pcg-data-structures
* pcg-evaluation
* pcg-exportable
* pcg-serialize
* pcg-tcm
* pcg-tcm-memo
* pcg-utility
* pretty-tree
* semigroupoids
* smallcheck
* text
* text-short
* text-show
* text-show-instances
* unordered-containers
* vector
* vector-binary-instances
* vector-instances
* xml

== Section test:analysis-tests ==
Redundant build-depends entry
* 
* pcg-analysis
* pcg-data-structures
* pcg-tcm

== Section test:data-structures-tests ==
Redundant build-depends entry
* 
* pcg-data-structures

== Section test:tcm-tests ==
Redundant build-depends entry
* 
* pcg-tcm
* pcg-tcm-memo

= Package pcg-language =

== Section library ==
Redundant build-depends entry
*

weeder reports as weeds that the re-exported modules were not compiled, however these modules need to be re-exported from the package's main library as the interface to be consumed by other packages in our code base.

Additionally, weeder reports the internal libraries which are dependencies of other build targets as redundant, however they are require to build the build target and cannot be removed.

I'm hoping to get your input on whether or not these weed reports are expected, they seem to be erroneous from my perspective in this use case.

Optimise .hi file parsing

For large projects it takes quite a while (~30 secs). Profiling shows basically all time is spent in the Hi file parsing. Probably move to an alternative string type (foundation?)

"Module not compiled" for all modules in aeson

Here's a reproduction for aeson master, am I doing something wrong? It looks like the mentioned modules are being compiled

$ ln -s stack-lts8.yaml stack.yaml
$ weeder . --build
aeson-1.2.0.0: configure (lib + test)
aeson-1.2.0.0: build (lib + test)
aeson-1.2.0.0: copy/register
aeson-1.2.0.0: Test running disabled by --no-run-tests flag.
Completed 2 action(s).

--  Dumping log file due to warnings: /Users/adam.bergmark/repos/hs/aeson/.stack-work/logs/aeson-1.2.0.0.log

Configuring aeson-1.2.0.0...
Preprocessing library aeson-1.2.0.0...
[ 1 of 23] Compiling Data.Attoparsec.Time.Internal ( attoparsec-iso8601/Data/Attoparsec/Time/Internal.hs, .stack-work/dist/x86_64-osx/Cabal-1.24.2.0/build/Data/Attoparsec/Time/Internal.o )
[ 2 of 23] Compiling Data.Attoparsec.Time ( attoparsec-iso8601/Data/Attoparsec/Time.hs, .stack-work/dist/x86_64-osx/Cabal-1.24.2.0/build/Data/Attoparsec/Time.o )
[ 3 of 23] Compiling Data.Aeson.Types.Internal ( Data/Aeson/Types/Internal.hs, .stack-work/dist/x86_64-osx/Cabal-1.24.2.0/build/Data/Aeson/Types/Internal.o )
[ 4 of 23] Compiling Data.Aeson.Types.Generic ( Data/Aeson/Types/Generic.hs, .stack-work/dist/x86_64-osx/Cabal-1.24.2.0/build/Data/Aeson/Types/Generic.o )
[ 5 of 23] Compiling Data.Aeson.Parser.UnescapePure ( pure/Data/Aeson/Parser/UnescapePure.hs, .stack-work/dist/x86_64-osx/Cabal-1.24.2.0/build/Data/Aeson/Parser/UnescapePure.o )
[ 6 of 23] Compiling Data.Aeson.Parser.Unescape ( Data/Aeson/Parser/Unescape.hs, .stack-work/dist/x86_64-osx/Cabal-1.24.2.0/build/Data/Aeson/Parser/Unescape.o )
[ 7 of 23] Compiling Data.Aeson.Parser.Time ( Data/Aeson/Parser/Time.hs, .stack-work/dist/x86_64-osx/Cabal-1.24.2.0/build/Data/Aeson/Parser/Time.o )
[ 8 of 23] Compiling Data.Aeson.Parser.Internal ( Data/Aeson/Parser/Internal.hs, .stack-work/dist/x86_64-osx/Cabal-1.24.2.0/build/Data/Aeson/Parser/Internal.o )
[ 9 of 23] Compiling Data.Aeson.Parser ( Data/Aeson/Parser.hs, .stack-work/dist/x86_64-osx/Cabal-1.24.2.0/build/Data/Aeson/Parser.o )
[10 of 23] Compiling Data.Aeson.Internal.Time ( Data/Aeson/Internal/Time.hs, .stack-work/dist/x86_64-osx/Cabal-1.24.2.0/build/Data/Aeson/Internal/Time.o )
[11 of 23] Compiling Data.Aeson.Internal.Functions ( Data/Aeson/Internal/Functions.hs, .stack-work/dist/x86_64-osx/Cabal-1.24.2.0/build/Data/Aeson/Internal/Functions.o )
[12 of 23] Compiling Data.Aeson.Types.FromJSON ( Data/Aeson/Types/FromJSON.hs, .stack-work/dist/x86_64-osx/Cabal-1.24.2.0/build/Data/Aeson/Types/FromJSON.o )
[13 of 23] Compiling Data.Aeson.Internal ( Data/Aeson/Internal.hs, .stack-work/dist/x86_64-osx/Cabal-1.24.2.0/build/Data/Aeson/Internal.o )
[14 of 23] Compiling Data.Aeson.Encoding.Builder ( Data/Aeson/Encoding/Builder.hs, .stack-work/dist/x86_64-osx/Cabal-1.24.2.0/build/Data/Aeson/Encoding/Builder.o )
[15 of 23] Compiling Data.Aeson.Encoding.Internal ( Data/Aeson/Encoding/Internal.hs, .stack-work/dist/x86_64-osx/Cabal-1.24.2.0/build/Data/Aeson/Encoding/Internal.o )
[16 of 23] Compiling Data.Aeson.Encoding ( Data/Aeson/Encoding.hs, .stack-work/dist/x86_64-osx/Cabal-1.24.2.0/build/Data/Aeson/Encoding.o )
[17 of 23] Compiling Data.Aeson.Types.ToJSON ( Data/Aeson/Types/ToJSON.hs, .stack-work/dist/x86_64-osx/Cabal-1.24.2.0/build/Data/Aeson/Types/ToJSON.o )
[18 of 23] Compiling Data.Aeson.Types.Class ( Data/Aeson/Types/Class.hs, .stack-work/dist/x86_64-osx/Cabal-1.24.2.0/build/Data/Aeson/Types/Class.o )
[19 of 23] Compiling Data.Aeson.Types ( Data/Aeson/Types.hs, .stack-work/dist/x86_64-osx/Cabal-1.24.2.0/build/Data/Aeson/Types.o )
[20 of 23] Compiling Data.Aeson.Text  ( Data/Aeson/Text.hs, .stack-work/dist/x86_64-osx/Cabal-1.24.2.0/build/Data/Aeson/Text.o )
[21 of 23] Compiling Data.Aeson       ( Data/Aeson.hs, .stack-work/dist/x86_64-osx/Cabal-1.24.2.0/build/Data/Aeson.o )
[22 of 23] Compiling Data.Aeson.Encode ( Data/Aeson/Encode.hs, .stack-work/dist/x86_64-osx/Cabal-1.24.2.0/build/Data/Aeson/Encode.o )
[23 of 23] Compiling Data.Aeson.TH    ( Data/Aeson/TH.hs, .stack-work/dist/x86_64-osx/Cabal-1.24.2.0/build/Data/Aeson/TH.o )
Preprocessing test suite 'tests' for aeson-1.2.0.0...
[ 1 of 17] Compiling Types            ( tests/Types.hs, .stack-work/dist/x86_64-osx/Cabal-1.24.2.0/build/tests/tests-tmp/Types.o ) [Data.Aeson.Types changed]
[ 2 of 17] Compiling Options          ( tests/Options.hs, .stack-work/dist/x86_64-osx/Cabal-1.24.2.0/build/tests/tests-tmp/Options.o ) [Data.Aeson.Types changed]
[ 3 of 17] Compiling Functions        ( tests/Functions.hs, .stack-work/dist/x86_64-osx/Cabal-1.24.2.0/build/tests/tests-tmp/Functions.o ) [Prelude.Compat changed]
[ 4 of 17] Compiling Instances        ( tests/Instances.hs, .stack-work/dist/x86_64-osx/Cabal-1.24.2.0/build/tests/tests-tmp/Instances.o ) [Data.Hashable.Time changed]
[ 5 of 17] Compiling SerializationFormatSpec ( tests/SerializationFormatSpec.hs, .stack-work/dist/x86_64-osx/Cabal-1.24.2.0/build/tests/tests-tmp/SerializationFormatSpec.o ) [Data.UUID.Types changed]
[ 6 of 17] Compiling ErrorMessages    ( tests/ErrorMessages.hs, .stack-work/dist/x86_64-osx/Cabal-1.24.2.0/build/tests/tests-tmp/ErrorMessages.o ) [Data.HashMap.Strict changed]
[ 7 of 17] Compiling Encoders         ( tests/Encoders.hs, .stack-work/dist/x86_64-osx/Cabal-1.24.2.0/build/tests/tests-tmp/Encoders.o ) [Data.Aeson.Types changed]
[ 8 of 17] Compiling Properties       ( tests/Properties.hs, .stack-work/dist/x86_64-osx/Cabal-1.24.2.0/build/tests/tests-tmp/Properties.o ) [Data.UUID.Types changed]
[ 9 of 17] Compiling UnitTests.NullaryConstructors ( tests/UnitTests/NullaryConstructors.hs, .stack-work/dist/x86_64-osx/Cabal-1.24.2.0/build/tests/tests-tmp/UnitTests/NullaryConstructors.o ) [Data.Aeson.Types changed]
[10 of 17] Compiling DataFamilies.Types ( tests/DataFamilies/Types.hs, .stack-work/dist/x86_64-osx/Cabal-1.24.2.0/build/tests/tests-tmp/DataFamilies/Types.o ) [Generics.Deriving.TH changed]
[11 of 17] Compiling DataFamilies.Instances ( tests/DataFamilies/Instances.hs, .stack-work/dist/x86_64-osx/Cabal-1.24.2.0/build/tests/tests-tmp/DataFamilies/Instances.o ) [Data.Aeson.Types changed]
[12 of 17] Compiling DataFamilies.Encoders ( tests/DataFamilies/Encoders.hs, .stack-work/dist/x86_64-osx/Cabal-1.24.2.0/build/tests/tests-tmp/DataFamilies/Encoders.o ) [Data.Aeson.Types changed]
[13 of 17] Compiling DataFamilies.Properties ( tests/DataFamilies/Properties.hs, .stack-work/dist/x86_64-osx/Cabal-1.24.2.0/build/tests/tests-tmp/DataFamilies/Properties.o ) [Prelude.Compat changed]
[16 of 17] Compiling UnitTests        ( tests/UnitTests.hs, .stack-work/dist/x86_64-osx/Cabal-1.24.2.0/build/tests/tests-tmp/UnitTests.o ) [Data.HashSet changed]

/Users/adam.bergmark/repos/hs/aeson/tests/UnitTests.hs:501:47: warning: [-Wunused-top-binds]
    Defined but not used: ‘smf’
[17 of 17] Compiling Main             ( tests/Tests.hs, .stack-work/dist/x86_64-osx/Cabal-1.24.2.0/build/tests/tests-tmp/Main.o ) [Prelude.Compat changed]
Linking .stack-work/dist/x86_64-osx/Cabal-1.24.2.0/build/tests/tests ...
Installing library in
/Users/adam.bergmark/repos/hs/aeson/.stack-work/install/x86_64-osx/lts-8.1/8.0.2/lib/x86_64-osx-ghc-8.0.2/aeson-1.2.0.0-J6hepn47qr4IYivjVNPua
Registering aeson-1.2.0.0...

--  End of log file: /Users/adam.bergmark/repos/hs/aeson/.stack-work/logs/aeson-1.2.0.0.log

Log files have been written to: /Users/adam.bergmark/repos/hs/aeson/.stack-work/logs/
= Package attoparsec-iso8601 =
No warnings

= Package aeson =

== Section library ==
Module not compiled
* Data.Aeson
* Data.Aeson.Encode
* Data.Aeson.Encoding
* Data.Aeson.Encoding.Builder
* Data.Aeson.Encoding.Internal
* Data.Aeson.Internal
* Data.Aeson.Internal.Functions
* Data.Aeson.Internal.Time
* Data.Aeson.Parser
* Data.Aeson.Parser.Internal
* Data.Aeson.Parser.Time
* Data.Aeson.Parser.Unescape
* Data.Aeson.TH
* Data.Aeson.Text
* Data.Aeson.Types
* Data.Aeson.Types.Class
* Data.Aeson.Types.FromJSON
* Data.Aeson.Types.Generic
* Data.Aeson.Types.Internal
* Data.Aeson.Types.ToJSON
* Data.Attoparsec.Time
* Data.Attoparsec.Time.Internal
Redundant build-depends entry
* attoparsec
* base
* base-compat
* containers
* deepseq
* dlist
* ghc-prim
* hashable
* scientific
* tagged
* template-haskell
* text
* time
* time-locale-compat
* unordered-containers
* uuid-types
* vector

== Section test:tests ==
Module not compiled
* Data.Aeson.Parser.UnescapeFFI
* Data.Aeson.Parser.UnescapePure
* DataFamilies.Encoders
* DataFamilies.Instances
* DataFamilies.Properties
* DataFamilies.Types
* Encoders
* ErrorMessages
* Functions
* Instances
* Options
* Properties
* SerializationFormatSpec
* Tests.hs
* Types
* UnitTests
* UnitTests.NullaryConstructors
Redundant build-depends entry
* HUnit
* QuickCheck
* aeson
* attoparsec
* base
* base-compat
* base-orphans
* base16-bytestring
* containers
* directory
* dlist
* filepath
* generic-deriving
* ghc-prim
* hashable
* integer-logarithms
* quickcheck-instances
* scientific
* tagged
* template-haskell
* test-framework
* test-framework-hunit
* test-framework-quickcheck2
* text
* time
* time-locale-compat
* unordered-containers
* uuid-types
* vector

Add --stack-yaml flag

I run weeder as part of my CI build. I also run a set of builds at different resolvers, to test different sets of dependencies. This is fairly easy to do with stack --resolver or stack --stack-yaml, but I can't seem to make weeder respect that.

As an example, see here: https://circleci.com/gh/thoughtbot/yesod-auth-oauth2/279

Notice how this build is meant to test lts-11.18 / 8.2.2, but...

stack exec --stack-yaml stack-lts-11.18.yaml weeder .
Downloading lts-12.2 build plan ...
Downloaded lts-12.2 build plan.

Sometimes this happens to work, but other times (as in this case), it fails with:

weeder: /root/project/.stack-work/dist/x86_64-linux/Cabal-2.2.0.1: getDirectoryContents:openDirStream: does not exist (No such file or directory)

Which makes sense because of the compiler mismatch between what's built and what's being weeded (I think).

Is there any way to run weeder at a specific resolver? Ideally it would somehow know it's being run in a stack exec with a given resolver/stack.yml and respect that, but having to pass another option to weeder itself wouldn't be that bad.

`redundant build-depends entry` for convenience library with C(++) sources

Hello,

I recently came across this corner case: in this cabal file, there is a ("main") library, and a convenience library which just builds C++ sources.

The main library declares the convenience library as a dependance, so that the c++ objects built by the convenience library are available at link time for projects that depend on the main library.

If I remove this dependency, the main library builds, but the projects that depend on this main library have "symbol not found" link errors.

weeder detects the convenience library as a redundant build-depends entry of the main library.

In a way, I think this is a bug, and weeder should consider that dependencies on convenience libraries building C / C++ sources are never redundant (considering that here the semantics of the dependency is "aggregate build results").

What do you think?

Thanks!

PS: The reason I need separate libraries is that one library builds c (for a .hsc file), the other builds c++, and build options are not compatible (and cxx-sources / cxx-options is not yet available in the version of cabal used!).

Release v1.0

I see no reason not to release v1.0. I should do that soonish, with no changes.

False positive to remove imports that are needed for coercions

Using Data.Coerce.coerce, we need constructors in scope. In some of my files I have lines like

import Foo.Bar as ForCoerce

This import is only used to bring all constructors into scope for coerce to work. But weeder tells me they can be deleted. Deleting them leads to a project that no longer can compiles.

Can provide an example if needed.

weeder as library

The current recommendation to use weeder via a downloaded travis.sh script (for travis) and to use a nightly stack build for local installation is prone to build instability if developers do not maintain a regular process of re-installing weeder.

I have solved this problem with hlint by importing it as a library and running it as a test, as seen here: https://github.com/kadena-io/pact/blob/master/tests/HlintSpec.hs . This way the hlint version can be managed and upgraded from the cabal file, nobody ever "forgets" to run hlint, and CI integration is solved.

Weeder however does not export a library so it is not possible to work with it this way. Happy to provide a PR for this.

weeder: src/Warning.hs:(80,9)-(84,53): Non-exhaustive patterns in function f

When running this:

weeder . > .weeder.yaml
weeder: src/Warning.hs:(80,9)-(84,53): Non-exhaustive patterns in function f

My guess is that the redirection creates an empty file, which weeder fails to parse (from glancing at the source code).

Use less meory

Currently on big projects Weeder uses lots of RAM. Two obvious improvements:

Use packed strings in the Hi data structure.
Only parse Hi files by need, rather than the full directory in advance.

Ignore template-haskell generated output

At the moment if you use the lens generator to generate prisms/lenses etc from TH, you get a lot of complaints about unused pieces.

One potential solution would be to enable adding ghc-options: -ddump-splices and exclude things generated by TH, if the files exist. Downside is people won't do this by default, so it's very much an opt-in experience.

weeder crashes with "getDirectoryContents:openDirStream: does not exist"

I just run weeder on a large project with multiple cabal packages. It worked for some of the packages but at some point it crashed with

weeder: <project>/.stack-work/downloaded/W6D33wgrWbLW/.stack-work/dist/x86_64-linux/Cabal-1.24.0.0: getDirectoryContents:openDirStream: does not exist (No such file or directory)

Re-running weeder fails with the same command. Other than this line there aren't any error-related output. Let me know if I can help with debugging (if you need output of a command etc.). Thanks.

EDIT: This is on Linux.

Appveyor fails because of weeder

Since two days, all buids of the Ampersand project at Appveyor fail (see this log) because of the use of weeder. On Travis however, where I do not use weeder, the project builds fine.

There is something wrong after building yaml-0.8.28. It fails with the message Command exited with code 1. On my local windows machine, I cannot reproduce this. This issue breaks our automatic release job.

False positive when reexporting data types

If I have a module A that reexports a Data type:

module A (Word8) where
import Data.Word

and a module B that uses it:

module B where
import A

main :: IO ()
main = putStrLn (show 25 :: Word8)

Weeder wrongly estimates that A is an unused import in B.

However, if a export the whole module it doesn't report it (rightly):

module A (module Data.Word) where
import Data.Word

(similarly, if A reexports a function instead of a data type, e.g. byteSwap32, Weeder rightly doesn't report it)

Deal with TemplateHaskell

@hsyl20 reports: I have a false positive when an import used only in a Template Haskell splice. It is reported as unused.

Detect definitions used but not reexported

When working in a large project, I don't want to duplicate the dependencies of files/packages I use.
In other words, I would like my abstraction to be complete: no other imports are needed to work with it.

For example, assume a package defines X.
Another package defines a Producer and Consumer of X (and thus depends on the package defining X).
A third package, which wants to use the Producer and Consumer in a decoupled way needs to store the results in buffer of type X.
When the package of the Producer and Consumer does NOT re-export X, the third package must also depends on the package defining X. I consider this dependency unwanted: you must be able to use an interface as is, i.e., in a context where Producer is defined also X must be defined.

However, I was unable to find any tool, including weeder, that performs such a check!

So my question: can weeder check that all necessary definitions are include in the export of a file / package?

Modules imported with nothing in without orphans should be weeds

Sometimes people do import Foo() and Foo doesn't have any orphan instances, which is a clear weed. Following #7 I can detect that.

Feature request: option to detect library exports not used in executables/tests

weeder must assume that a library component can be depended upon by external packages and that all of its exports must be 'live'. However, a common pattern is for conceptually executables-only packages to have a library component, so that code can be shared across multiple executables and the test suite without recompilation; in this design, any exports from the library that aren't used by other components are 'weeds', and it would be handy for weeder to report them. Since weeder can't automatically distinguish between this case and the also-common pattern of a suitable-for-external-consumption library + executables all in one package, it would need to be an explicit toggle (e.g., probably a command-line switch).

Add flag to select which project

If a stack file lists multiple projects there should be a way to pick which projects are actually run.