Code Monkey home page Code Monkey logo

hoopl's Introduction

The hoopl Package Hackage Build Status

Hoopl: A Higher-Order OPtimization Library

API documentation can be found on Hackage. For detailed explanation of the library design see paper "Hoopl: A Modular, Reusable Library for Dataflow Analysis and Transformation"

Directory Contents
src/ The current official sources to the Cabal package
testing/ Tests, including a sample client. See testing/README

Development Notes

Building and testing

To build the library run:

cabal configure
cabal build
cabal install --enable-documentation

To run the tests in the testing/ folder run:

cabal configure --enable-tests
cabal test

To run the tests with the test coverage report run:

cabal configure --enable-tests --enable-coverage
cabal test

You'll need a Haskell Platform, which should include appropriate versions of Cabal and GHC.

Coding style

Please follow Johan Tibell's Haskell Style Guide for all new/modified code.

Checklist for Making Releases

In order to facilitate GHC development's workflow, the version in hoopl.cabal is to be bumped as soon as a change requires a respective version bump (according to the PVP) relative to the last released hoopl version.

  1. Make sure hoopl passes Travis for all GHC versions in the build-matrix
  2. Update Changelog (& git commit)
  3. Generate source tarball via cabal sdist and upload a candidate to Hackage (see note below), and inspect the result.
  4. If everything checks out, make an annotated and GPG-signed Git release tag: git tag -a -s v${VER} -m "hoopl ${VER}"
  5. Publish (there's a button for that on Hackage) the package candidate
  6. Work on next release

Note: To upload to Hackage,

cabal sdist
cabal upload dist/hoopl-*.tar.gz

However, it's recommended use the Hackage feature for uploading a candidate.

hoopl's People

Contributors

andreasvoellmy avatar bgamari avatar brk avatar dterei avatar erikd avatar ezyang avatar foxik avatar ggreif avatar gwils avatar hvr avatar igfoo avatar lewurm avatar lexpank avatar michalsosn avatar michalt avatar mlite avatar nrnrnr avatar ryanglscott avatar simonmar avatar thomie avatar treeowl avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

hoopl's Issues

hoopl-3.10.2.2 test suite does not compile

Preprocessing test suite 'hoopl-test' for hoopl-3.10.2.2..
Building test suite 'hoopl-test' for hoopl-3.10.2.2..
[1 of 1] Compiling Main             ( testing/Main.hs, dist/build/hoopl-test/hoopl-test-tmp/Main.o )

testing/Main.hs:8:1: error:
    Could not find module ‘Test’
    Use -v to see a list of the files searched for.
  |
8 | import qualified Test
  | ^^^^^^^^^^^^^^^^^^^^^

Maybe that file is missing from the release tarball on Hackage?

Stateful transformation causes non-termination in Hoopl analysis.

This ticket was filed by Andreas at https://ghc.haskell.org/trac/ghc/ticket/9853.

I can think of the following solutions:

  1. investigate Andreas's proposal of rolling back the side effects when a rewritten graph is dropped.
    2a. declare it's a fact of life, and document the limitation: A dataflow analysis and transformation that use non-deterministic algorithm to generate new variables, and the dataflow fact includes the new variables should be split to two part. a) use a pre tranformation step to introduce all new variables, b) a analysis and tranformation step that only simplfies the pre transformed code.
    2b. declare it's a fact of life, document it, and provide the utility functions to support the pre transformation.

Any other better solutions?

Runtime error caused by fromJust in module Dataflow

In line 274 of DataFlow, function arfx uses fromJust assuming that looking up a fact always produces a result. Would you please change the implementation into:

    arfx arf thing fb =
      case lookupFact (entryLabel thing) (joinInFacts lattice fb) of
        Nothing -> error $
                   "Dataflow.hs: arfx => Clueless on label:"
                   ++ (show . entryLabel) thing
        Just x -> arf thing x

It was hard to spot where the error was raised. Including clues in the error message should help.

Regards

Inconsistency between paper and library

Paper page 5

analyzeAndRewriteFwdBody
:: ( CkpointMonad m -- Roll back speculative actions
, NonLocal n ) -- Extract non-local flow edges
=> FwdPass m n f -- Lattice, transfer, rewrite
-> [Label] -- Entry point(s)
-> Graph n C C -- Input graph
-> FactBase f -- Input fact(s)
-> m ( Graph n C C -- Result graph
, FactBase f ) -- ... and its facts

Again same but single line

analyzeAndRewriteFwdBody :: ( CkpointMonad m, NonLocal n ) => FwdPass m n f -> [Label] -> Graph n C C -> FactBase f -> m (Graph n C C, FactBase f )

library

analyzeAndRewriteFwdBody :: forall m n f entries. (CheckpointMonad m, NonLocal n, LabelsPtr entries) => FwdPass m n f -> entries -> Body n -> FactBase f -> m (Body n, FactBase f)

[Label] is type compatible with entries. But Graph n C C and Body are not type compatible. It's not a problem with the code though, the test case also shows that the library works. The paper is mostly clear but it's just not exactly the same. Test case uses a different function

analyzeAndRewriteFwd :: forall m n f e x entries. (CheckpointMonad m, NonLocal n, LabelsPtr entries) => FwdPass m n f -> MaybeC e entries -> Graph n e x -> Fact e f -> m (Graph n e x, FactBase f, MaybeO x f)

Clean up the repository

Repository currently stores things that should not be there:

  • prototype versions of the library that have absolutely no practical use
  • source of the paper
  • response from reviewers (these should probably be never made public)
  • src directory stores some unused modules like Compiler.Hoopl.OldDataflow (perhaps some other as well?).

There are other files that might be good candidates for removal but need some more thinking before making such a decision:

  • various notes spread throughout source directories (src/LOOPS, src/Compiler/Hoopl/NOTES)
  • mkfiles - do we actually use them? If so, do they do anything that cannot be done using Cabal?

Upload tag v3.10.2.2 to Hackage

Hello! Could you please upload the 3.10.2.2 release to Hackage.

Its absence is causing problems with downstream tools. My specific motivating example:

$ stack list-dependencies
Didn't see hoopl-3.10.2.2 in your package indices.
Updating and trying again.
Selected mirror https://s3.amazonaws.com/hackage.fpcomplete.com/
Downloading timestamp
No updates to your package list were found
Updating package index Hackage (mirrored at https://s3.amazonaws.com/hackage.fpcomplete.com/) ...The following package identifiers were not found in your indices: hoopl-3.10.2.2
Possible candidates: hoopl-3.10.0.0, hoopl-3.10.0.1, hoopl-3.10.0.2, hoopl-3.10.1.0, hoopl-3.10.2.0, hoopl-3.10.2.1.

Thanks!

Hoopl collection classes could use enummapset?

I have found myself using https://hackage.haskell.org/package/enummapset a lot in my code. It provides wrappers around IntSet/IntMap that uses Enum rather than Int, which allows for better type safety.

Now I am doing more work with Hoopl in my project and it lacks certain useful functions from IntMap. I am looking at restrictKeys at the moment. I am trying to wrap my head around how to implement it, but I also realize that in some way this collection code in Hoopl is just the same thing as is also provided by enummapset. I was thinking that maybe it would better to migrate the code in Hoopl to make use of enummapset rather than rolling its own duplicate that tends to be out of date (I have added some missing functions before).

I wonder what the maintainer and people using Hoopl think about this?

Interleaving of multiple passes

There should be a combinator to take an arbitrary list of passes, and interleave them into a single pass where the fact type is a type-level list of other fact types:

interleavePasses :: [forall f. FwdPass m n f] -> FwdPass m n '[]

This seems quite Effective (pun most certainly intended).

Conditional jumps with fall-through not always ordered properly by postorder_dfs

When flattening the blocks to a final linear output order, postorder_dfs seems useful. One of my exit nodes is a CondJump, which has a destination and a fall-through label that is expected to be the next block in the flattened CFG.

This works most of the time, but there are cases where it fails. Consider the following sequence (an if-then-else) dispatching with a CondJump:

     CondJump L4 L6 conditional
L6:  ..
     Jump L3
L4:  ..
     Jump L3
L3:

There are two blocks L4 and L6 (the fall-through) which handles the true and false case of the if. This comes out as expected from postorder_dfs.

However, when the conditional expressions contains && and ||, I need to create a dispatch sequence, and it may be rewritten as:

     CondJump L6 L9 conditional'
L9:  CondJump L4 L6 conditional''
L6:  ..
     Jump L3
L4:  ..
     Jump L3
L3:

However, postorder_dfs comes up with the following:

     CondJump L6 L9 conditional'
L9:  CondJump L4 L6 conditional''
L4:  ..
     Jump L3
L6:  ..
     Jump L3
L3:

The fall-through label from the second CondJump does not follow immediately after!

The reason is that Hoopl has no idea about the concept of conditional jumps, postorder_dfs sees the first CondJump with [L6,L9] and start to process L6. This results in that the block acc gets L6 followed by blocks that comes after it, in this case L3 and whatever comes after that.

Now acc is [L6,L3,..], and then it looks at L9 with [L4,L6] as children. It starts with L4 and add whatever comes after (nothing new in this case), so we end up with [L4,L6,L3,..], then it gets to L6, but as it is already visited, so it is skipped.

Questions:

  1. Should postorder_dfs be able to handle this?
  2. Should Hoopl know about conditional jumps with fall-through? It does not seem to have such concept now.
  3. What is the correct way to flatten the CFG for final output?

Passes using multiple types of facts (aka snooping)

Imagine a pass which can use multiple types of facts - for example, a cache locality optimisation pass might want information about aliasing, branch selection frequency, and, optionally, the size of the code in a loop.

This might mean multiple types of fact, as well as the ability to OPTIONALLY use a fact type (i.e. if there is no provider of this type of fact, the pass is still valid, but probably not optimal).

It seems to me that the clearest way to handle this is with Effectful typing: each fact is represented with a separate effect type, and optional effects could be handled by a nondeterministic effect. Something like, e.g.:

myPassAnalysis :: (Member AliasFact r, Member BranchPredictionFact r, Member NonDetEff BlockSize r) => Eff r (Fact x f) -> n e x -> Fact x f

Decide on coding style

Hi all,

When reading through the library I often find that inconsistent indentation and coding style is making things more difficult to read and understand. A few examples:

  • IIRC I've seen code indented with anything between 1 to 4 spaces.
  • Do blocks often use braces/semicolon syntax instead of the more accepted (and IMHO far more readable) syntax based on indentation. Although both are used in various places.
  • Very long lines.
  • Trailing whitespace.

So I was wondering if we could simply agree to some already accepted coding style, e.g.,
https://github.com/tibbe/haskell-style-guide/blob/master/haskell-style.md

What do you think? (I'd be happy to send PRs to convert at least some of the code to the new style, e.g., Dataflow.hs which is non-trivial)

More advanced lattices for more precise analysis

So, let's say I have a lattice which I know happens to be a Heyting algebra - that is, a lattice with an implication operator (every finite, distributive lattice is an example of a Heyting algebra; so are intuitionistic type theories).

There must be some way in which we can use these more advanced properties! For example, a simple constant propagation analysis might produce this fact at an if-then-else node: (b → x = 1) ∧ (!b → x = 5)

Now, this same fact might be able to be derived from interleaving in a dataflow analysis, but it would probably be much harder to automatically take advantage of more complicated facts.

test build failure

Found on stackage nightly after re-enabling the test suite

Building test suite 'hoopl-test' for hoopl-3.10.2.2..
[1 of 1] Compiling Main             ( testing/Main.hs, dist/build/hoopl-test/hoopl-test-tmp/Main.o )

testing/Main.hs:8:1: error:
    Could not find module ‘Test’
    Use -v to see a list of the files searched for.
  |
8 | import qualified Test
  | ^^^^^^^^^^^^^^^^^^^^^

Will add to expected test failures under compile fail tag.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.