Code Monkey home page Code Monkey logo

mechanic's Introduction

Join the chat at https://gitter.im/fsprojects/Mechanic

Mechanic

Project idea

The F# compiler needs to get files passed in correct order for compilation, so that everything is declared before usage. There are some different opinions if this should be seen as a feature or as a limitation. This project won't settle this dispute, but it wants to make it easier for F# developers to maintain that order in fsproj files.

The idea is to create a dotnet core tool that uses the FSharp.Compiler.Service to find a valid file order. Since there may exist more than one valid order, the tool will write the order into the F# project file (the fsproj file). The user will then commit the fsproj into version control and build servers will use exactly the one committed version. In contrast to multi-pass compilers this will remain a step that is done explicitly by the programmer and there always needs to be at least one valid order. In this sense the tool and the fsproj are very similar to paket and the paket.lock file.

In addition to creating a standalone dotnet core CLI tool, we would probably also want to create a standalone library that could be integrated into editors. This would make integration a little simpler and could also take advantage of existing FSharp.Compiler.Service caches. In this case, we would probably want to take an exisiting FSharpChecker instance as a parameter.

Algorithm

This depends a lot on how much information we get from the FSharp.Compiler.Service. But the following properties should hold:

  • If the fsproj already contains a valid order then we don't touch it
  • If we need to change the order then we try to keep the original order as stable as possible.
  • If we there are multiple possible orders then we take the first one and exit
  • There may be heuristics needed to prefer certain parts of the search tree. We may try to make it opionated and force good practices
  • Files that have similar names should be glued together if possible.
  • Files in the same folder should be hold together if possible.
  • If we can't find a valid order we report an error text that is easy to understand for the programmer
  • If we can't find a valid order we may suggest how to fix the issue

Algorithm discussion at #10

Current algorithm documentation here.

Commands

In real world editing scenarios we will hardly ever need to reorder the whole file list. Usually programmers already have a working state and perform changes to it. These changes may introduce small ordering issues or even circular dependencies and the tool should help to resolve these. The following commands seem to be useful:

  • Reorder everything based on the fsproj
  • Reorder everything based on all files in the folder structure
  • Move file up as far as possible
  • Move file down as far as possible
  • Move file somewhere below file Y

Getting started

In order to test the tool you can clone the repo and run the following command:

 dotnet run -p src/Mechanic.CommandLine [path/to/a/projectfile.fsproj]

Advanced usage:

USAGE: [--help] [--pattern <string> <string>] [--dry-run] [--log-ast-tree] [--log-collected-symbols] [--log-file-dependencies]
          [--log-file-dependencies-with-symbols] [<string>]

<string>              Project file.

OPTIONS:

    --pattern <string> <string>
                        Alternative to project file - specify directory and wildcard pattern. Only print out resulting order.
    --dry-run             Don't update project file.
    --shuffle-test        Do extensive testing of the correctness of Mechanic on given project.
                          Tries varoius order of source files and check Mechanic result by compiler.
    --log-ast-tree        Print out AST tree for each source file from project.
    --log-collected-symbols
                        Print out collected symbols for each source file from project.
    --log-file-dependencies
                        Print out file dependencies in project.
    --log-file-dependencies-with-symbols
                        Print out file dependencies (with depended symbols) in project.

Tools

The following toolchain is implemented to get us started:

To build the project and run the tests, simply run one of the build scripts, depending on your environment; build.ps1, build.sh (with the --use-mono flag if under non-Windows) or build.cmd.

In the future, we might also want to use e.g.

  • Argu for command line parsing

Tasks

  • find a name (see #12)
  • extend this readme
  • split the work
  • setup project structure with CI and tests
  • write some code
  • choose a license
  • testing, testing, testing
  • integration into Ionide (see ionide-mechanic)
  • integration into VS

Maintainers

This project will be done mostly by newcomers to the F# OSS ecosystem. This is an explicit decision in order to get more people involved. Long time members of the F# OSS sphere will mentor and try to help if things get stuck.

  • @tlycken - Infrastructure/Build
  • @jindraivanek - Topological sorting
  • @donopj2 - Project file manipulation
  • @forki - Mentoring
  • @Krzysztof-Cieslak - Mentoring for FCS questions and Ionide integration

What about you?

If you are interested in joining this project, then welcome! We are really interested in bringing new people into the F# OSS ecosystem. Don't be shy! One way to get started is to take a look at the list of issues to see if there is anything there you feel like you'd like to contribute with. There's a couple of ongoing discussions on how to structure the library, and especially #10 has some details on where the code base might be heading.

mechanic's People

Contributors

chrsteinert avatar donopj2 avatar dpraimeyuu avatar fbehrens avatar forki avatar jichang avatar jindraivanek avatar matthid avatar nosami avatar reallinfo avatar tforkmann avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

mechanic's Issues

A project needs a name

While things like "FSharp.Project.FileOrderer" are technically correct I personally prefer things that are puns or slightly related terms im foreign languages or just completely unrelated words.

So let's do some bikeshedding here.

I would like to join the project

Hello folks,

I've been a long-time C# and (until about a year ago) Dynamics NAV programmer who has been exploring F# for a couple of years on my own time. I'm really eager to work on a real project in F# (particularly OSS since I've never done that before either). This would fit in great with the #100DaysOfCode challenge I started last week (my first week has been devoted to learning FSharp.Data and FSharp.Charting).

I would say my F# skills are at the high end of beginner. However, I have little to no experience with the tooling, outside of Visual Studio and FSI. I suspect that part of the point of this project is to help people like me improve in this area, while also helping the community out.

So if you'll have me, I'd love to join the team.

Thanks,
Jason

Mechanic failed to resolve file order on Fulma-demo

I am working on the VSCode extension and tried to run Mechanic against Fulma-demo and it's failing to resolve the file order.

It's resolve an empty list: TopologicalOrder []

I tried to run it at the root level or from the src folder the result is the same.

Library Approach

Hey guys, I'm definitely keen to jump in. This is my first foray into OSS, so apologies for any faux pas. I had a look at the project setup that @tlycken put together, and have been giving some thought to the algorithms and task list @forki laid out. I've also been playing around with FSC and reading through the docs, but its new to me (eager to learn though!). I guess my question is, what is the best approach to start? A few thoughts:

  • Having looked through the FSC docs, it appears most of the API's require passing different files (.fs, .fsproj, etc), depending on the service. I was thinking the Project Analysis service could be quite useful. It seems to have rich functionality for compiling a full project and exploring errors and symbols.
  • To that end it appears we will need a set of utilities like the Utils.fs file in the Paket project to walk the source code directory and build data structures for each project and its contents. This might be a good module to start on?
  • As a starting point for the algorithm outline a FSharpCheckProjectResults with an empty Errors array would indicate a correctly ordered project and thus no action. Easy enough ;)

Naturally determining an incorrectly ordered project and finding a correct order will be a bit more involved. I'll continue to explore the FSC APIs and try to devise an approach while working on a files module. Perhaps we could also post thoughts in this thread as they come together that will hopefully serve to flesh out the requirements more broadly?

If this all sounds reasonable I'll crack on, but please do let me know if I'm off base.

Pattern don't seems to work

When running dotnet mech.dll with a pattern I got a no matches found: *.fs

If I remove the pattern then it's working. Is the pattern args supposed to work ? I am asking because, I got zsh: in front of my CLI so I am not sure if this is my bash being broken or mechaninc.

➜  netcoreapp2.0 git:(36460d3) dotnet mech.dll ~/Workspace/Github/MangelMaxime/fulma-demo/src/Demo.fsproj *.fs
zsh: no matches found: *.fs
➜  netcoreapp2.0 git:(36460d3) dotnet mech.dll ~/Workspace/Github/MangelMaxime/fulma-demo/src/Demo.fsproj
TopologicalOrder
  ["/Users/mangelmaxime/Workspace/Github/MangelMaxime/fulma-demo/src/Import/Fable.Import.Showdown.fs";
   "/Users/mangelmaxime/Workspace/Github/MangelMaxime/fulma-demo/src/Import/Fable.Import.Lowdb.fs";
   "/Users/mangelmaxime/Workspace/Github/MangelMaxime/fulma-demo/src/Global/Render.fs";
   "/Users/mangelmaxime/Workspace/Github/MangelMaxime/fulma-demo/src/Global/Logger.fs";
   "/Users/mangelmaxime/Workspace/Github/MangelMaxime/fulma-demo/src/Global/Router.fs";
   "/Users/mangelmaxime/Workspace/Github/MangelMaxime/fulma-demo/src/Global/Database.fs";
   "/Users/mangelmaxime/Workspace/Github/MangelMaxime/fulma-demo/src/Question/Index/Types.fs";
   "/Users/mangelmaxime/Workspace/Github/MangelMaxime/fulma-demo/src/Question/Index/Rest.fs";
   "/Users/mangelmaxime/Workspace/Github/MangelMaxime/fulma-demo/src/Question/Index/State.fs";
   "/Users/mangelmaxime/Workspace/Github/MangelMaxime/fulma-demo/src/Question/Index/View.fs";
   "/Users/mangelmaxime/Workspace/Github/MangelMaxime/fulma-demo/src/Question/Show/Answer/Types.fs";
   "/Users/mangelmaxime/Workspace/Github/MangelMaxime/fulma-demo/src/Question/Show/Answer/Rest.fs";
   "/Users/mangelmaxime/Workspace/Github/MangelMaxime/fulma-demo/src/Question/Show/Answer/State.fs";
   "/Users/mangelmaxime/Workspace/Github/MangelMaxime/fulma-demo/src/Question/Show/Answer/View.fs";
   "/Users/mangelmaxime/Workspace/Github/MangelMaxime/fulma-demo/src/Question/Show/Types.fs";
   "/Users/mangelmaxime/Workspace/Github/MangelMaxime/fulma-demo/src/Question/Show/Rest.fs";
   "/Users/mangelmaxime/Workspace/Github/MangelMaxime/fulma-demo/src/Question/Show/State.fs";
   "/Users/mangelmaxime/Workspace/Github/MangelMaxime/fulma-demo/src/Question/Show/View.fs";
   "/Users/mangelmaxime/Workspace/Github/MangelMaxime/fulma-demo/src/Question/Create/Types.fs";
   "/Users/mangelmaxime/Workspace/Github/MangelMaxime/fulma-demo/src/Question/Create/Rest.fs";
   "/Users/mangelmaxime/Workspace/Github/MangelMaxime/fulma-demo/src/Question/Create/State.fs";
   "/Users/mangelmaxime/Workspace/Github/MangelMaxime/fulma-demo/src/Question/Create/View.fs";
   "/Users/mangelmaxime/Workspace/Github/MangelMaxime/fulma-demo/src/Question/Dispatcher/Types.fs";
   "/Users/mangelmaxime/Workspace/Github/MangelMaxime/fulma-demo/src/Question/Dispatcher/State.fs";
   "/Users/mangelmaxime/Workspace/Github/MangelMaxime/fulma-demo/src/Question/Dispatcher/View.fs";
   "/Users/mangelmaxime/Workspace/Github/MangelMaxime/fulma-demo/src/Types.fs";
   "/Users/mangelmaxime/Workspace/Github/MangelMaxime/fulma-demo/src/State.fs";
   "/Users/mangelmaxime/Workspace/Github/MangelMaxime/fulma-demo/src/App.fs"]

Count me in!

@forki

I'd like to join, collaborate and participate as well. I may not be quite visible or known in F# community, but basically I just want to get my feet wet quickly for this tool 🙂

New logo for Mechanic

I wanted to contribute to Mechanic. I designed a logo for Mechanic. If you like it, I'll send you the files.

mechanic

Logo

Now that we have a name, we need a logo! :)

(On a more serious note, this might be a little premature, but it's also a good way for all you graphically skilled people to become involved, even if you don't want to contribute code just yet!)

Give us your best!

What are the specific problems that exist that this project is attempting to solve?

I can imagine some cases which could lead to an invalid file ordering for an fsproj file but I'm curious if I'm understanding the main motivation for attempting to a brute force method for finding a valid project ordering from an existing project.

Here are the cases I can dream up:

  1. As a developer using and IDE that does not support file re-ordering I add a new file which adds the new file in an incorrect order.
  2. As a developer who has made a change a .fs file to depend on a file that comes later in complication order according to the current fsproj file I need to move that dependency so that it precedes the file that now depends on that one being moved.
  3. As a developer who has been working with an F# compiler tool chain that does not us fsproj but instead some other way of feeding the files in with a specific order I want to generate an fsproj file or correct the implied ordering created by some tool which may have generated the fsproj file.
  4. As a developer who would like to take advantage of the new ability for .NET project files to not have to explicitly declare individual file references but need to satisfy the need for explicit file ordering for F# projects and would like some automatic tooling to make this less painful.

From my perspective it would seem to me that cases 1 and 2 would be the most common and would be facilitated through support for just simple explicit position translation commands either exposed through the library to the IDE or through the CLI.

The case for 4, if it even exists in the wild, seems far less regular as it should be inherently a onetime activity per project at most and may only happen in cases where existing projects are migrated to fsproj or from an older version to a new version.

As for case 5 it's not clear to me how or if this project would solve this issue as it sounds to me that the proposal is to still modify the existing fsproj file to include explicit file references to dictate compilation order. I personally have noticed the benefits of the single-pass style compilation in F# and I hear from many others who feel the same way so it's not clear to me how or if this would be a goal but I wanted to list it for completeness.

So as one may be able to see is that from the cases listed above it's hard for me to understand the significance of the value for determining a valid file compilation order from an arbitrary set of F# files or an fsproj in comparison with some of the other stated goals for providing a consistent CLI and library interface for modifying file order for fsproj files. Likewise the former seems at a glance much more complex and resource intensive without as much bang for the buck in comparison to the later. I'm hopeful that somebody can set me straight though and clarify some points I may have gotten wrong or gaps in my general understanding and perhaps this can help formulate some material that can help guide the improvement of the documentation for this project to state the goals and the problems that it addresses.

Failing test on AppVeyor

Project file tests/Project file is loaded and created from disk failed in 00:00:00.2030000. 
File contents are correct.
          Expected string to equal:
          "<?xml version="1.0" encoding="utf-8"?>
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>netstandard2.0</TargetFramework>
</PropertyGroup>
<ItemGroup>
<Compile Include="File1.fs" />
<Compile Include="File2.fs" />
<Compile Include="File3.fs" />
</ItemGroup>
</Project>"
                                                 �
          The string differs at index 38.
          "<?xml version="1.0" encoding="utf-8"?><Project Sdk="Microsoft.NET.Sdk"><PropertyGroup><TargetFramework>netstandard2.0</TargetFramework></PropertyGroup><ItemGroup><Compile Include="File1.fs" /><Compile Include="File2.fs" /><Compile Include="File3.fs" /></ItemGroup></Project>"
                                                 �
          String does not match at position 38. Expected char: '\010', but got '<'.
  C:\projects\mechanic\src\Mechanic.Tests\Files.fs(42,1): [email protected](AssertException exn)

Support moving a file relative to it's current position within fsproj file

Concerning the suggested commands that would be supported, in the one in particular that states Move file somewhere below file Y I think it would be useful to consider the default value for Y to be the original file and thus the relative to it's current position.

Moving Constants.fs up one relative to itself:

> forklift up 1 Constants.fs

In this example I used my suggested project name forklift but it could easily be substituted with mech or whichever name is decided upon and the same goes for the up 1 syntax that I just threw out there as an example to work with.

Result:

The effect would be that of moving the file reference for Contants.fs in the following project file up one spot.

<?xml version="1.0" encoding="utf-8"?>
<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <TargetFramework>netcoreapp2.0</TargetFramework>
  </PropertyGroup>
  <ItemGroup>
+    <Compile Include="Constants.fs" />
    <Compile Include="UsesConstants.fs" />
-    <Compile Include="Constants.fs" />
    <Compile Include="Program.fs" />
  </ItemGroup>
</Project>

If the general approach is reasonable then perhaps the overall brief description for this command could instead something like Move file X somewhere below file Y where Y is optional and is the same as X by default.

How would the above up sub command work with X and Y both supplied? Proposing syntax isn't really the focus of this issue but just to demonstrate that a single sub command could cover both cases of relative translation perhaps one of the examples below would make sense.

Moving Constants.fs up one relative to the position of UsesConstants.fs

Example 1:

> forklift up 1 Constants.fs UsesConstants.fs

Example 2:

> forklift up 1 Constants.fs --from UsesConstants.fs

I'd like to help as well

Hello guys, I'd like to help, the project looks amazing! But how to start? Looks like there are lot of things to do, but don't know with what I can start.

Fix Travis build

The builds on Travis-CI are currently failing, because the build script doesn't work if dotnet isn't installed (see discussion in #16 for details).

There are two potential fixes:

  • List dotnet as a prerequisite, remove everything pertaining to installing dotnet from build.fsx and update .travis.yml to make sure that dotnet is installed prior to running the build script.

or

  • Fix the build script parts that check the dotnet version so it can install dotnet correctly.

Unit Test Library

Shall we have a discussion on unit testing libraries? @forki mentioned Expecto in the readme. I haven't used it but I'd be keen it to give it a try. Thoughts?

Doesn't build on Windows

Count me in! Happy to help out with this.

I cloned the project in Ubuntu and ran ./src/build.sh --use-mono. Everything built fine,

Cloned in Windows, ran .\src\build.cmd and got several errors. It's not obvious to me what the problem is.

I can post the output here if people want to see it.

Publish as dotnet cli tool

It be nice if Mechanic was available as clitool.
Running dotnet mechanic would format the fsproj in the current folder.
Or show a help page if no (single) fsproj is found in the current directory.
This way other editors than vscode could easily integrate Mechanic.

Me too!

I would love to help out with this.

Scaffold and build system

Hi @tlycken,
I want to continue up our short discussion in #20 about this.

When I suggested to copy this things from established example projects, I agree to you, that we should only copy those parts that we need and understand.

For my own learning understanding of the dotnet-core platform, I have created a minimal scaffold from scratch: https://github.com/fbehrens/Dotnetcorescaffold .
Important for me was that I get familiar with the native dotnet commands.
This went very smooth. (big thanks to @forki and @Krzysztof-Cieslak for paket and inonide)

In the readme are the steps of creation documented.
Its completely wired up with

  • 3 Projects (Library, Cli and Tests )
  • uses Paket for dependencies (Fuchu,Expecto)
  • has ci (Travis and Appveyor)
  • and borrows its structure and build system from the safe-bookstore.

I did this because I needed to understand how all this works.

Here are some differences to what is currently here:

  • no need for --use-mono argument
  • build scripts can invoke fake targets (like ./build.sh Clean)
  • build scripts (fsx,cmd,sh) are simpler
  • the git-version stuff is missing
  • uses Expecto instead of xUnit
  • it builds much faster (don't know why, but on my Mac its 17s against 37s for a full build)

Please have a look at it. What do you think about it?
If you like it, I would make a pull request for switching to it.

Count me in!

I saw a link here posted on Twitter, and of course I want to help!

The readme has a number of suggestions for tooling - how fixed is that list? I think it would make sense to do as much as possible in the dotnet cli realm, and not mix with eg paket. If there are no objections, I could start setting up a CI pipeline on eg Appveyor and/or Travis to get things started.

First milestone

This is list of issues that we need to do to have first working version:

Project editing:

  • Decide what tool to use #32
  • Loading source files #61
  • Saving source files order #62

Ordering alg:

  • Inner opens #59
  • Defintions of other symbols #60

This is also good place to discuss heading of this project.

Failing test on linux (travis)

Project file tests/Paths and filename are combined correctly failed in 00:00:00.1310000. 
Paths combined correctly.
          Expected string to equal:
          "C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETCore\v4.5.1\TestLib.dll"
           ↑
          The string differs at index 0.
          "\Reference Assemblies\Microsoft\Framework\.NETCore\v4.5.1/TestLib.dll"
           ↑
          String does not match at position 0. Expected char: 'C', but got '\\'.
  /home/travis/build/fsprojects/Mechanic/src/Mechanic.Tests/Environment.fs(16,1): [email protected](AssertException exn)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.