other possibilities are to render to .svg, and basically wrap/ call graphviz cli

love your work on moon 💖

from moon.

RFC: moon graph

I took some time to research how we can tackle this and I am writing it down here.

Some notes

• When I say dep graph, I mean the graph of dependencies installed via package manager (npm, pnpm, yarn etc). astro, next etc will be part of the dep graph.
• When I say project graph, I mean the relationships between different repositories that are present in the workspace. The browser package, vue-vite-app etc will be part of the project graph*.

*In the context of the example repo.

Graphing libraries

Some popular graphing libraries that can be used to render the graph on the frontend.

• Cytoscope: Used by NX
• Sigma.js: Promising because it can render a lot of nodes which might be needed by the dep graph
• React Flow: Looks good
• Webcola: The Edgedb UI is built with this and it looks pretty good

other possibilities are to render to .svg, and basically wrap/ call graphviz cli

I am against this because it would be difficult to make the graph interactive, or allow us to extend behavior or add custom styling/interactions.

Problems with using rust

There is also the possibility of using a rust only solution by leveraging solutions such as yew charts. However, they are very immature and will likely result in a lot of developer pain since we might have to implement a graphing library in rust from scratch.

Serving the graph

There could be many possible solutions:

• A separate SPA that will be bundled with the Moon CLI.

  When we run moon graph, we start up an axum/rocket/warp server that will serve these bundled files. The server will have rest endpoints/graphql resolvers from where data can be fetched and displayed (using on mount if we choose svelte for the SPA). A --watch option (like the one NX has) can be easily added by making the frontend establish a websocket connection with the backend and subscribe to changes. The backend can set up a file watcher to detect changes to the dependency graph on the fly. This will come at the cost of not being SEO friendly (but is that really a problem in this context?)

• A yew app that will be bundled with the Moon CLI.

  This will have the same problems as described above.

• A separate process is started.

  We can have the moon graph command start a next-js/remix/nuxt-js server that can handle all the calculation/rendering of the graph. This will have some problems:

  • We will have to interop between rust and js code, or have to rewrite parts of graph calculation in js.
  • Spawning a long running process (such as a server) within rust is non-trivial

Conclusion

I have left the minute details out of this RFC because we can refine/iterate on them later. That being said, I would love to get some feedback on the above.

from moon.

@IgnisDa Great writeup. There's another piece to this that is not mentioned, and that's the VS Code extension (and future editor extensions). The graph visualizer will also be present in the extension as its own view, so this means we need to be able to share the code between the CLI and the extension.

The biggest question like you said, is how to serve it. The graph will be built in JS of course, but once we support non-JS languages, we can't assume everyone will be using Node.js, so possibly serving with Rust is the best course?

from moon.

@milesj I am not sure how vscode extension packaging works, but lets say we write the graph visualizer in svelte (I have heard it has great integration with vscode extensions). We could simply build the SPA with svelte and then bundle it within the CLI. And we can also import svelte components into the vscode project and use them to render in a webview environment.

so possibly serving with Rust is the best course?

Agreed.

from moon.

Ok my thoughts.

The visualizer would need to work from both the CLI and from the VS Code extension. Code must be shared between the two.

Writing the JS

While svelte is great, I feel like this can simply be built in vanilla JS. Most of the interactivity will be in the graph itself, which will be abstracted away from us based on the lib we choose. Anything else would be simple DOM updates.

Serving the HTML/JS

For the extension, it relies on web views (https://code.visualstudio.com/api/extension-guides/webview) where raw HTML is simply rendered. The visualizer JS will most likely be included via a script tag (not bundled or anything), something like:

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Graph visualizer</title>
    <script src="https://unpkg.com/@moonrepo/graph-visualizer@latest"></script>
</head>
<body>
    <script>window.GRAPH_DATA = {}</script>
    <div id="graph" />
</body>
</html>

For the script src, we can use a CDN system like https://unpkg.com that points directly to the visualizer JS code. This allows us to:

• Not bundle the JS code in @monorepo/cli or the VS Code extension.
• We can continually publish updates to the visualizer without having to also update downstream consumers. Just use @latest for the version.
• Can be built in isolation.

For the Rust code, we can probably use the same exact approach. moon project-graph would spin up a local HTTP server and serve the same HTML above. That's it. Doesn't need to get more complicated than that.

Visualizer requirements

All of those graphing libs you listed above are great, but I want to also include more information and functionality besides the graphs. For example:

Project graph

• When clicking on a project (node in the graph), it will also display all relevant information from the project's moon.yml. Primarily tasks and project metadata.
• Graph would need advanced filtering, like, "only show applications", "only show javascript projects", so on and so forth.
• Being able to only focus a project and see its dependencies and dependents.
• Maybe some kind of grouping like: https://ialab.it.monash.edu/webcola/examples/smallgroups.html

Dependency graph

• When clicking on a target (a node), it displays information on the task and owning project.
• Filters also?

from moon.

The CDN route sounds really good! But for functionalities like filtering, serching and grouping, i think it would be best to use some kind of UI framework. That would also ensure that we are able to add more advanced features later down the line without any major overhaul.

from moon.

Closing this since the initial work is done. Thanks a lot for it!

from moon.