loehnertz / steinmetz Goto Github PK

What is the enhancement about?

The Walktrap graph clustering algorithm by Pons and Latapy seems to be useful as well.
It could be added in the future.

How important is the enhancement?

✨

Are there additional remarks?

N/A

What is the enhancement about?

Neo4j supports two other graph clustering algorithms that could easily be added to the application.

How important is the enhancement?

✨✨

Are there additional remarks?

It would be nice if the usage of the existing 'Louvain' algorithm and the two new ones could be abstracted in some way.

What is the enhancement about?

Currently, when uploading files to analyze for a new project, it might take a whole minute for large input files. The three main steps in the analysis could be run concurrently however.

How important is the enhancement?

✨

Are there additional remarks?

In all likelihood, Kotlin's coroutines would be the best bet here!

What is the enhancement about?

The fidelity of the dynamic analysis is highly volatile. Thus, a metric should be displayed in the front-end that represents the fidelity quality of it.

How important is the enhancement?

✨✨

Are there additional remarks?

In a perfect world, the amount of classes from the static analysis that were also part of the dynamic analysis should be equal. Consequently, the percentage of this amount could prospectively be used.

What is the enhancement about?

The models need to be modeled via the neo4j-ogm which is an ORM (respectively OGM) for Neo4j.

How important is the enhancement?

✨✨✨

Are there additional remarks?

The OGM library is build for Java. Here is an example for the models in Kotlin: https://gist.github.com/michael-simons/1433ad4969023d1a32b359f908f6ce2f

What is the enhancement about?

Currently, the edges of each relationship do not scale very well, meaning, that edges with a weight of 1 look pretty much exactly the same as ones with weight 3.

How important is the enhancement?

✨✨

Are there additional remarks?

Stephan suggested using a logarithmic increase in size.

Hi @loehnertz

Interesting project, I wanted to have a look at the running application ;)

I tried to run with docker-compose but it seems like you configured a dependency somewhere which cannot be found within the configured repositories:

[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  01:01 min
[INFO] Finished at: 2020-08-03T09:51:10Z
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project steinmetz: Could not resolve dependencies for project codes.jakob:steinmetz:jar:1.0-SNAPSHOT: Could not find artifact de.tudarmstadt.lt:chinese-whispers:jar:0.0.1-SNAPSHOT in jcenter (https://jcenter.bintray.com) -> [Help 1]
[ERROR] 

Best regards,
Stefan

What is the enhancement about?

Implement a basic API for the front-end to access the graph data inside of Neo4J.
This is useful so that the front-end does not directly has to talk to the database and so that the back-end can change the formatting or process the data in between.

How important is the enhancement?

✨✨✨

Are there additional remarks?

N/A

What is the enhancement about?

Logical coupling, so the historic coupling of classes within an OO-application can be analyzed via mining the VCS history.

How important is the enhancement?

✨✨✨

Are there additional remarks?

N/A

What is the enhancement about?

The semantic similarity between two class files can be analyzed via NLP.
The information gathered from this could be incorporated into the edge weighting as well!

How important is the enhancement?

✨✨✨

Are there additional remarks?

N/A

What is the enhancement about?

Currently, only the JFR is supported for dynamic program analysis.
A Java-Agent could be added as well for higher fidelity.

How important is the enhancement?

✨✨

Are there additional remarks?

N/A

What is the enhancement about?

The Chinese Whispers algorithm seems to be feasible as well as it works on weighted graphs.

How important is the enhancement?

✨

Are there additional remarks?

The de.tudarmstadt.lt.chinese-whispers Java project seems to be easy enough to integrate.

What is the enhancement about?

Currently, the edge weighting formula is very basic.
It has to be analyzed if and how it can be improved.
Additionally, it might be a good idea to come up with sensible defaults while still giving the user the option to change e.g. factors within the formula.

How important is the enhancement?

✨✨✨

Are there additional remarks?

One potential approach might be to look at the formula as a plain mathematical function and to e.g. use derivation to find it's local maximums in terms of the resulting metrics.

What is the enhancement about?

For actually implementing the recommended microservice candidates, the visual representation is subpar. Thus, a textual representation should be added below the rendered graph.

How important is the enhancement?

✨✨

Are there additional remarks?

N/A

What is the enhancement about?

Currently, the UI of the front-end is enormously ugly and it would be nice to improve its looks.

How important is the enhancement?

✨

Are there additional remarks?

Probably, Bulma as the CSS framework would be nice to use.

What is the enhancement about?

Currently, the interfaces from one clustered service to another one is not highlighted at all.
Therefore, it is hard to see where which those edges are.

How important is the enhancement?

✨✨

Are there additional remarks?

For now, the edges could just be colored differently.

What is the enhancement about?

Some graph clustering algorithms are run multiple times to optimize their outputs, those are in need of a clustering metric to optimize toward.
To the front-end, a select dropdown as well as some highlighting has to be added.
To the back-end, the selection of the best scoring run has to be added.

How important is the enhancement?

✨✨

Are there additional remarks?

N/A

What is the enhancement about?

The weights for the edge weighting formula can be chosen arbitrarily,
it would be nice however if sensible can be provided.
This could be solved by optimizing these parameters toward an evaluative metric.

How important is the enhancement?

✨✨

Are there additional remarks?

Probably, a genetic algorithm could work well here.

What is the enhancement about?

The idea is that the user can directly upload their application data through the front-end.
This would for example mean that they upload .jar file for JVM-based applications
as well as a Java Flight Recorder (JFR) file for now (i.e. featuring static as well as dynamic program analysis).

How important is the enhancement?

✨✨✨

Are there additional remarks?

N/A

What is the enhancement about?

Once the user uploaded a .jar file to the back-end, it has to be transformed into the format the Neo4j models currently have.

How important is the enhancement?

✨✨✨

Are there additional remarks?

N/A

What is the enhancement about?

The next important step after incorporating several graph clustering algorithms is to devise metrics that can be applied to the clusterings and eventually be rendered in the front-end.

How important is the enhancement?

✨✨✨

Are there additional remarks?

The metrics should just be send as an extra field in the HTTP response, next to the already existing Dynamic Analysis Quality.

What is the enhancement about?

After the graph can be build now in a basic manner, the next step should be to cluster it utilizing a suitable graph clustering algorithm.

How important is the enhancement?

✨✨✨

Are there additional remarks?

The algorithm that seems to be the most promising is the Markov Cluster Algorithm by Stijn van Dongen. More information and an implementation can be found at: https://micans.org/mcl/

What is the bug about?

In JVM languages, anonymous inner classes get compiled into extra files and will be therefore picked up by certain analysis tools.
They are inherently part of their parent class and should be therefore merged into the nodes/edge parents after assembling the baseGraph.

How to reproduce the bug?

Upload a new project and set a break point after the creation of the baseGraph in the GraphInserter.

How severe is the bug?

🐞🐞

Are there additional remarks?

N/A

What is the enhancement about?

Currently, the back-end completely ignores dynamic program analysis to build the unit graphs and instead only utilizes static analysis for that.
A very basic way of uploading dynamic program analysis archives to the back-end which get analyzed in a similar fashion as the static ones should be added soon.

How important is the enhancement?

✨✨✨

Are there additional remarks?

For now, only the JVM platform with .jfr files will be covered.

What is the bug about?

A live re-rendering mode was added to the front-end recently but has been deactivated owing to attached bugs.
Primarily the rendering of the service nodes doesn't work properly with the mode enabled.

How to reproduce the bug?

Do not flush the graph in the Graph component when the prop liveRerenderModeActive is true.

How severe is the bug?

🐞

Are there additional remarks?

N/A

What is the enhancement about?

The algorithms MCL and Walktrap both take tunable parameters that essentially decide how many clusters are being generated. Due to the fact that this is not known a priori and that this cannot be set equally with them, the algorithms could be executed multiple times.
Consequently, the result with the highest Total Coupling Score Modularity should be chosen as the final one.

How important is the enhancement?

✨✨

Are there additional remarks?

N/A

What is the enhancement about?

Now that a first API endpoint for retrieving graph data in JSON format is available,
this data can now be rendered in the front-end.

How important is the enhancement?

✨✨✨

Are there additional remarks?

Possibly, vis.js will be leveraged to render the graph.

What is the enhancement about?

As an additional clustering metric would be nice, the 'Modularization Quality' metric could be added.

How important is the enhancement?

✨✨✨

Are there additional remarks?

N/A

What is the bug about?

When uploading a new project, especially the JFR files can be really large in size.
The back-end just throws a weird exception in that case: java.io.IOException: Broken delimiter occurred.

How to reproduce the bug?

Upload a large file (tested with 3GB and upwards to the back-end) via multi-part.

How severe is the bug?

🐞🐞

Are there additional remarks?

N/A