nexb / dependency-inspector Goto Github PK

• See also https://github.com/search?q=maven+dependency+tree&type=repositories
• Note that the mvn dependency:tree command would typically process all pom.xml in the codebase at once, and would generate one tree for each each pom.xml
• a good set of args to run the command would be : mvn dependency:tree -DoutputFile=maven-dependency-tree.lock -DoutputType=graphml and optionally with-Dmaven.test.skip=true and -DskipTests to skip tests, or various other options (such as scope) to filter the tree. See https://maven.apache.org/plugins/maven-dependency-plugin/tree-mojo.html
• See also https://github.com/jfrog/maven-dep-tree

Note that Maven now has a JSON format!

• apache/maven-dependency-plugin#325
• then apache/maven-dependency-plugin#391

After a discussion with the original author and designer of the Eclipse P2 dep resolver, both p2 and libsolv could be candidates for universal dep resolution.

There are many ways to get the resolved dependencies of a project. Some of these are:

1. Run the package manager(s) "install" command to fetch and install locally the dependencies, then scan them
2. Run the package manager(s) command to generate lockfiles then parse and collect
3. Collect the deps from parsing an existing lockfile or lockfile-like file
4. Run the package manager(s) command to resolve the dependencies, then parse the output or files generated from these commands
5. Run some tool to simulate the dependency resolution, then parse the output or files generated from these commands
6. Run a build and trace it with tracecode to collect fetched archives, and used files

Unless there is a locked, reproducible build process (backed by committed lockfiles as in 3.), all approaches are approximations of the dependency resolution. (Short of yet another approach with a binary, deployed code analysis)

Yet, 1,2,4, and 5 all require some extensive setup or guessing to setup a build environment as it is arbitrarily hard to reproduce the build environments reliably at scale.

Therefore, I propose a different approach here:

• Check if there are lockfiles available, optionally fail when not
• When there is no lockfile available, generate one by providing instructions and tools to run this IN THE CONTEXT OF THE BUILD, by and with the project team intimate understanding of this build
• If there is no such lockfile in the ecosystem (such as Maven), invent one from existing or new tools and promote conventions on naming, format and location
• Then always parse a pre-computed, lockfile
• Ensure that we merge lockfiles with their parent package manifest (in SCTK)

Separately, continue supporting dependency resolution simulation with all its problems with the various inspectors.

Some insights and concrete todos:

• #6
  • See also https://github.com/search?q=maven+dependency+tree&type=repositories
  • Note that the mvn dependency:tree command would typically process all pom.xml in the codebase at once, and would generate one tree for each each pom.xml
  • a good set of args to run the command would be : mvn dependency:tree -DoutputFile=maven-dependency-tree.lock -DoutputType=graphml and optionally with-Dmaven.test.skip=true and -DskipTests to skip tests, or various other options (such as scope) to filter the tree. See https://maven.apache.org/plugins/maven-dependency-plugin/tree-mojo.html
• Create Gradle gradle.lockfile using built-in features and gradle dependencies --write-locks
  • See https://docs.gradle.org/current/userguide/dependency_locking.html
  • https://github.com/GeekMasher/gradle-lock-dependency-submission-action
• Create Python frozen requirements file using pip freeze
  • this may need a specific convention as a requirements file is not your typical lockfile-only format
• Create .Net lock file with dotnet restore and other built in commands
• #3
• #4
• #5
• Create composer lock file with composer
• Create Ruby lock file with bundler
• Create go.sum lock file with go (though this should be standard by now)
• Create cargo/Rust lock file
• Create cocoapods lockfile
• Create SwiftPM lockfile
• Create Dart/pub lockfile