[Feature Request] Dependency graph introspection about packwerk HOT 3 OPEN

Hey @Epigene we actually do a version of this in our CI suite using ParsePackwerk https://github.com/rubyatscale/parse_packwerk

There are two important things to note here:

1. You'd need to include violations in the graph traversal to more accurately predict the blast radius of a change
2. There are likely ways that one package impacts another through a mechanism that packwerk cannot pick up, so you may just want to make sure you have a mechanism to manage this (e.g. always running all tests on main, even if you don't run them all on feature branches).

I think to do this, we just need to do a breadth first search of a package's ancestors. Our implementation actually doesn't yet look at violations Here's our implementation:

require 'parse_packwerk'

class AffectedPackagesHelper
  def initialize(modified_packs)
    @packs = modified_packs
    @memo = {}
  end

  def dependent_packs
    @packs.each do |pack|
      inbound_dependencies_traversal(pack)
    end
    return @memo.keys
  end

  # Using BFS traversal, finds list of explicitly dependent packages defined
  # by each pack's `package.yml` and adds it to the memoized traversed packages
  # array @memo. 
  #
  # @param String the name of a pack in `packs/some_pack` format
  def inbound_dependencies_traversal(package_name)
    return if @memo.key?(package_name)

    queue = [package_name]

    while !queue.empty?
      curr_package = queue.pop
      # Our implementation doesn't yet look at violations, but it should look at 
      # ParsePackwerk.all.select { |package| ParsePackwerk::DeprecatedReferences.for(package).violations.any?{|v| v.to_package_name == curr_package }
      inbound_dependencies = @memo[curr_package] ||
                     ParsePackwerk.all.select { |package| package.dependencies.include?(curr_package) }.map(&:name)
      if !@memo.key?(curr_package)
        @memo[curr_package] = inbound_dependencies
      end

      inbound_dependencies.each do |dependency|
        if !@memo.key?(dependency)
          queue << dependency
        end
      end
    end
  end
end

Want to let me know if this technique works for you locally? I'd love to be able to support more CI tools like this in the packwerk ecosystem and want to work out the kinks and figure out what/where the API should live.

from packwerk.

@alexevanczuk Thanks for the quick reply, I'll give parse_packwerk a look.
I agree with both points you make, about needing to include violations (contents of pack deprecated_references.yml) in traversal and having a final-safety CI pass on everything once in a while.

from packwerk.

Awesome, let me know what you come up with @Epigene ! I love this effort because allowing packwerk to open doors to conditional builds (as we refer to them) for fast (and thus cheaper) builds is a huge opportunity. Long-term, we hope that the same concept can allow for conditional deployments – that is, deploying a package separately once we've ascertained programmatically that it can stand in its own application (i.e. all dependencies fully specified) and all consumers are talking with it via a network-boundary-agnostic API (this is one of many reasons why we feel privacy enforcement is so important).

from packwerk.