Parallel compilation of large BSV projects about bsc HOT 4 CLOSED

We have just gotten an external parallel build working using bluetcl and make. The working bits in the end were very simple.

First, we have a bluetcl script that generates a dependency graph of make rules thusly:

foreach i 
[depend make $topfile] {
  set tgt [lindex $i 0]
  set deps [join [lindex $i 1]]
  puts [append tgt ": " $deps]
}

Then the following rule can build any %bo in the design, along with any verilog modules that have been delineated using synthesis boundaries.

%.bo:
	bsc -verilog $(BSC_COMPILATION_FLAGS) -p $(BSC_PATH) $<

Given that this seems to be rather trivial, one would imagine that implementing this behaviour by default in the compiler would not be too difficult.

Having said that it is trivial, it should be noted that this took 2 of us most of a week to arrive at, and there are a large number of people using a much less efficient build (which was developed by the Bluespec company, no less!) so it would be much appreciated if this behaviour was integrated and enabled with a flag.

from bsc.

@jonwoodruff where can I checkout your external parallel build environment? I would like to use it for some of my projects.

from bsc.

@neelgala You can find the build we're using in:
https://github.com/CTSRD-CHERI/Toooba/tree/CHERI/builds/Resources

genDependencies.tcl is the one that asks the bluespec compiler to return a dependency graph structure, and then generates makefile rules. This one takes flags that should be passed to the Bluespec build in environment variables. There is a rule in Include_verilator.mk that invokes this script to generate the makefile include with these dependencies. The rules from Include_verilator.mk are below:

.depends.mk: $(REPO)/builds/Resources/genDependencies.tcl build_dir
	$(info generating bsv dependency graph)
	BSC_PATH=$(BSC_PATH) \
	BSC_DEFINES="$(BSC_DEFINES)" \
	BSC_BUILDDIR=build_dir \
	BSC_TOPFILE=$(REPO)/src_Testbench/Top/Top_HW_Side.bsv \
	OUTPUTFILE=$@ \
	$<

include .depends.mk

%.bo:
	$(info building $@)
	bsc -verilog -elab -bdir build_dir -vdir Verilog_RTL $(BSC_COMPILATION_FLAGS) -p $(BSC_PATH) $<

Sorry for the delay! We've been testing things before we merged it into our main branch.

from bsc.

Hi all,

It sounds like this is no longer a pressing issue for us, at the moment. The Bluetcl-generated makefile is doing the job, so I'll close the issue.

Of course, it still might be useful to have a parallel build feature inside the compiler itself, and I'm still interested hear anyone's feedback on that idea. So please feel free to reopen.

from bsc.