Error enum values not propagated as exit codes for benchmark comparison in CI. about package-benchmark HOT 5 OPEN

Ah well I just saw that this has already an open issue upstream , if I correctly understand that this is the same problem.

from package-benchmark.

Yeah, it's the same issue. It's possible to work around by running the benchmark tool directly without swiftpm - running the benchmark with --debug you can see the actual command line required to drive the benchmark tool and instead run that in ci, the you should get the correct error code surfaced.

from package-benchmark.

Is this also reflected in the docs? I could not find anything of the sort and was going entirely by the 0/2/4 exit codes for swift package benchmark baseline check.

from package-benchmark.

Thanks for the report, unfortunately away for a couple of weeks and won't have time to check on this until then, sorry for delay.

from package-benchmark.

No worries! Also thank you for pointing out the workaround

from package-benchmark.