No comments on pull requests from coveralls in emqx/emqx about github-action HOT 2 CLOSED

Hi @afinetooth.

Thanks a lot for the detailed explanation, very helpful! I'll troubleshoot the problem on our end, now that I know where to look.

from github-action.

Hi @id.

The explanation of this issue is complicated, and somewhat confusing due to a lot of seemingly related but ultimately irrelevant datapoints.

But the fix is simple, so let's start there and, if you want any further background, we can get into that.

Missing base build -> No PR Comment

Basically, in order to send a PR Comment, Coveralls needs to be able to find a PR build's base build---which is to say a coverage report for its base build---to which it compares the current build to derive a report showing the change in coverage between the two relevant commits.

That change in coverage is the key content of the PR Comment and when Coveralls can't find the base build, it can't derive the contents of the PR Comment and, as a result, it won't send one.

We are working on improvements to the UX of this scenario, which include a PR Comment with an exception message and a link to troubleshooting steps, but for the time being, your clue that this is your issue is that your PR build will read "** FIRST BUILD**" in the header of its build info table.

For example:
https://coveralls.io/builds/67917577

Problem

While there appear to be a few issues with your builds, leading to misdirection, like the fact that the branch name is missing from your push builds:

Which points to a "git hash" missing from your coverage reports that Coveralls integrations expect to pick up from the CI environment, but since your coverage reports still contain the minimum required git data---commit_sha:

This isn't your issue because the presence of commit_sha allows Coveralls to find your PR builds' base builds.

Your problem is that your base builds were never completed.

In other words, your PR builds' base builds are in a state known as "pending completion" and, as a result, they do not appear in your official Build History and cannot be used for coverage calculations or comparisons.

Here is an example using the same PR build from above:
https://coveralls.io/builds/67917577

Here is the PR info we have from GitHub for that build:

PR Info:

{
  "title": "ci: OTP 26 (26.2.5-1) for docker images",
  "head": {
    "sha": "c41be7ee22b310e46786116e20088b84bb528a3e",
    "label": "zmstone:0605-otp26-for-docker-images",
    "ref": "0605-otp26-for-docker-images"
  },
  "base": {
    "sha": "ae0379f974a25f46ee7ba81d44c3dde68f18956e",
    "label": "emqx:release-57",
    "ref": "release-57"
  }
}

Where "base"["sha"] is the commit associated with your base build, and the one Coveralls is looking for, for comparison.

And that build exists:

Build.find(67917577).repo
=> #<GithubRepo id: 229018, name: "emqx/emqx" [...]

repo.builds.where(commit_sha: "ae0379f974a25f46ee7ba81d44c3dde68f18956e").first
=> #<Build id: 67904199, repo_id: 229018, [...] service_name: "github", [...] service_number: "9383783553", service_event_type: "push", [...] state: "pending" [...]

And here's the Coveralls build page for that build:
https://coveralls.io/builds/67904199

The problem is that it's pending completion---it's not finished / not completed, and its coverage was never calculated:

I'm not sure why this is, but the evidence points to these builds never fully uploading all of their expected parallel uploads (in your case 110 uploads), and, more to the point, never sending Coveralls the "close parallel build" signal, which means no steps in your Ci config ever called the (close) parallel build webhook.

Using this example, here's the PR build with 109 uploads (per my Admin info), which was completed:
https://coveralls.io/builds/67917577

And here's the base build, with only 105 uploads, and never completed:
https://coveralls.io/builds/67904199

To back that up, here's another example:

PR build with 109 uploads, completed:
https://coveralls.io/builds/67881752

PR Info:

{
  "title": "fix(resource manager): force kill process if stuck when stopping/removing",
  "head": {
    "sha": "f17aefe3d7746da91c917907536a7f6ae2b52317",
    "label": "thalesmg:force-stop-connector-r57-20240604",
    "ref": "force-stop-connector-r57-20240604"
  },
  "base": {
    "sha": "cda6d5f63636199932d9c1015d690b011c46e243",
    "label": "emqx:release-57",
    "ref": "release-57"
  }
}

Base build with 105 uploads, never completed:
https://coveralls.io/builds/67881435

Fix

I assume that if you can figure out what's happening with those CI builds (from their build logs) in terms of why the uploads are stopping at 105, and why the (close) parallel build signal is never being sent, and fix that, you'll fix this problem.

If you have trouble figuring that out, please share the relevant portions of your CI build log(s), and, even better, if you can invoke debug or verbose mode that will help us both diagnose what's happening more easily. If you're using the Coveralls GitHub Action, just use the debug input option:

    - name: Coveralls Parallel Upload Step
      uses: coverallsapp/github-action@v2
      with:
        flag-name: whatever
        parallel: true
        debug: true

And:

    - name: Coveralls Close Parallel Build Step
      uses: coverallsapp/github-action@v2
      with:
        parallel-finished: true
        debug: true

from github-action.