Codecov is not in sync and shows inaccurate coverage about airflow HOT 4 CLOSED

To be honest, the number of times it broke in the past (and often not our fault) was so big that I kinda stopped paying attention. @Taragolis had made the last attempt to fix it and since then the only change was to make it works with new codecov plugin (which required adding a key) - because of nodejs warning we got.

And codecov has a history of having some quirks in the process that make the coverage works (or does not) somewhat mysteriously.

There is a good reason why calculation on PRs can never be good - because we are running selective checks to save build and feedback time, vast majority of the PRs will simply run a subset of tests relevant to the change.

That's why the only relevant measure of coverage should be taken only from main (i.e. canary builds) and I believe it's been like that last time @Taragolis fixed it. But coverage from PR will never work well - that's why we disabled it.

Not sure what happened here - maybe some issue where switching to the new codecov plugin, but to be honest almost no-one consistently looks at those metrics for it to be high priority :). But if someone would like to take a look and figure out what's wrong, that would be great.

from airflow.

Gotcha. I agree that PR calculations are not ideal for coverage calculation. In the issue's description, I referred only to commits in main when talking about the actual output on Codecov.

I'd be happy to try tackling it later on - I dealt with installing both Codecov and SonarQube in my workplace, so there's a chance I'll figure it out. However, I'll need to check how to integrate it with my forked repo, and later have permissions to work on Airflow's Codecov account.

from airflow.

This is one of the difficulties there, that it's notoriously difficult to test :D

from airflow.

Should be fixed now (hopefully) it was bad token .

from airflow.