codecov / codecov-circleci-orb Goto Github PK

View Code? Open in Web Editor NEW

20.0 12.0 40.0 533 KB

License: MIT License

Python 75.00% TypeScript 25.00%

codecov-circleci-orb's Introduction

codecov-circleci-orb

codecov-circleci-orb's People

Contributors

Stargazers

Watchers

codecov-circleci-orb's Issues

Automatic Publishing not enabled. Updates are not being published.

It appears changes being merged here are not reflecting on CircleCI.com
https://circleci.com/developer/orbs/orb/codecov/codecov
Version Published: November 25, 2020

There appears to be multiple issues with the config file al the moment causing the deployments to fail: https://app.circleci.com/pipelines/github/codecov/codecov-circleci-orb

Type error for argument publish-token-variable: expected type: env_var_name, actual value: "$ORB_PUBLISH_TOKEN" (type string)

Error calling job: 'orb-tools/publish-dev'

Error calling workflow: 'lint_pack-validate_publish-dev'

Recommendation:
Update the configuration file to match the latest deployment process:
https://github.com/CircleCI-Public/Orb-Project-Template

As described here: https://circleci.com/docs/2.0/orb-author/

Issue upgrading to v3.1.1

We are currently using v1.0.2 of the orb without any issues, but I'm trying to upgrade to 3.1.1 to use the new codecov uploader. My circleci builds are failing because codecov returns an exit code of 4. The weird thing is the upload seems to finish okay. Here is a screenshot:

I'm not sure what is generating the exit code of 4 and how to fix it.

Allow specifiying other commandline args

It would be convinient if it were possible to pass arbitrary extra arguments to the command the orb wraps, perhaps via an extra_args parameter?

We keep seeing slow uploads of coverage data, and I'd like to add some curl timeouts to speed up retries - but this isn't currently exposed by the orb.

Checksum Bash Uploader - Security Update

Following today's security announcement from Codecov: https://about.codecov.io/security-update/ should this orb verify the bash uploader following https://docs.codecov.io/docs/about-the-codecov-bash-uploader#validating-the-bash-script?

1.1.5 requires fairly new perl for new shasum utility

The 1.1.5 orb requires perl >5.28 in order to get the --ignore-missing flag of --shasum. This version of perl is not yet supported in Debian Buster, and thus not supported in docker base images that build off buster. This is preventing adoption of the latest version, requiring us to pin to 1.1.4.

File download errors are hidden

The orb performs file downloads via curl with the -s flag which silences error messages by default. This makes troubleshooting difficult as the only output from the orb is Codecov upload failed as described here (support discussion).

Ideally the curl operation would be performed using the --show-error flag to ensure download errors are reported to the end user.

Orb doesn't work in Alpine images

The orb doesn't work if it's run inside an Alpine Linux Docker image. CircleCI is running the script in the orb's run step using /bin/sh (which in Alpine is the "traditional" version of sh, not bash), and the script is using bash-specific syntax and failing:

====>> Upload Coverage Results
  #!/bin/sh -eo pipefail
bash <(curl -s https://codecov.io/bash) \
  -f coverages \
  -n ${CIRCLE_BUILD_NUM} \
  -t ${CODECOV_TOKEN} \
  -y .codecov.yml \
  -F  \
  

/bin/sh: syntax error: unexpected "("
Error: Exited with code 2
Step failed
Error: runner failed

Make the `file:` parameter optional, or support multiple files.

I've used the codecov-bash for quite a while in my CI jobs like follows:

bash <(curl -s https://codecov.io/bash)

I love that CodeCov had created a CircleCI orb, but I'm not happy with the required file parameter.

Call me lazy, but I love the original functionality of the codecov script just finding the coverage file on its own. It's one less concern for me when configuring CI.

Not to mention I have many projects which produce coverage reports in multiple formats. One such projects creates clover, junit, and lcov reports. I'm not sure whether any one is better than any other. I am currently arbitrarily choosing one of them. Would CodeCov prefer one of the others? I don't know.

I have also tried to solve this problem like: file: "coverage/*", but only the first matching report is uploaded, rather than all matching reports.

So my feature request could be any of the following:

The file: parameter is no longer required.
The file: parameter takes an array (or files: is introduced).
Functionality of file: taking a regex is improved upon.

code open

Validating the bash script

It appears that this orb does not do the necessary steps to validate the bash script as described here:
https://docs.codecov.io/docs/about-the-codecov-bash-uploader#validating-the-bash-script

Based on this security breach:
https://about.codecov.io/security-update/

It seems like the orb should be validating the sha sum.

Default upload_name is ${CIRCLE_BUILD_NUM} instead of the build number.

CircleCI does not substitute the value in the parameter default definition, and later the parameter gets used to make a new env var that just has the ${CIRCLE_BUILD_NUM} as the literal value.

Codecov uploader download is randomly failing

I'm randomly getting the following error on the Download Codecov uploader step:

#!/bin/bash -eo pipefail
family=$(uname -s | tr '[:upper:]' '[:lower:]')
os="windows"
[[ $family == "darwin" ]] && os="macos"

[[ $family == "linux" ]] && os="linux"
[[ $os == "linux" ]] && osID=$(grep -e "^ID=" /etc/os-release | cut -c4-)
[[ $osID == "alpine" ]] && os="alpine"
echo "Detected ${os}"
echo "export os=${os}" >> $BASH_ENV

filename="codecov"
[[ $os == "windows" ]] && filename+=".exe"
echo "export filename=${filename}" >> $BASH_ENV
[[ $os == "macos" ]] && brew install gpg
curl -Os "https://uploader.codecov.io/latest/${os}/${filename}"

Detected linux

Exited with code exit status 92
CircleCI received exit code 92

My orb version is codecov/[email protected].

I think the issue is that the curl command isn't doing any retrying, and so any transient network issue can cause it to fail.

Coverage report not found when using the orb and the -f option

Thanks for putting this orb together.

I've recently tried integrating it into a project, and I've noticed that the upload command is not able to find my coverage reports, despite me indicating where they are located. I have experimented with manually re-creating the step in the orb, and it works, so I am uncertain where the issue lies.

.circleci/config.yml file - broken version

  unittest:
    machine:
      enabled: true
      docker_layer_caching: true
    steps:
      - checkout
      - run:
          name: Install Dependencies.
          command: pip install docker-compose
      - run:
          name: Run those tests.
          command: docker-compose -f docker-compose.test.yml up --build --exit-code-from test
      - run:
          name: List that cov file
          command: cat /tmp/auth/.coverage
      - codecov/upload:
          file: /tmp/auth/.coverage

Note that's just one job in the file and not the entire file.

docker-compose.test.yml

version: '3'
services:
  database:
    image: "postgres:latest"
    expose:
      - 5435
    ports:
      - 5435:5432
    environment:
      - POSTGRES_DB=auth_test
      - POSTGRES_USER=auth
      - POSTGRES_PASSWORD=s0_s3cr3t
    logging:
      driver: none
  redis:
    image: "redis:latest"
    logging:
      driver: none
  test:
    stdin_open: true
    tty: true
    build: src
    command: ["/wait-for-it/wait-for-it.sh", "-t", "5", "localhost:5435", "--", "coverage", "run", "management/test.py"]
    volumes:
      - .:/wait-for-it
      - ./src:/app
      - /tmp/auth:/tmp
    depends_on:
      - redis
      - database
    environment:
      ... other env variables related to project
      - PYTHONPATH=$$PYTHONPATH:/app
      - COVERAGE_FILE=/tmp/.coverage

I am mapping /tmp/auth on the host to /tmp in the container. In circle I am validating the output of the script by cat /tmp/auth/.coverage. Here is sample output:

#!/bin/bash -eo pipefail
cat /tmp/auth/.coverage

!coverage.py: This is a private format, don't read it directly!{"lines":{"/app/management/test.py":[1,2,3,4,5,8,9,11,12,14,17,40,61,62,41,43,46,52,53,56,23,24,26,27,30,31,32,37,57,58],"/app/management/migrate.py":[1,2,3,5,6,8,9,11,14,25,15,20,21,22],"/app/models/__init__.py":[2,3,4,5,6,7,8,9,10,11,14,15,16,17,18,19,20,21,22,23],"/app/models/entity.py":[1,2,3,4,5,6,9,13,15,17,18,19,21,22,23,26,27,28,30,32,33,34,35,36],"/app/database.py":[1,2,4,5,6,7,8,9,10],"/app/models/entity_api_key.py":[1,2,3,4,5,6,7,8,9,10,11,14,18,21,23,25,26,27,28,29,31,33,34,35,37,38,39,42,43,44,46,15,48,49,50,51,52,53],"/app/models/entity_password.py":[1,2,3,4,5,6,7,8,9,10,13,17,19,21,22,23,24,27,28,29,31,32,33,36,37,38,40,48,42,43,44,45,50],"/app/models/permission.py":[1,2,3,4,5,6,7,8,9,12,16,18,20,21,22,23,24,26,27,30,31,32,34,35,36,39,40,41,43,53,45,46,47,48,49,50],"/app/models/permission_set.py":[1,2,3,4,5,6,7,8,9,12,15,17,19,20,21,22,23,25,26,29,30,31,33,34,35,38,39,40,42,44,45,46,47,48,49,50],"/app/models/permission_set_permission.py":[1,2,3,4,5,6,7,8,9,12,15,17,19,20,22,23,24,26,27,28,31,32,33,34],"/app/models/entity_permission.py":[1,2,3,4,5,6,7,8,9,12,15,17,19,20,22,23,24,26,27,28,31,32,33,34,36],"/app/models/entity_permission_set.py":[1,2,3,4,5,6,7,8,9,12,15,17,19,20,22,23,24,26,27,28,31,32,33,35],"/app/models/access_token.py":[1,2,3,4,5,6,7,8,11,18,19,20,22,24,25,26],"/app/models/refresh_token.py":[1,2,3,4,5,6,7,8,9,12,16,17,22,23,25,27,28,29,19,20],"/app/models/migrations/__init__.py":[1],"/app/models/migrations/0001_initial.py":[1,2,3,4,5,6,7,8,9,12,13,15,16,18,19,20,21,22,23,24,25,28],"/app/models/migrations/0002_seed_auth_data.py":[1,2,3,4,5,6,7,8,9,10,11,12,13,16,20,22,24,27,28,29,30,34,39,42,43,44,45,46,47,48,49,50,51,52,54,55,56,57],"/app/utils.py":[1,2,3,4,5,6,7,8,9,10,13,17,43,60,93,104,113,120,132,47,49,50,51,52,53,54,57,20,21,22,35,36,37,38,40,99,100,101,108,109,110,139,141,124,126,114,117,127,129,142,143,65,66,67,68,69,70,71,72,76,78,14,79,84,85,86,87,90],"/app/settings.py":[1,4,5,6,9,10,11],"/app/models/migrations/0003_create_access_refresh_token.py":[1,2,3,6,7,9,10,11,12,15],"/app/tests/endpoint_tests/__init__.py":[1],"/app/tests/endpoint_tests/test_entities.py":[1,2,3,4,5,8...

CircleCI output - broken version

#!/bin/bash -eo pipefail
bash <(curl -s https://codecov.io/bash) \
  -F  \
  -f /tmp/auth/.coverage \
  -n ${CIRCLE_BUILD_NUM} \
  -t ${CODECOV_TOKEN} \
  -y .codecov.yml \


  _____          _
 / ____|        | |
| |     ___   __| | ___  ___ _____   __
| |    / _ \ / _` |/ _ \/ __/ _ \ \ / /
| |___| (_) | (_| |  __/ (_| (_) \ V /
 \_____\___/ \__,_|\___|\___\___/ \_/
                              Bash-0b37652


==> Circle CI detected.
    project root: .
--> token set from env
    Yaml found at: .circleci/codecov.yml
==> Running gcov in . (disable via -X gcov)
==> Python coveragepy not found
==> Searching for coverage reports in:
    + .
--> No coverage report found.
    Please visit http://docs.codecov.io/docs/supported-languages

You can see that -f is being passed properly, but no coverage report was found.

Alternatively, when I update my .circleci/config.yml file to do the upload step manually, it works.

.circleci/config.yml - working version

unittest:
    machine:
      enabled: true
      docker_layer_caching: true
    steps:
      - checkout
      - run:
          name: Install Dependencies.
          command: pip install docker-compose
      - run:
          name: Run those tests.
          command: docker-compose -f docker-compose.test.yml up --build --exit-code-from test
      - run:
          name: List that cov file
          command: cat /tmp/auth/.coverage
      - run:
          name: Upload Coverage Results
          command: |
            bash <(curl -s https://codecov.io/bash) -t $CODECOV_TOKEN -f /tmp/auth/.coverage

CircleCI output - working version

#!/bin/bash -eo pipefail
bash <(curl -s https://codecov.io/bash) -t $CODECOV_TOKEN -f /tmp/auth/.coverage


  _____          _
 / ____|        | |
| |     ___   __| | ___  ___ _____   __
| |    / _ \ / _` |/ _ \/ __/ _ \ \ / /
| |___| (_) | (_| |  __/ (_| (_) \ V /
 \_____\___/ \__,_|\___|\___\___/ \_/
                              Bash-0b37652


==> Circle CI detected.
    project root: .
    Yaml found at: .circleci/codecov.yml
    -> Found 1 reports
==> Detecting git/mercurial file structure
==> Reading reports
    + /tmp/auth/.coverage bytes=10300
==> Appending adjustments
    http://docs.codecov.io/docs/fixing-reports
    -> No adjustments found
==> Gzipping contents
==> Uploading reports
    url: https://codecov.io
    query: branch=ryan%2Fcodecov&commit=<hash_obfuscated>&build=544&build_url=&name=&tag=&slug=sokanu%2Fauth&service=circleci&flags=&pr=&job=0
    -> Pinging Codecov
    -> Uploading
    -> View reports at https://codecov.io/github/sokanu/auth/commit/<hash_obfuscated>

Any thoughts?

DeprecationWarning: The -y flag is no longer supported by Codecov

I'm getting this warning on all of our successful codecov/upload results:

DeprecationWarning: The -y flag is no longer supported by Codecov.
  codecov.yml must be located underneath the root, dev/, or .github/ directories

We currently have a .codecov.yml file at the root of our repository that the orb is picking up. Should this be renamed or changed to something else?

Remove dot from .codecov.yml default

codecov/codecov-bash#281

Random failures in uploading report

We've seen a couple of random upload failures. I think the orb should retry the uploading a couple of times if there are any failures. I do notice that we are using an old version of the orb, but I checked the changelog and didn't see any fixes related to this type of issues.

Here is the output:

#!/bin/bash -eo pipefail
cat codecov | bash -s -- \
  -Q "codecov-circleci-orb-1.1.4" \
  -t "${CODECOV_TOKEN}" \
  -n "${CIRCLE_BUILD_NUM}" \
  -F "" \
  -Z || echo 'Codecov upload failed'


  _____          _
 / ____|        | |
| |     ___   __| | ___  ___ _____   __
| |    / _ \ / _` |/ _ \/ __/ _ \ \ / /
| |___| (_) | (_| |  __/ (_| (_) \ V /
 \_____\___/ \__,_|\___|\___\___/ \_/
                              Bash-1.0.5


==> git version 2.20.1 found
==> curl 7.64.0 (x86_64-pc-linux-gnu) libcurl/7.64.0 OpenSSL/1.1.1d zlib/1.2.11 libidn2/2.0.5 libpsl/0.20.2 (+libidn2/2.0.5) libssh2/1.8.0 nghttp2/1.36.0 librtmp/2.3
Release-Date: 2019-02-06
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtmp rtsp scp sftp smb smbs smtp smtps telnet tftp 
Features: AsynchDNS IDN IPv6 Largefile GSS-API Kerberos SPNEGO NTLM NTLM_WB SSL libz TLS-SRP HTTP2 UnixSockets HTTPS-proxy PSL 
==> Circle CI detected.
    project root: .
    Yaml found at: .codecov.yml
==> Running gcov in . (disable via -X gcov)
==> Python coveragepy exists disable via -X coveragepy
    -> Running coverage xml
==> Searching for coverage reports in:
    + .
    -> Found 1 reports
==> Detecting git/mercurial file structure
==> Reading reports
    + ./coverage.xml bytes=908099
==> Appending adjustments
    https://docs.codecov.io/docs/fixing-reports
    + Found adjustments
==> Gzipping contents
        100K	/tmp/codecov.mRKPoc.gz
==> Uploading reports
    url: https://codecov.io
    query: branch=[REDACTED]&commit=[REDACTED]&build=478862&build_url=&name=478862&tag=&slug=[REDACTED]%2F[REDACTED]&service=circleci&flags=&pr=10219&job=2&cmd_args=Q,t,n,F,Z
->  Pinging Codecov
https://codecov.io/upload/v4?package=codecov-circleci-orb-1.1.4-1.0.5&token=<hidden>&package=codecov-circleci-orb-1.1.4-1.0.5&token=************************************&branch=[REDACTED]&commit=[REDACTED]&build=478862&build_url=&name=478862&tag=&slug=[REDACTED]%2F[REDACTED]&service=circleci&flags=&pr=10219&job=2&cmd_args=Q,t,n,F,Z

301
==> Uploading to Codecov
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   99k    0     0  100   99k      0  3094k --:--:-- --:--:-- --:--:-- 3094k
    
Codecov upload failed
CircleCI received exit code 0

Default behaviour of the latest GPG (2.4.1) fails a build

The latest gnupg version (2.4.1) falls back to keyboxd when ~/.gnupg folder doesn't exist.

https://dev.gnupg.org/source/gnupg/browse/master/NEWS$8

It's makes the public key importing to trustedkeys.gpg ineffective and tries to use non-existing default .gnupg/trustedkeys.kbx instead, and effectively fails the build:

#!/bin/bash --login -eo pipefail
source $BASH_ENV
echo "${CODECOV_PUBLIC_PGP_KEY}" | \
  gpg --no-default-keyring --keyring trustedkeys.gpg --import
# One-time step
sha_url="https://uploader.codecov.io"
sha_url="$sha_url/${version}/${os}"
sha_url="$sha_url/${filename}.SHA256SUM"
echo "Downloading ${sha_url}"
curl -Os $sha_url
curl -Os $sha_url".sig"
gpgv $filename.SHA256SUM.sig $filename.SHA256SUM
shasum -a 256 -c $filename.SHA256SUM || \
  sha256sum -c $filename.SHA256SUM

gpg: directory '/Users/distiller/.gnupg' created
gpg: /Users/distiller/.gnupg/trustdb.gpg: trustdb created
gpg: key 806BB28AED779869: public key "Codecov Uploader (Codecov Uploader Verification Key) <[email protected]>" imported
gpg: Total number processed: 1
gpg:               imported: 1
Downloading https://uploader.codecov.io/latest/macos/codecov.SHA256SUM
gpgv: unknown type of key resource 'trustedkeys.kbx'
gpgv: keyblock resource '/Users/distiller/.gnupg/trustedkeys.kbx': General error
gpgv: Signature made Thu Apr 20 16:45:56 2023 GMT
gpgv:                using RSA key 27034E7FDB850E0BBC2C62FF806BB28AED779869
gpgv: Can't check signature: No public key

Exited with code exit status 2
CircleCI received exit code 2

Simple hack to fix it consist of mkdir ~/.gnupg before executing codecov/upload:

      - run:
          name: Quick fix for GPG error in Codecov
          command: mkdir -p ~/.gnupg
      - codecov/upload

Missing CODECOV_TOKEN env var is not error

If running codecov/upload step without the CODECOV_TOKEN environment variable set (or not set correctly) then it doesn't upload to CodeCov (as expected) but CIrcleCI doesn't fail the step. Presumable something is not propagating a non-zero exit code correctly?

Release 1.2.4 and 1.2.5 is missing from Github while available in CircleCI

Hi All,

Release 1.2.4 and 1.2.5 is missing from GitHub while available in CircleCI.
CircleCI ORB registry
.

GitHub

Thanks,
Andor

Process hangs after successful upload

Hey there! 👋 Recently I have observed more and more issues related to report uploads, did anyone else ran into similar problems? 👀

Codecov report uploader 0.4.0
[2023-03-30T10:37:37.927Z] ['info'] => Project root located at: /home/circleci/project
[2023-03-30T10:37:37.929Z] ['info'] ->  Token found by arguments
[2023-03-30T10:37:37.946Z] ['info'] Searching for coverage files...
[2023-03-30T10:37:38.250Z] ['info'] Warning: Some files located via search were excluded from upload.
[2023-03-30T10:37:38.251Z] ['info'] If Codecov did not locate your files, please review https://docs.codecov.com/docs/supported-report-formats
[2023-03-30T10:37:38.251Z] ['info'] => Found 1 possible coverage files:
  coverage/coverage.xml
[2023-03-30T10:37:38.251Z] ['info'] Processing /home/circleci/project/coverage/coverage.xml...
[2023-03-30T10:37:38.321Z] ['info'] Detected CircleCI as the CI provider.
[2023-03-30T10:37:38.323Z] ['info'] Pinging Codecov: https://codecov.io/upload/v4?package=codecov-circleci-orb-3.2.3-uploader-0.4.0<REDACTED>
[2023-03-30T10:37:38.540Z] ['info'] Uploading...
[2023-03-30T10:37:38.779Z] ['info'] {"status":"success","resultURL":"<REDACTED>""

And there it just hangs there (thankfully we have timeouts implemented so in the end job is killed and that cancels the whole build)

Seems like it might be related to the upload endpoint itself as we didn't really changed anything 🤔

The token is written out to the output logs on CircleCI

This is the output from when coverage is uploaded using the orb:

#!/bin/bash -eo pipefail
curl -s https://codecov.io/bash | bash -s -- \
  -f "./coverage.xml" \
  -t "${CODECOV_TOKEN}" \
  -n "${CIRCLE_BUILD_NUM}" \
  -y ".codecov.yml" \
  -F "django,backend" \
  -Z || echo 'Codecov upload failed'

  _____          _
 / ____|        | |
| |     ___   __| | ___  ___ _____   __
| |    / _ \ / _` |/ _ \/ __/ _ \ \ / /
| |___| (_) | (_| |  __/ (_| (_) \ V /
 \_____\___/ \__,_|\___|\___\___/ \_/
                              Bash-8a28df4


==> Circle CI detected.
    project root: /home/circleci/repo
    Yaml found at: .codecov.yml
    -> Found 1 reports
==> Detecting git/mercurial file structure
==> Reading reports
    + ./coverage.xml bytes=528972
==> Appending adjustments
    http://docs.codecov.io/docs/fixing-reports
    + Found adjustments
==> Gzipping contents
==> Uploading reports
    url: https://codecov.io
    query: branch=support%2Ftest&commit=c7b8107b96d9b77f32d5c307aaa1b5946e661a8b&build=11868&build_url=&name=11868&tag=&slug=test%2Frepo&service=circleci&flags=django,backend&pr=&job=0
    -> Pinging Codecov
https://codecov.io/upload/v4?package=bash-8a28df4&token=<<TOKEN-LIES-HERE>>&branch=support%2Ftest&commit=c7b8107b96d9b77f32d5c307aaa1b5946e661a8b&build=11868&build_url=&name=11868&tag=&slug=test%2Frepo&service=circleci&flags=django,backend&pr=&job=0
    -> Uploading
    -> View reports at https://codecov.io/github/test/repo/commit/c7b8107b96d9b77f32d5c307aaa1b5946e661a8b

The section << TOKEN-LIES-HERE >> is output to the logs, and is visible for anyone to read.

Releases are not tagged, Changelog is missing and Version in README is outdated

In the CircleCI Orb registry, version 1.1.1 is recent.

The main branch README.md states 1.1.0 is current.

The latest tag is 1.0.6.

There is no change log.

This makes it somewhat hard for me to understand what code is actually in the Orb and what changed between versions.

It would be great if this could be cleaned up somehow, and if there was some description which place is authoritative.

Uploading multiple coverage report files

I'm running into the following problem: We dynamically group multiple module builds and tests into a single circleci job (to take advantage of the fact that those modules have shared dependencies and therefore it's more resourceful to build them together). This means we will generate multiple coverage reports per job.

I'm unable to find a way to upload multiple coverage reports from a job using the orb:

The file parameter only accepts a single file (although it seems like the underlying uploader supports multiple files?)
I can't have multiple codecov/upload steps because the number of modules built in a job is dynamic and therefore I can't list all steps in my config upfront
I can't let the uploader find the coverage report files itself because it seems to try to upload random XML files

Is there any workaround for the current orb implementation? Would it be possible to allow multiple file uploads from a single step?

Config file not able to be read

It looks like the orb can't find the config file when the path specified is not within the project root. Should me a quick fix to the Orb.

Here's my config:

      - codecov/upload:
          conf: ~/repo/codecov.yml
          file: coverage/.resultset.json

Using orb inside alpine docker image with a Ubuntu host

The problem:
When running a job with a docker image that uses Alpine as the base image, the orb will download the linux codecov binary. The specific error I am getting is:

"version":"v0.1.5"
/bin/bash: line 7: ./codecov: No such file or directory

At first I thought it wasn't downloading it, but doing ls shows the binary is indeed there. Upon doing a bit more research, it seems like when there are missing shared libraries, this above error is thrown.

Am I missing something here?

I've solved the issue by copying the orb code into my circleci.yml and hardcoding os=alpine and it works as expected.

Version 3.0.0 missing from Github

Hi,

On CircleCI the orb is in version 3.0.0:

It seems that the 2.0.0 was published as 3.0.0:

Commit subject: [semver:major] 2.0.0.
Orb `codecov/codecov@dev:8e2cbc4` was promoted to `codecov/[email protected]`.

(from https://app.circleci.com/pipelines/github/codecov/codecov-circleci-orb/518/workflows/d5807f06-b5d6-4604-9282-f54f2ce1471a/jobs/1618)
I see two commits with the message [semver:major] 2.0.0 on master, I think that's why the version in now 3.0.0

Would it be possible to document version 2.0.0 and 3.0.0 in Github releases and to add version 3.0.0 to the changelog?

Thanks!

Missing: Usage example(s)

At least one usage example is required of all Certified/Partner orbs.
https://circleci.com/docs/2.0/orb-concepts/#usage-examples

Example:
source: https://github.com/CircleCI-Public/slack-orb/tree/master/src/examples
registry: https://circleci.com/developer/orbs/orb/circleci/slack#usage-examples

Uploader URL error

On CircleCI I have the following problem.

orbs:
  codecov: 'codecov/[email protected]'

Step codecov/upload gives the following error:

source

#!/bin/bash -eo pipefail
source $BASH_ENV
echo "${CODECOV_PUBLIC_PGP_KEY}" | \
  gpg --no-default-keyring --keyring trustedkeys.gpg --import
# One-time step
sha_url="https://uploader.codecov.io"
sha_url="$sha_url/${version}/${os}"
sha_url="$sha_url/${filename}.SHA256SUM"
echo "Downloading ${sha_url}"
curl -Os $sha_url
curl -Os $sha_url".sig"
gpgv $filename.SHA256SUM.sig $filename.SHA256SUM
shasum -a 256 -c $filename.SHA256SUM || \
  sha256sum -c $filename.SHA256SUM

output

gpg: directory '/home/circleci/.gnupg' created
gpg: keybox '/home/circleci/.gnupg/trustedkeys.gpg' created
gpg: /home/circleci/.gnupg/trustdb.gpg: trustdb created
gpg: key 806BB28AED779869: public key "Codecov Uploader (Codecov Uploader Verification Key) <[email protected]>" imported
gpg: Total number processed: 1
gpg:               imported: 1
Downloading https://uploader.codecov.io///.SHA256SUM
gpgv: can't allocate lock for '/home/circleci/.gnupg/trustedkeys.gpg'
gpgv: no valid OpenPGP data found.
gpgv: the signature could not be verified.
Please remember that the signature file (.sig or .asc)
should be the first file given on the command line.

Exited with code exit status 2

CircleCI received exit code 2

The interesting part in the source is:

sha_url="https://uploader.codecov.io"
sha_url="$sha_url/${version}/${os}"
sha_url="$sha_url/${filename}.SHA256SUM"

The related output is:

Downloading https://uploader.codecov.io///.SHA256SUM

https://circleci.com/developer/orbs/orb/codecov/codecov#commands-upload
version would be a valid parameter, but the os and filename aren't.

Docker executor with image: 'cimg/php:8.1.16-browsers'

1.1.5 shasum fails on macos executor

Excerpt of our circle config:

version: 2.1

orbs:
  codecov: codecov/[email protected]

executors:
  env_executor:
    macos:
      xcode: "12.0.0"

Pipeline fails at "Validate codecov bash uploader" step:

#!/bin/bash --login -eo pipefail
VERSION=$(grep 'VERSION=\"[0-9\.]*\"' codecov | cut -d'"' -f2);
for i in 1 256 512
do
  shasum -a $i -c --ignore-missing <(curl -s https://raw.githubusercontent.com/codecov/codecov-bash/${VERSION}/SHA${i}SUM) ||
  shasum -a $i -c <(curl -s https://raw.githubusercontent.com/codecov/codecov-bash/${VERSION}/SHA${i}SUM)
done

Unknown option: ignore-missing
Type shasum -h for help
(23) Failed writing body
codecov: OK
shasum: env: 
env: FAILED open or read
shasum: WARNING: 1 listed file could not be read

Exited with code exit status 1
CircleCI received exit code 1

Question: How to use the latest version of the orb?

Hi!

I'm not sure if this is the way to ask about the following, I tried to use the v1.0.6 from CI but Circle can't find it:

#!/bin/sh -eo pipefail
# Cannot find codecov/[email protected] in the orb registry. Check that the namespace, orb name and version are correct.
# 
# -------
# Warning: This configuration was auto-generated to show you the message above.
# Don't rerun this job. Rerunning will have no effect.
false

Is it ready to be used, or do we have to wait for a publish on the orb's registry?
In the second case, do you have an ETA?

Upload failed without flags args

The default var for flags is empty string. But if omit this, upload failed.

My .circleci.yml (partial)

      - codecov/upload:
          file: build/coverage.txt

Packed orb config (partial)

   # Orb 'codecov/[email protected]' resolved to 'codecov/[email protected]'
    - run:
        name: Upload Coverage Results
        command: |
          bash <(curl -s https://codecov.io/bash) \
            -F  \
            -f build/coverage.txt \
            -n ${CIRCLE_BUILD_NUM} \
            -t ${CODECOV_TOKEN} \
            -y .codecov.yml \

CI Log (partial)

==> Uploading reports
    url: https://codecov.io
    query: branch=my-branch&commit=<hash>&build=1000&build_url=&name=&tag=&slug=shoma%2Fawesomerepo&service=circleci&flags=-f&pr=&job=0
    -> Pinging Codecov
    -> Uploading
    X> Failed to upload

The flags field have -f in URL Query part.

Convert 'token' parameter type from "string" to "env_var_name".

https://github.com/codecov/codecov-circleci-orb/blob/master/orb.yml#L22

Guide: https://circleci.com/docs/2.0/orbs-best-practices/#parameters

When possible, use defaults for parameters unless a user input is essential. Utilize the “env_var_name” parameter type to secure API keys or other sensitive data.

https://circleci.com/docs/2.0/reusing-config/#environment-variable-name

Consider adding the orbtoberfest label on this issue to make this issue eligible for #Orbtoberfest https://circleci-public.github.io/hacktoberfest

Create a CI/CD pipeline for the orb.

@KyleTryon Is this something you could assist with?

Use `install-ci-tools` from circleci/build-tools

https://circleci.com/developer/orbs/orb/circleci/build-tools#commands-install-ci-tools

For some reason the wrong coverage files are uploaded

I am seeing this when using the codecov circleci orb

Codecov report uploader 0.1.0
[2021-09-16T06:13:58.093Z] ['info'] => Project root located at: /home/circleci/repo
[2021-09-16T06:13:58.095Z] ['info'] ->  Token found by Codecov yaml config
[2021-09-16T06:13:58.595Z] ['info'] Searching for coverage files...
[2021-09-16T06:13:58.964Z] ['info'] => Found 5 possible coverage files:
  codecov.SHA256SUM
  codecov.SHA256SUM.sig
  venv/bin/coverage-3.8
  venv/lib/python3.8/site-packages/coverage/htmlfiles/coverage_html.js
  venv/lib64/python3.8/site-packages/coverage/htmlfiles/coverage_html.js

my commands in the circleci yaml is

- run: # run tests
          name: run tests
          command: |
            . venv/bin/activate
            pytest -c pytest.ini -x --cov-config=.coveragerc --cov-report xml
      - codecov/upload

The orb i use is

orbs:
  codecov: codecov/[email protected]

Add churn

Add churn to files which displays which files are changed more frequently.

AKA changing percentages.

Proper use of semantic versioning in this orb

Orb version 1.2.3 had the token parameter of type string.

Orb version 1.2.4 (and 1.2.5), published last Friday on 7/16/2021, now has the token parameter as type env_var_name.

In my mind, this is a non-backwards compatible breaking change, and thus should have been a major version bump (not a bugfix version bump).

According to https://circleci.com/docs/2.0/orbs-faq/#how-to-use-the-latest-version-of-an-orb, CircleCI uses semantic versioning for its orbs. I had pinned the orb based on the major version of the uploader:

orbs:
  # SEE: https://circleci.com/orbs/registry/orb/codecov/codecov
  codecov: codecov/codecov@1

Since last Friday, all of my repos using this orb need an update for the way the token is passed. Just keep this in mind please.

Cannot find a definition for job named codecov/upload

Hello there,

I'm learning circleci.
I'm not sure I am doing something wrong, or there is something wrong with this orb.

Below is my circleci config file;
apparently it is invalid cause Cannot find a definition for job named codecov/upload.

version: 2.1

orbs:
  node: circleci/[email protected]
  codecov: codecov/[email protected]

workflows:
  unit-tests:
    jobs:
      - node/test:
          version: 15.9.0
      - codecov/upload:
          file: coverage/coverage-final.json

Is there any problem with this configuration? (what?).

Thank you

Fails without flags parameter

Without any flags parameter the bash script will send a request with a bad flag value, which looks like:

https://codecov.io/upload/v4?package=bash-8a28df4&token=secret&branch=python-3&commit=a70e4ff657765330120c5455625338bc68421c8c&build=20&build_url=&name=&tag=&slug=hitta-skyddsrum%2Fservices&service=circleci&flags=-f&pr=&job=0

And gives

HTTP 400
flags must match pattern ^[\w\,]+$

Test is in progress

Testing in progress

Public repo failing to upload w/o token

I'm not sure if this is an issue in the codecov-bash or the orb, however, when a token is not provided (note that this is a pubic repo, so a token shouldn't be necessary), the -y commandline argument is passed as the token value. See the following example failure:

https://circleci.com/gh/rpkilby/tox-factor/54

https://codecov.io/upload/v4?package=bash-8a28df4&token=-y&branch=master&commit=6e360866d29e94fbeb3f4a058296e38139cf6638&build=54&build_url=&name=54&tag=&slug=rpkilby%2Ftox-factor&service=circleci&flags=&pr=&job=0
HTTP 400
Provided token is not a UUID.

The error being: &token=-y

Bash script validation uses invalid option

When running the new codecov orb in CircleCI, several of our builds are now failing with the following error:

Unknown option: ignore-missing
Type shasum -h for help
Unknown option: ignore-missing
Type shasum -h for help
Unknown option: ignore-missing
Type shasum -h for help

In particular, this occurs in the circleci/node Docker container.
This output shows the valid options supported by shasum in this container:
docker run circleci/node:12 shasum --help

Usage: shasum [OPTION]... [FILE]...
Print or check SHA checksums.
With no FILE, or when FILE is -, read standard input.

  -a, --algorithm   1 (default), 224, 256, 384, 512, 512224, 512256
  -b, --binary      read in binary mode
  -c, --check       read SHA sums from the FILEs and check them
  -t, --text        read in text mode (default)
  -U, --UNIVERSAL   read in Universal Newlines mode
                        produces same digest on Windows/Unix/Mac
  -0, --01          read in BITS mode
                        ASCII '0' interpreted as 0-bit,
                        ASCII '1' interpreted as 1-bit,
                        all other characters ignored
  -p, --portable    read in portable mode (to be deprecated)

The following two options are useful only when verifying checksums:
  -s, --status      don't output anything, status code shows success
  -w, --warn        warn about improperly formatted checksum lines

  -h, --help        display this help and exit
  -v, --version     output version information and exit

When verifying SHA-512/224 or SHA-512/256 checksums, indicate the
algorithm explicitly using the -a option, e.g.

  shasum -a 512224 -c checksumfile

The sums are computed as described in FIPS PUB 180-4.  When checking,
the input should be a former output of this program.  The default
mode is to print a line with checksum, a character indicating type
(`*' for binary, ` ' for text, `U' for UNIVERSAL, `^' for BITS, `?'
for portable), and name for each FILE.

Report shasum bugs to [email protected]

Don't fail the build if CodeCov is down

The bash uploader docs suggest adding || echo 'Codecov failed to upload' so that the bash uploader command always returns exit 0 and doesn't fail the build if CodeCov is down momentarily.

https://docs.codecov.io/docs/about-the-codecov-bash-uploader

Should that be included into this orb?

We do not currently use the orb, but have been bitten by this issue before in our circle yml.

Let me know if you'd like me to suggest a PR.

No `when` key

There's no when key for the CircleCI config.yml which is annoying if you have lint problems or something irrelevant.

codecov/upload fails quite often

I'm seeing that the uploading of reports fails quite often on our project. Generating the reports works just fine, but when we run the codecov/upload step, we get failures. This causes some wonkiness in our reports because an entire section is missing.

Here's our CircleCI config.yml (note: all of our packages use the flags option):

- codecov/upload:
    file: packages/web/coverage/coverage-final.json
    flags: web

This is the output I see in CircleCI:

#!/bin/bash -eo pipefail
curl -s https://codecov.io/bash | bash -s -- \
  -f "packages/web/coverage/coverage-final.json" \
  -t "${CODECOV_TOKEN}" \
  -n "${CIRCLE_BUILD_NUM}" \
  -y ".codecov.yml" \
  -F "web" \
  -Z || echo 'Codecov upload failed'
Codecov upload failed
CircleCI received exit code 0

It's possible this is a timeout because this particular step took 2m9s whereas our other 6 uploads took 2s each and succeeded.

[Feature Request] Support versioning/caching the codecov orb used on CI

Observed Behavior

We have in the past observed a few cases where there are outages in fetching the codecov script from remote, causing our CI pipelines to fail when trying to upload. Though we can just let these pipelines pass, we'd like to know when we are no longer getting reports uploaded

Expected Behavior

We would like to reduce the likelihood that we need to fetch a Codecov script from remote. In addition, it would be beneficial to specify a particular version of the tooling and upgrade when our team is ready to pull the newer features in.

Design

Given that the standard behavior to Codecov orb is to pull the latest version, we should preserve that functionality for all current use cases. However, if someone were to specify a codecov-cli-version parameter, then we could detect the addition of the parameter and add a caching step. After the first fetch, this would result in an "offline" fetch of the codecov CLI, which reduces load on your registry and also de-risks the gathering of the reporting script.

Would this be something we can contribute? We plan to prototype it in a fork of this orb but are happy to upstream.

Flags

Hi I was wondering if more than one flag can be associated with an uploaded coverage report?

Add support for Codecov Enterprise

The orb should allow the url to be overridden for use with Codecov Enterprise.

File upload from CircleCI failing with or without token

Hello,

I'm getting the following output in CircleCI:

https://codecov.io/upload/v4?package=bash-20200430-d757c17&token=secret&branch=pull%2F13&commit=3cffe9653cf89f48f74bc405bd88a4f20bebc597&build=14&build_url=&name=14&tag=&slug=TheLabbingProject%2Fdjango_analyses&service=circleci&flags=&pr=13&job=0
 
HTTP 400
 
Provided token is not a UUID.

I added the token to the environment but it didn't help.
Also, the exit code is 0 even though the upload wasn't successful.

mismatch between downloaded codecov bash script and checksums

It seems something went wrong with your auto-releasing of https://github.com/codecov/codecov-bash , which in turn has broken all versions of this orb that validate the bash script. The orb downloads the latest published version from

codecov-circleci-orb/src/@orb.yml

Line 31 in f37e3b2

default: "https://codecov.io/bash"

which is currently the latest in master of codecov-bash https://github.com/codecov/codecov-bash/blob/c3ccd596467b6c08e6fba7dba6e048dd0e341421/codecov
with checksums https://github.com/codecov/codecov-bash/blob/c3ccd596467b6c08e6fba7dba6e048dd0e341421/SHA1SUM
https://github.com/codecov/codecov-bash/blob/c3ccd596467b6c08e6fba7dba6e048dd0e341421/SHA256SUM
https://github.com/codecov/codecov-bash/blob/c3ccd596467b6c08e6fba7dba6e048dd0e341421/SHA512SUM

However the latest release is a different commit https://github.com/codecov/codecov-bash/releases/tag/1.0.3 with different checksums, and it is these the orb validates against by using the version string