Description

When running sigstore locally with the OAuth OIDC flow, I occasionally get HTTP 400s. Happens with both sigstore.dev and sigstage.dev as the instance. The error code indicates that it's almost certainly a payload construction error on our side, possibly due to raciness when setting up the local HTTP listener.

Example invocation:

$ sigstore sign README.md

Example traceback:

BUG:urllib3.connectionpool:https://oauth2.sigstage.dev:443 "POST /auth/token HTTP/1.1" 400 70
Traceback (most recent call last):
  File "/Users/william/devel/sigstore-python/sigstore/_internal/oidc/oauth.py", line 193, in get_identity_token
    resp.raise_for_status()
  File "/Users/william/devel/sigstore-python/env/lib/python3.10/site-packages/requests/models.py", line 960, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: https://oauth2.sigstage.dev/auth/token

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/william/devel/sigstore-python/env/bin/sigstore", line 8, in <module>
    sys.exit(main())
  File "/Users/william/devel/sigstore-python/env/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/Users/william/devel/sigstore-python/env/lib/python3.10/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/Users/william/devel/sigstore-python/env/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/william/devel/sigstore-python/env/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/william/devel/sigstore-python/env/lib/python3.10/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/Users/william/devel/sigstore-python/sigstore/_cli.py", line 110, in _sign
    identity_token = get_identity_token(
  File "/Users/william/devel/sigstore-python/sigstore/_internal/oidc/oauth.py", line 195, in get_identity_token
    raise IdentityError from http_error
sigstore._internal.oidc.IdentityError

Description

Once this library supports ambient credential detection in #31, we should update our release workflow to sign via GitHub's ambient OIDC identity instead of explicitly requesting it.

Blocked on #31.

Once we get to (or close to) a stable release, we should begin to keep a CHANGELOG.

In order to implement signing, we need to write a Python client to interact with Fulcio. We need to write this in a generic way so that other programs can use pysign's Fulcio client for their own purposes.

I'm not a complete beginner and I still have trouble understanding what exactly I can actually verify with sigstore-python at the moment... It would be helpful if there was an example in the documentation of verifying e.g. that a sigstore-python release was actually built by the github action of the project.

The status at the moment seems to be:

• certificate and signature could easily be published along the artifacts (since sign --output-* options were just added) but currently they are not published
• There is a somewhat undocumented verify --cert-email option that seems to be what I want, just slightly badly named: it looks like it would allow me to check that SAN contains URI:https://github.com/sigstore/sigstore-python/.github/workflows/release.yml@refs/tags/$VERSION
• I imagine in addition to SAN I should verify that the signing key is a github key? Is there a process for this or should the verifying script just hardcode the keyid? Should there be an option in verify for this?

I'd really like to use sigstore-python for this exact purpose so I'm willing to do some work here but I think I need a bit of guidance at least: Is my understanding of how to verify that build artifacts were built by a specific github action correct? Am I missing anything with regards to my last bullet point: how do I verify that the cert actually comes from GitHub?

Once we've written the Fulcio and Rekor clients, we should be fill out the importable API. The API should roughly line up with the subcommands that cosign supports.

The CLI itself (which we'll implement in #3) will be a thin wrapper that wires the argparse flags to the API. So the meat of the CLI logic will be covered in this task.

I expect that we'll separate this into multiple issues when we begin working on it.

In #114 @woodruffw raised that this file may become stale on new releases.

To resolve this and provide a tighter feedback loop on a release -> update, we could

• a) add a pin to the requirements.in file
• b) introduce a workflow that ensures the compiled file is up to date, and
• c) also introduce a dependabot-like workflow that runs on every new new release to create a PR to update the pin (something like https://github.com/di/pip-api/blob/master/.github/workflows/pip-feed.yml should work).

Description

When trying to use the --output-* flags with default values, invocation fails because they try to consume the filename of the artifact to be signed:

$ python -m sigstore sign --output-signature --output-certificate hi.txt
Usage: python -m sigstore sign [OPTIONS] FILE [FILE ...]
Try 'python -m sigstore sign --help' for help.

Error: Missing argument 'FILE [FILE ...]'.

Once we have SET and Merkle inclusion merged, we should set aside some time to add unit tests that fully exercise both.

Once we've implemented the sigstore API, we can write a CLI wrapper to call into it with various flags.

We should aim for CLI compatibility with cosign where applicable.

Related to #31 and #50: we should support disabling ambient credential detection and use, with the same CLI flag that cosign uses (--oidc-disable-ambient-providers as of sigstore/cosign#1832).

We did this for pip-audit; we should do it here too.

Testing against the staging instances of Fulcio, Rekor, Sigstore OIDC provider, etc will be a common use case so we should have a --staging flag that sets --fulcio-url, --rekor-url and --oidc-provider as appropriate.

Let's leave this until after #27 since we haven't exposed --rekor-url yet.

Breaking this out for discussion from #107: --cert-email can be misleading when the OIDC identity in question isn't an email, but rather a GitHub workflow or other URL. We could provide CLI aliases that make this clearer.

cc @di for thoughts.

Title says it all: we should add this flag, and make it required IFF --cert-email is supplied.

See #106 and sigstore/cosign#1947.

Also see #27.

This doesn't seem to exist in pyca/cryptography. We should check whether there's another library that includes this functionality. If not, we might need to hand-roll it ourselves.

Description

When signing, the following exception is displayed:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/sigstore/_internal/fulcio/client.py", line 233, in post
    sct = FulcioSignedCertificateTimestamp(resp.headers["SCT"])
  File "/usr/local/lib/python3.10/site-packages/requests/structures.py", line 54, in __getitem__
    return self._store[key.lower()][1]
KeyError: 'sct'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/local/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/usr/local/lib/python3.10/site-packages/sigstore/__main__.py", line 22, in <module>
    main()
  File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.10/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/sigstore/_cli.py", line 119, in _sign
    result = sign(
  File "/usr/local/lib/python3.10/site-packages/sigstore/_sign.py", line 102, in sign
    certificate_response = fulcio.signing_cert.post(certificate_request, identity_token)
  File "/usr/local/lib/python3.10/site-packages/sigstore/_internal/fulcio/client.py", line 236, in post
    raise FulcioClientError from exc
sigstore._internal.fulcio.client.FulcioClientError

Version

This affects all currently published versions of sigstore.

Fix

This is blocked on a release that includes #84.

I noticed this exception raised when running verification:

Traceback (most recent call last):
  File "/Users/tetsuo/Code/sigstore-python/env/bin/sigstore", line 8, in <module>
    sys.exit(main())
  File "/Users/tetsuo/Code/sigstore-python/env/lib/python3.9/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/Users/tetsuo/Code/sigstore-python/env/lib/python3.9/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/Users/tetsuo/Code/sigstore-python/env/lib/python3.9/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/tetsuo/Code/sigstore-python/env/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/tetsuo/Code/sigstore-python/env/lib/python3.9/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/Users/tetsuo/Code/sigstore-python/sigstore/_cli.py", line 299, in _verify
    if verify(
  File "/Users/tetsuo/Code/sigstore-python/sigstore/_verify.py", line 128, in verify
    store_ctx.verify_certificate()
  File "/Users/tetsuo/Code/sigstore-python/env/lib/python3.9/site-packages/OpenSSL/crypto.py", line 1896, in verify_certificate
    raise self._exception_from_context()
OpenSSL.crypto.X509StoreContextError: [20, 0, 'unable to get local issuer certificate']

This is trivial to trigger for me. All I need to do is this:

$ sigstore sign test_file.txt --output-signature --output-certificate
$ sigstore verify test_file.txt --cert test_file.txt.crt --signature test_file.txt.sig

From looking around, the error seems to indicate that we might need to pass the entire certificate chain into verification.

Description

In order to catch regressions, we should run scheduled tests against rekor/fulcio staging.

We could optionally run these in CI as well, but allow failures.

Fulcio will soon support Certificate Signing Requests (sigstore/fulcio#527 is merged but not deployed at the time of writing).

Once deployed, we should switch to using CSRs instead, using roughly this pattern:

https://github.com/trailofbits/sigstore-python/blob/6cb0b25127ab21f90704145458df6e454d6a83e1/sigstore/_sign.py#L35-L50

We currently fail rather than returning None if the virtualization product's ID isn't what we expect, which causes problems for e.g. Docker containers:

sigstore._internal.oidc.ambient.AmbientCredentialError: GCP: product name file exists, but product name is 'BHYVE'; giving up

As mentioned at #24 (comment), Fulcio will soon be supporting embedded SCTs. We should switch over to using them at some point.

#45 added license headers to files, but this is not enforced, we should check for it in CI.

It might be helpful to reuse https://github.com/pypa/warehouse/blob/main/bin/licenses

At the moment, we've decided to check in the CTFE public key and use it to verify SCTs. The way this should really work is that we should check in the root key (root.json) and use it to download the CTFE key via TUF.

We should check the signing timestamp that Rekor gives us and ensure that it is before the expiry of the signing certificate.

As part of verification, we get an inclusion proof from Rekor. At the moment we have a verify_merkle_inclusion function which is currently a no-op but we should fill it in.

As part of the verify command, we should verify the SET that we get from Rekor.

Fulcio has a separate proof flow for GitHub Actions ODIC tokens which don't have an email claim: https://github.com/sigstore/fulcio/blob/main/config/config.jsn

We should provide the signed sub claim as proof for GHA tokens instead of attempting to get the email claim from the token.

This issue was automatically created by Allstar.

Security Policy Violation
Found 3 outside collaborators with admin access.
This policy requires all users with this access to be members of the organisation. That way you can easily audit who has access to your repo, and if an account is compromised it can quickly be denied access to organization resources. To fix this you should either remove the user from repository-based access, or add them to the organization.

• Remove the user from the repository-based access. From the main page of the repository, go to Settings -> Manage Access.
  (For more information, see https://docs.github.com/en/account-and-profile/setting-up-and-managing-your-github-user-account/managing-access-to-your-personal-repositories/removing-a-collaborator-from-a-personal-repository)

OR

• Invite the user to join your organisation. Click your profile photo and choose “Your Organization” → choose the org name → “People” → “Invite Member.” (For more information, see https://docs.github.com/en/organizations/managing-membership-in-your-organization/inviting-users-to-join-your-organization)

If you don't see the Settings tab you probably don't have administrative access. Reach out to the administrators of the organisation to fix this issue.

This issue will auto resolve when the policy is in compliance.

Issue created by Allstar. See https://github.com/ossf/allstar/ for more information. For questions specific to the repository, please contact the owner or maintainer.

Create basic package structure and empty modules so that we can begin collaborating.

In order to implement uploading/verifying of signatures, we need to write a Python client to interact with Rekor. We need to write this in a generic way so that other programs can use pysign's Rekor client for their own purposes.

We should look at what flags that cosign provides and consider adding them to sigstore-python. Some examples that we've already identified include controlling the URL of the Fulcio and Rekor instances that we call out to.

Description

I'd like to explore a format for a "verification configuration bundle". This would be:

• a single human-readable file (probably JSON)
• with a specific extension (like .sigstore)
• that includes the verification materials for one or more signatures for one or more artifacts (likely keyed by SHA256 hash)
• as well as configuration for additional extension verification

So for #106, this could be something like:

{
  "config": {
    "cert-oidc-issuer": "https://token.actions.githubusercontent.com",
    ...
    "<some other extension>": "<some other value>"
  },
  "artifacts": {
    "<sha256 hash>": {
      "bundles": [
        {
          "sig": "<signature>"
          "crt": "<base64 encoded certificate>"
        },
        ...
        <potentially additional signatures>
     ]
    },
    "<another sha256 hash>": {
        <same as above>
    }
  }
}  

Which would be used like:

# For multiple artifacts:
$ python -m sigstore verify <something>.sigstore artifacts_to_verify/*

# Same bundle, but works for a single artifact:
$ python -m sigstore verify <something>.sigstore artifacts_to_verify/artifact.ext

Some open questions:

• What should the name of this file be, if it signs multiple artifacts?
• Should the "additional verification" be across all artifacts, or specific to individual artifacts? Or just potentially override-able?
• How would a user create/edit this file with a sigstore client?

As part of #12, we've disabled interrogate while we're in the initial phases of development. When things are looking more solid, let's add a docstring for everything and reenable interrogate.

Description

Before the signing process is invoked, we should warn (or perhaps refuse to sign / output verification materials) if the sign will overwrite existing files with the new verification materials.

We could possibly, additionally, provide a flag that permits the overwrite.

Cosign supports 'ambient credential detection' for a number of environments where OIDC identities are available by default. We should also similarly support:

• GitHub Actions (#59)
• Google Cloud: VMs, GKE clusters, Cloud Build, etc. (#88, #91)
• GitLab (SaaS): https://docs.gitlab.com/ee/integration/openid_connect_provider.html
  • blocked on sigstore/fulcio#243 (sigstore/fulcio#983)
    • blocked on https://gitlab.com/groups/gitlab-org/-/epics/7335
• CircleCI: https://circleci.com/docs/2.0/openid-connect-tokens/
  • blocked on sigstore/fulcio#591
    • blocked on https://circleci.canny.io/cloud-feature-requests/p/customizable-audience-claim-in-oidc-tokens
  • implementation: #72
• BuildKite (#499)
  • Fulcio: sigstore/fulcio#890

See also https://dlorenc.medium.com/a-bit-of-ambiance-comes-to-sigstore-f80d1d6b1c30

This issue is tracking support for SaaS products and not self-hosted instances, e.g. GitLab's hosted product and not their on-premise or self-hosted services. Self-hosted services are out-of-scope, pending further discussion with Fulcio.

Description

We should support the ability to write the certificate and signature(s) to files.

Cosign does this via --output-certificate and --output-signature:

$ COSIGN_EXPERIMENTAL=1 cosign sign-blob --help
Sign the supplied blob, outputting the base64-encoded signature to stdout.

...

      --output-certificate string                write the certificate to FILE
      --output-signature string                  write the signature to FILE

...

If you folks would like we could bring Bandit into CI. I am maintainer over there if we needed anything in particular. We should not feel obliged though, whatever you all see as fitting.

Similar to cosign, this library should support passing globs as filenames. Currently, it fails:

$ sigstore sign --identity-token=$(...) dist/*
Usage: sigstore sign [OPTIONS] FILE
Try 'sigstore sign --help' for help.

Error: Invalid value for 'FILE': 'dist/*': No such file or directory

We have some hardcoded assumptions that we're using ECDSA keys with SHA256 hashes to generate signatures. Fulcio supports other signing and hashing algorithms so we should allow them too. For example, the staging instance is currently using an RSA CTFE key for signing.

Transfer this repo to https://github.com/sigstore and make all the necessary updates to docs/URLs.

Description

As described in the specification, a Rekor response can be stored and verified without hitting the log. The client should:

• Verify the SET
• Verify that the signature over the blob matches what's in the payload's body
• Compare the integrated time against the certificate
• Compare hashes of the Rekor payload body to the provided blob

See code in Cosign for more details.

Figure out what dependencies we're going to need to implement the pysign API.

• Fulcio and Rekor seem to expose a REST API so requests should cover that.
• Creating key pairs and using them to sign packages could be interesting. pyca/cryptography looks like it could be a good candidate.

We should do the same thing pip-audit does. We might want to hold off on it until the repository is migrated, however.

Now that we have an automated release workflow, we should constraint it to a custom environment to take advantage of more robust secret management.

We should populate the README with an explanation of what this repo is and provide some CLI usage examples.

We've temporarily disabled coverage testing while we're fleshing out the basic signing/verification functionality. Once things take a bit more shape, we should begin adding back coverage.

I expect this issue to be broken up into smaller PRs to test each module separately since there's probably quite a lot of work to do.