The repository contains the resources for the json-ld contexts that are being used. Nonetheless these aren't supplied to the issuers/verifiers so the issuers/verifiers need to supply those same contexts themselves.

I wonder if we should pass these to the issuers/verifiers or if it isn't worth it. The only way I can think to do it is to pass a json dictionary like {context-uri: context-filename} as a keyword argument. But that seems kind of silly?

When running the shell command given on step 8, I received the following error:

brent@Lappy2000:~/dev/w3c/vc-test-suite$ cd implementations/ && node generate.js
/home/brent/dev/w3c/vc-test-suite/implementations/generate.js:25
      (Object.keys(test.err).length === 0) ? 'success' : 'failure';
              ^

TypeError: Cannot convert undefined or null to object
    at Function.keys (<anonymous>)
    at results.tests.forEach (/home/brent/dev/w3c/vc-test-suite/implementations/generate.js:25:15)
    at Array.forEach (<anonymous>)
    at files.forEach (/home/brent/dev/w3c/vc-test-suite/implementations/generate.js:23:17)
    at Array.forEach (<anonymous>)
    at Object.<anonymous> (/home/brent/dev/w3c/vc-test-suite/implementations/generate.js:16:7)
    at Module._compile (module.js:652:30)
    at Object.Module._extensions..js (module.js:663:10)
    at Module.load (module.js:565:32)
    at tryModuleLoad (module.js:505:12)

Timeout for tests seems to be set to 2000 ms
generating actual keys for CL signatures may take much much longer than this.
Can the timeout be changed?

Update table of contents on the implementations report.

There are several JSON-LD input documents where the proof property does not adhere to the spec data model requirements of a data integrity proof outlined in https://www.w3.org/TR/vc-data-integrity/#proofs. Here's an illustrative example in example1.jsonld (which is meant to be correct)

{
  "@context": [
    "https://www.w3.org/2018/credentials/v1",
    "https://www.w3.org/2018/credentials/examples/v1"
  ],
  "id": "http://example.edu/credentials/58473",
  "type": ["VerifiableCredential", "AlumniCredential"],
  "issuer": "https://example.edu/issuers/14",
  "issuanceDate": "2010-01-01T19:23:24Z",
  "credentialSubject": {
    "id": "did:example:ebfeb1f712ebc6f1c276e12ec21",
    "alumniOf": "Example University"
  },
  "proof": {
    "type": "RsaSignature2018"
  }
}

But the following properties are mandated (MUST) by the spec:

• verificationMethod
• proofPurpose

Looking at the spec for the proof type RsaSignature2018 I can't see where/if it defines an alternative data model - https://w3c-ccg.github.io/lds-rsa2018/ and when looking at the context

This isn't the case for all of the examples, for instance example-5.jsonld has the correct construction:

{
  "@context": [
    "https://www.w3.org/2018/credentials/v1",
    "https://www.w3.org/2018/credentials/examples/v1"
  ],
  "id": "http://example.gov/credentials/3732",
  "type": ["VerifiableCredential", "UniversityDegreeCredential"],
  "issuer": "https://example.edu",
  "issuanceDate": "2017-06-18T21:19:00Z",
  "credentialSubject": {
    "id": "did:example:ebfeb1f712ebc6f1c276e12ec21",
    "degree": {
      "type": "BachelorDegree",
      "name": "Bachelor of Science in Mechanical Engineering"
    }
  },
  "proof": {
    "type": "RsaSignature2018",
    "created": "2017-06-18T21:19:10Z",
    "proofPurpose": "assertionMethod",
    "verificationMethod": "https://example.com/jdoe/keys/1",
    "jws": "eyJhbGciOiJSUzI1NiIsImI2NCI6ZmFsc2UsImNyaXQiOlsiYjY0Il19..TCYt5XsITJX1CxPCT8yAV-TVkIEq_PbChOMqsLfRoPsnsgw5WEuts01mq-pQy7UJiN5mgRxD-WUcX16dUEMGlv50aqzpqh4Qktb3rk-BuQy72IFLOqV0G_zS245-kronKb78cPN25DGlcTwLtjPAYuNzVBAh4vGHSrQyHUdBBPM"
  }
}

Could y'all point me to where the schema for this is defined that means that the values aren't required? And if they are required should we update the input documents?

Hi,

I was able to run and test against our current implementation of VC. But I was just curious about the various tests suite, especially refresh, evidence, status, tou etc.

Our implementation does not explicitly deal with these items, but the tests are correctly passing. So my question is, are these tests with the sole purpose of ensuring that these properties are not dropped when the VC issuance occurs, or are we supposed to alter the properties to some extent?

Thanks for your answers and thank you for this tool.

Files are not copyrighted

sorry for the nit pick =)
Apparently the copyright is needed for open source standards.
util.js has no copyright etc.

Hi,

so I am working to move our library to compliance with the VC standard.
The second test of the basic suite is @context MUST be one or more URIs (negative) https://github.com/w3c/vc-test-suite/blob/gh-pages/test/vc-data-model-1.0/10-basic.js#L39, and the fixture only has one value in the array https://github.com/w3c/vc-test-suite/blob/gh-pages/test/vc-data-model-1.0/input/example-1-bad-cardinality.jsonld#L3.

However if I look at the spec: https://www.w3.org/TR/vc-data-model/#contexts, or even at what vc-js does: https://github.com/digitalbazaar/vc-js/blob/44ca660f62ad3569f338eaaaecb11a7b09949bd2/lib/vc.js#L498, it is not transparent that @context must have more than one value. Actually even the wording of the test spec says "one or more".

Hence my question, is this test actually valid? And why is vc-js marked as passing when no check is explicitly made on the length of the array?

Thanks

example-5-bad-proof.jsonld is supposed to fail because the proof is bad.
Before it can fail due to a bad proof, it fails due to a bad issuanceDate

meanwhile, example-5-bad-proof-missing-type.jsonld is completely blank.

Lines like

   // The following binary will, given an input file, produce a valid
   // Verifiable Credentials Data Model document.
   // use "type" on Windows as a "cat" replacement
   "generator":"/bin/cat",

Do you think it might be a plan to write it like

   "--": "The following binary will, given an input file, produce a valid",
   "--": "Verifiable Credentials Data Model document.",
   "--": "use 'type' on Windows as a "cat" replacement",
   "generator":"/bin/cat",

?

RFC 8259 says "The names within an object SHOULD be unique." We could just suffix numbers to the key names. But I think this is one of the odd situations whose existence is the reason it's SHOULD and not MUST.

README.nd says

### Running the Test Suite

1. `npm install`
2. `cp config.json.example config.json`
3. Modify `config.json` for your run (be sure to remove all comments)

config.json.example says
/**

• An example local configuration for the test suite. To use:
• 1. Copy this file to one called config.json.
• 2. Modify the file and replace with appropriate values for your system.
• 3. npm install

I don't *think* it makes a difference, but it's inconsistent.

The ES256K private key in config.json.jwt.example uses curve "P-256K", which is not in RFC 7518 or the JSON Web Key Elliptic Curve registry.

COSE and JOSE Registrations for WebAuthn Algorithms (draft) specifies "P-256K" for JWK. Later drafts changed it to "secp256k1", possibly to avoid confusion with "P-256" which is a different curve id. The draft was published as RFC 8812, entering "secp256k1" in the JWK Elliptic Curve registry.

The current Process delegates full authority to the Director in making decision to sign Memoranda of Understanding; the only option for the Membership to disagree is through the appeal process. In particular, the process doesn't call for any prior engagement from the Team with the Membership prior to signing such an MoU.

As we're moving towards Director-free, and given recent experience where the Membership has indicated interest in greater involvement in these discussions, the Process should probably consider whether having a required advance notice or even a formal AC Review before reaching such a decision would satisfy this need.

Originally posted by @dontcallmedom in w3c/w3process#606

The following are negative cases for a VerifiablePresentation without verifiableCredential:

• example-8-bad-missing-verifiableCredential.jsonld
• example-015-zkp-vp-bad-missing-verifiableCredential.jsonld

But the Verifiable Presentation Request Specification has an example DID Authentication response which is a VerifiablePresentation without a verifiableCredential:
https://w3c-ccg.github.io/vp-request-spec/#example-4-a-didauth-response

The Verifiable Credentials Data Model says "If present," about the verifiableCredential property of a Presentation, which I think implies that it is optional:
https://w3c.github.io/vc-data-model/#presentations-0

Should vc-test-suite be updated to allow a VP without VC?

What is correct JWK for the decoding JWS into the credentials data model tests (--jwt-decode parameter), example-016-jwt.jwt? It's RS256 but the one defined at config.json.example is not valid:

      "rs256PrivateKeyJwk":{
         "kty":"RSA",
         "n":"0vx7agoebGcQSuuPiLJXZptN9nndrQmbXEps2aiAFbWhM78LhWx4cbbfAAtVT86zwu1RK7aPFFxuhDR1L6tSoc_BJECPebWKRXjBZCiFV4n3oknjhMstn64tZ_2W-5JsGY4Hc5n9yBXArwl93lqt7_RN5w6Cf0h4QyQ5v-65YGjQR0_FDW2QvzqY368QQMicAtaSqzs8KJZgnYb9c7d0zgdAZHzu6qMQvRL5hajrn1n91CbOpbISD08qNLyrdkt-bFTWhAI4vMQFh6WeZu0fM4lFd2NcRwr3XPksINHaQ-G_xBniIqbw0Ls1jF44-csFCur-kEgU8awapJzKnqDKgw",
         "e":"AQAB",
         "d":"X4cTteJY_gn4FYPsXB8rdXix5vwsg1FLN5E3EaG6RJoVH-HLLKD9M7dx5oo7GURknchnrRweUkC7hT5fJLM0WbFAKNLWY2vv7B6NqXSzUvxT0_YSfqijwp3RTzlBaCxWp4doFk5N2o8Gy_nHNKroADIkJ46pRUohsXywbReAdYaMwFs9tv8d_cPVY3i07a3t8MN6TNwm0dSawm9v47UiCl3Sk5ZiG7xojPLu4sbg1U2jx4IBTNBznbJSzFHK66jT8bgkuqsk0GjskDJk19Z4qwjwbsnn4j2WBii3RL-Us2lGVkY8fkFzme1z0HbIkfz0Y6mqnOYtqc0X4jfcKoAC8Q",
         "p":"83i-7IvMGXoMXCskv73TKr8637FiO7Z27zv8oj6pbWUQyLPQBQxtPVnwD20R-60eTDmD2ujnMt5PoqMrm8RfmNhVWDtjjMmCMjOpSXicFHj7XOuVIYQyqVWlWEh6dN36GVZYk93N8Bc9vY41xy8B9RzzOGVQzXvNEvn7O0nVbfs",
         "q":"3dfOR9cuYq-0S-mkFLzgItgMEfFzB2q3hWehMuG0oCuqnb3vobLyumqjVZQO1dIrdwgTnCdpYzBcOfW5r370AFXjiWft_NGEiovonizhKpo9VVS78TzFgxkIdrecRezsZ-1kYd_s1qDbxtkDEgfAITAG9LUnADun4vIcb6yelxk",
         "dp":"G4sPXkc6Ya9y8oJW9_ILj4xuppu0lzi_H7VTkS8xj5SdX3coE0oimYwxIi2emTAue0UOa5dpgFGyBJ4c8tQ2VF402XRugKDTP8akYhFo5tAA77Qe_NmtuYZc3C3m3I24G2GvR5sSDxUyAN2zq8Lfn9EUms6rY3Ob8YeiKkTiBj0",
         "dq":"s9lAH9fggBsoFR8Oac2R_E2gw282rT2kGOAhvIllETE1efrA6huUUvMfBcMpn8lqeW6vzznYY5SSQF7pMdC_agI3nG8Ibp1BUb0JUiraRNqUfLhcQb_d9GF4Dh7e74WbRsobRonujTYN1xCaP6TO61jvWrX-L18txXw494Q_cgk",
         "qi":"GyM_p6JrXySiz1toFgKbWV-JdI3jQ4ypu9rbMWx3rQJBfmt0FoYzgUIZEVFEcOqwemRN81zoDAaa-Bk0KWNGDjJHZDdDmFhW3AN7lI-puxk_mHZGJ11rxyR8O55XLSe3SPmRfKwZI6yU24ZxvQKFYItdldUKGzO6Ia6zTKhAVRU",
         "alg":"RS256",
         "kid":"did:example:0xab#verikey-1"
      }

test-suite input files contain example contexts that don't resolve. when producing the credential, type
is dependant on the example context. How should the producer dereference in a way that will pass the test-suit?

This test suite uses did-auth-jose, which is no longer maintained. Its readme recommends against using it: decentralized-identity/did-auth-jose@2e39867

The current implementation report does not contain links to current (public) implementations. I would like to invite everyone to also provide a link to the corresponding repo when submitting test results. This would help a lot with transparency regarding W3C process and CR requirements.

Please feel free to reply to this issue and I'll put together the list.

Are there any tests that show a credential issued by one of the implementations and checked by another?

Perhaps the answer is "all of them"? If so, are the credential documents checked in anywhere? If not, would you please share an example and explain how I could reproduce the results?

I'm trying to understand what sort of market is established by this spec. To what extent should I expect credentials issued, by, for example, Sovrin-Ken_Ebert to be verified by uPort and vice versa?

Since that's part of the definition for "conforming document" from https://w3c.github.io/vc-data-model/#json-ld and w3c/vc-data-model#1381.

My config.json contains:

"sectionsNotSupported": ["refresh", "evidence", "tou", "jwt"]

This causes the status tests to be skipped.

Any concurrent PRs updating implementation reports cause a merge conflict in implementations/index.html. Could we use an Actions workflow to build the index.html from the main branch and commit it to gh-pages, so that index.html could be removed from the main branch and remove this source of merge conflicts?

Add key/legend section for test results (check, X, untested, etc)

The tests for date-time checking are using a regex for ISO8601. The spec requires date-times to adhere to RFC3339

Could you please explain the meaning of @context bad cardinality check from basic group? I just cannot understand what's wrong with this definition of VC (from example-1-bad-cardinality.jsonld):

{
  "@context": [
    "https://www.w3.org/2018/credentials/v1"
  ],
  "id": "http://example.edu/credentials/58473",
  "type": ["VerifiableCredential", "AlumniCredential"],
  "issuer": "https://example.edu/issuers/14",
  "issuanceDate": "2010-01-01T19:23:24Z",
  "credentialSubject": {
    "id": "did:example:ebfeb1f712ebc6f1c276e12ec21",
    "alumniOf": "Example University"
  },
  "proof": {
    "type": "RsaSignature2018"
  }
}

Hi there,

First of all thank you for implementing and providing guidance to use this particular test-suite, I really appreciate all the hard work you guys have done.

I would like to raise an issue on the generated template for the test suite.
Specifically, I would like to direct your attention to the Basic Documents section itself.

There are certain test cases that "do not belong" there, and upon closer inspection, instead these tests are from the Linked Data Proof and Credential Schema sections.

looking through the raw mocha results and 10-basic.js, I cannot seem to find these tests as well. Also, it seems like only this individual Guillaume-Daumas has these tests passing, whereas the rest of the implementations do not.

This might have been a git merge mistake which has persisted since thereafter.

Is it possible to take a look at this?
Thank you.

The verifiable credential data model doesn't dictate a proof type or a specific ZKP implementation. If implementers implement ZKPs that differ from the CL signature example, should the test suit for ZKPs fail?

I believe the content type is wrong for the required context: https://w3.org/2018/credentials/v1. Example test suite file failes to load using the json-ld playground

Examples input files that contain attribute/value pairs like:

"name": "<span lang='fr-CA'>Baccalauréat en musiques numériques</span>",

should be changed to something like:

"name": "Bachelor of Science in Mechanical Engineering",

If no explicit rule is specified, properties are encoded in the same way as with a standard verifiable credential, and are added to the vc property of the JWT.

The README directions for submitting an implementation report have some errors.
4. Rename implementation/results.json to implementation/YOUR_IMPLEMENTATION-results.json.
should be:
4. Rename implementations/report.json to implementations/YOUR_IMPLEMENTATION-report.json.

5. git add implementations/YOUR_IMPLEMENTATION-results.json and submit a pull request for your implementation.
should be:
5. git add implementations/YOUR_IMPLEMENTATION-report.json and submit a pull request for your implementation.

And it is unclear what the purpose of this instruction is:
6. cd implementations/ && node generate.js

This test passes for me (I get a green check mark) when I run it with /bin/cat as my credential processor (i.e. the test passes when the output is the input).
My assumption is that this should not be the case.
There seem to be two problems here:

1. The test passes even though my processor doesn't return an error
2. The test file example-1-bad-cardinality.jsonld seems to have good cardinality (the context URIs are out of order, but there ARE two of them)

The auto-generated report.json document only has passed and failed as results from tests. This is regardless of whether a MUST or a MAY was tested, or whether the feature was implemented or not implemented.
If my implementation does not provide @context it does not conform to the VC spec.
If my implementation does not provide expirationDate it does conform to the VC spec.
However, both of those reports would only indicate failure. There is nothing in the report to differentiate between the 'failure' of an optional element test (e.g., for MAY, SHOULD, RECOMMENDED), and the 'failure' of an essential element test (e.g., for MUST).

The type field is a MUST in the VC spec, so each proof field in the input example JSON-LD files needs to have a type (except where it explicitly shouldn't in order to test error handling of a proof object without a type field).

Trying to figure out the test-suite and be able to run this on my implementation, I see the follow statement
"1) encoded as standard JOSE header parameters, 2) encoded as registered JWT claim names, or ..."

I have no idea what "registered" means and where would one register and why I have to register my JWT claims?

Could you please explain the meaning of type bad cardinality check from basic group? It's defined at example-3-bad-cardinality.jsonld as following:

{
  "@context": [
    "https://www.w3.org/2018/credentials/v1",
    "https://www.w3.org/2018/credentials/examples/v1"
  ],
  "id": "http://example.edu/credentials/3732",
  "type": "VerifiableCredential",
  "issuer": "https://example.edu/issuers/14",
  "issuanceDate": "2010-01-01T19:23:24Z",
  "credentialSubject": {
    "id": "did:example:ebfeb1f712ebc6f1c276e12ec21",
    "degree": {
      "type": "BachelorDegree",
      "name": "Bachelor of Science in Mechanical Engineering"
    }
  },
  "proof": {
    "type": "RsaSignature2018"
  }
}

From the other hand, the same type definition can be found in several other test cases, e.g. in example-016-jwt.jsonld which should be processed with no error.

In example-8.jsonld, the embedded verifiable credential contains no context. If the presentation validation software validates each embedded credential, the validation will fail.

"verifiableCredential": [{
"id": "http://example.edu/credentials/3732",

would become something like

"verifiableCredential": [{
"@context": [
"https://www.w3.org/2018/credentials/v1",
"https://www.w3.org/2018/credentials/examples/v1"
],
"id": "http://example.edu/credentials/3732",

This is my terminal output when trying to run the report:

brent@Lappy2000:~/dev/w3c/vc-test-suite$ npm run report

[email protected] report /home/brent/dev/w3c/vc-test-suite
mocha --recursive test/vc-data-model-1.0/ -R json > implementations/report.json

npm ERR! Linux 4.15.0-50-generic
npm ERR! argv "/usr/bin/node" "/usr/bin/npm" "run" "report"
npm ERR! node v8.10.0
npm ERR! npm v3.5.2
npm ERR! code ELIFECYCLE
npm ERR! [email protected] report: mocha --recursive test/vc-data-model-1.0/ -R json > implementations/report.json
npm ERR! Exit status 8
npm ERR!
npm ERR! Failed at the [email protected] report script 'mocha --recursive test/vc-data-model-1.0/ -R json > implementations/report.json'.
npm ERR! Make sure you have the latest version of node.js and npm installed.
npm ERR! If you do, this is most likely a problem with the vc-test-suite package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR! mocha --recursive test/vc-data-model-1.0/ -R json > implementations/report.json
npm ERR! You can get information on how to open an issue for this project with:
npm ERR! npm bugs vc-test-suite
npm ERR! Or if that isn't available, you can get their info via:
npm ERR! npm owner ls vc-test-suite
npm ERR! There is likely additional logging output above.

npm ERR! Please include the following file with any support request:
npm ERR! /home/brent/dev/w3c/vc-test-suite/npm-debug.log

I was able to directly run the command:
mocha --recursive test/vc-data-model-1.0/ "--timeout" "30000" -R json > implementations/report.json

When decoded, VC inside JWS (example-016-jwt.jwt) is incorrect:

{
  "@context": [
    "https://w3.org/2018/credentials/v1",
    "https://example.com/examples/v1"
  ],
  "credentialSubject": {
    "degree": {
      "name": "Bachelor of Science in Mechanical Engineering",
      "type": "BachelorDegree"
    }
  },
  "expirationDate": "2019-11-06T08:42:03Z",
  "id": "http://example.edu/credentials/3732",
  "issuanceDate": "2018-11-06T08:42:04Z",
  "issuer": "did:example:abfe13f712120431c276e12ecab",
  "type": [
    "VerifiableCredential",
    "UniversityDegreeCredential"
  ]
}

as the first context must be https://www.w3.org/2018/credentials/v1 (according to https://w3c.github.io/vc-data-model/#contexts).
And here www is missing (currently the context is https://w3.org/2018/credentials/v1).

As per recommendation from @bumblefudge opening this issue here. 1

The VC Data Model describes the credentialSubject as "a set of objects that contain one or more properties".

Looking at the vc-test-suite this isn't tested (it only checks for the omission of a credential subject). Is there relevance in requiring the credentialSubject to be non-empty?

If so, should the vc-test-suite test for this requirement from the specification?

@jyasskin I've raised this issue to track the WG response to the feedback you provided on the test suite and to track the work required to address the limitations you've uncovered.

From feedback on VC Data Model v1.1

The implementation testing described in
https://w3c.github.io/vc-test-suite/implementations/#testing-methodology is
insufficient for these changes, as it doesn't test that the full set of
values that can be generated for the `id`, `proof`, and `credentialSchema`
properties by one implementation can be understood and used by all the
other conforming implementations.

@clehner has volunteered to work on this, but assistance from others is also welcome.

https://w3c.github.io/vc-test-suite/implementations/#conformance-testing-results
does not seem to include all the tests my implementation passed.
Only 1 of 9 ZKP tests shows as passing even though the implementation passed all ZKP tests at the last run of the tests.

As a result, the following flags are passed to a generator for zkp tests:

• --jwt
• --jwt-aud
• --jwt-presentation

A VC implementation may successfully create conforming credentials, but how that implementation should interact with the test suite to prove itself is unclear.
The test suite could use better documentation (what precisely will be passed to the implementation by the test suite, what is the test suite expecting as output from the implementation, etc. ?)
A simple working example would also be very helpful.

See https://www.w3.org/TR/vc-data-model/#evidence

DocumentVerification2018...

this tremendously poorly defined and should be defined for all serializations or removed.

Ideally we would have a test suite that shows how this property supports interoperability, and that suite would cover popular feature such as identity assurance in sufficient detail.

The context: https://www.w3.org/2018/credentials/examples/v1 is invalid JSON. If you paste the context into https://jsonlint.com/ the following parse error occurs:

Error: Parse error on line 24:
...			"schema:alumniOf",			"@type": rdf: H
-----------------------^
Expecting ':', got ','

How should the verifier know if a derived zkp verifiable credential is missing a credentialSchema as opposed to a non derived verifiable credential that does not require credentialSchema?

The example @context has IRIs with the wrong content type.

Both https://www.w3.org/2018/credentials/v1 and https://www.w3.org/2018/credentials/examples/v1 have a content type
content-type: application/octet-stream and should be content-type: application/ld+json

Hello,

I've currently finished integrating this test-suite to my implementation of W3C VerifiableCredentials.
However, I'm also interested in finding out if my implementation signs a VC correctly. And if a signed VC from my implementation can be verified by someone else.

I'm searched online for libraries that would do such a thing and I've found a few that seem promising. However, they all differ in some way or another, and I'm afraid that none follow the official W3C specifications. Therefore, even if I pass their tests, that doesn't assure me that my implementation is compliant.
Why leads me to my question: is there any plan on W3C's side to make such a library? (Or to integrate it with this one).

This would be extremely useful. I understand that W3C DIDs and VCs are still quite young. But I would argue that this is even more of a reason to establish such a library. Because, as of right now, many different libraries are doing their own thing, and an "official library" would help bring things together and avoid having many slightly different implementations in the future.

I haven't been able to find much information about this online, so I figured I'd ask here.

Thank you for your help.