latchset / pkcs11-provider Goto Github PK
View Code? Open in Web Editor NEWA pkcs#11 provider for OpenSSL 3.0+
License: Other
A pkcs#11 provider for OpenSSL 3.0+
License: Other
In src/sessions.c we try to come up with some numbers to use to limit the amount of sessions we use at the same time and the number we can keep open as a cache.
This code really needs to be rethought, as these numbers are pulled out of thin air and the calculations are rough and inconsistent.
Hey,
before, I start doing this (it will require some tweaking), we had great success with adding and using .clang-format
to reformat files automatically. I can try to model the .clang-format
according to the current code (or just use linux
format). But before I do that, is this something you would be willing to accept? There are numerous plugins to editors to format on save and it makes the life much easier in the end.
Describe the bug
openssl req command fails.
No errors are displayed as a result of this command but no csr file is created and the return value from the openssl req command is 1.
openssl pkey and openssl rsautl commands are working well.
To Reproduce
Steps to reproduce the behavior:
$ openssl req -new -batch -key "pkcs11:type=private;id=%02" -out tmp.csr
Expected behavior
csr file and return valur 0
Operating environment (please complete the following information):
Token and application used (please complete the following information):
Describe the bug
Doing RSA PKCS1.5 signature with pre-calculated hash with hash information is calculated wrong. A DigestInfo struct should be included in the data to be signed.
To Reproduce
Patch to add for the test suite:
diff --git a/tests/tbasic b/tests/tbasic
index 7153cb3..07015e9 100755
--- a/tests/tbasic
+++ b/tests/tbasic
@@ -54,6 +54,19 @@ pkeyutl -verify -inkey "${PUBURI}"
-in ${TMPPDIR}/sha256.bin
-sigfile ${TMPPDIR}/sha256-sig.bin'
+title PARA "Sign and Verify with provided Hash and RSA with DigestInfo struct"
+ossl 'dgst -sha256 -binary -out ${TMPPDIR}/sha256.bin ${SEEDFILE}'
+ossl '
+pkeyutl -sign -inkey "${PRIURI}" -pkeyopt digest:sha256
+ -in ${TMPPDIR}/sha256.bin
+ -out ${TMPPDIR}/sha256-sig.bin'
+
+ossl '
+pkeyutl -verify -inkey "${PUBURI}" -pkeyopt digest:sha256
+ -pubin
+ -in ${TMPPDIR}/sha256.bin
+ -sigfile ${TMPPDIR}/sha256-sig.bin'
+
title PARA "Sign and Verify with provided Hash and EC"
ossl '
pkeyutl -sign -inkey "${ECBASEURI}"
Steps to reproduce the behavior:
make check
Expected behavior
Modified test suite passes.
Additional context
The OpenSSL core will properly set rsasig context, and result in log:
Set OSSL_SIGNATURE_PARAM_DIGEST to sha256
However, the sequence to call sign/verify will end up in a call to p11prov_sig_set_mechanism(digest_sign=false)
and the digest gets completely ignored there.
It seems PKCS11 does not support signature with any of the CKM_SHA256_RSA_PKCS
etc., so it is likely that the provider code should calculate and produce the DigestInfo
structure with the hash info and use CKM_RSA_PKCS
sign/verify mechanism.
Originally posted by gilweis November 4, 2022
Hi,
I am trying to check the library with yubikey token.
OpenSSL 3.0.5 5 Jul 2022 (Library: OpenSSL 3.0.5 5 Jul 2022)
latest pkcs11-provider library
pkcs11 module: /usr/lib/x86_64-linux-gnu/libykcs11.so
key-type: "rsa:2048"
The operation of reading the public key is ok:
$ openssl pkey -in "pkcs11:type=public;id=%02" -pubin -pubout -out testkey2.pub
The operation with private key fails:
$ openssl req -new -batch -key "pkcs11:type=private;id=%02" -out tmp.csr
Error making certificate request
4007D721007F0000:error:0580006F:x509 certificate routines:X509_PUBKEY_set:unsupported algorithm:../crypto/x509/x_pubkey.c:359:
Describe the bug
git HEAD is 47e1c47
When configuring in a build directory (build) outside of the source directory make check
fails:
doug@XU-22:/a/appl/pkcs11-provider-git/build$ make check
Making check in src
make[1]: Entering directory '/a/appl/pkcs11-provider-git/build/src'
make[1]: Leaving directory '/a/appl/pkcs11-provider-git/build/src'
Making check in tests
make[1]: Entering directory '/a/appl/pkcs11-provider-git/build/tests'
make tsession tgenkey ttls tdigests \
helpers.sh setup-softhsm.sh setup-softokn.sh softhsm-proxy.sh test-wrapper tbasic teccsha2 thkdf toaepsha2 trsapss
make[2]: Entering directory '/a/appl/pkcs11-provider-git/build/tests'
make[2]: 'tsession' is up to date.
make[2]: 'tgenkey' is up to date.
make[2]: 'ttls' is up to date.
make[2]: 'tdigests' is up to date.
make[2]: Nothing to be done for '../../pkcs11-provider/tests/helpers.sh'.
make[2]: Nothing to be done for '../../pkcs11-provider/tests/setup-softhsm.sh'.
make[2]: Nothing to be done for '../../pkcs11-provider/tests/setup-softokn.sh'.
make[2]: Nothing to be done for '../../pkcs11-provider/tests/softhsm-proxy.sh'.
make[2]: Nothing to be done for '../../pkcs11-provider/tests/test-wrapper'.
make[2]: Nothing to be done for '../../pkcs11-provider/tests/tbasic'.
make[2]: Nothing to be done for '../../pkcs11-provider/tests/teccsha2'.
make[2]: Nothing to be done for '../../pkcs11-provider/tests/thkdf'.
make[2]: Nothing to be done for '../../pkcs11-provider/tests/toaepsha2'.
make[2]: Nothing to be done for '../../pkcs11-provider/tests/trsapss'.
make[2]: Leaving directory '/a/appl/pkcs11-provider-git/build/tests'
make check-TESTS
make[2]: Entering directory '/a/appl/pkcs11-provider-git/build/tests'
make[3]: Entering directory '/a/appl/pkcs11-provider-git/build/tests'
SKIP: basic-softokn
SKIP: basic-softhsm-proxy
SKIP: eccsha2-softokn
SKIP: oaepsha2-softokn
SKIP: hkdf-softokn
SKIP: rsapss-softokn
SKIP: digests-softokn
SKIP: digests-softhsm-proxy
SKIP: genkey-softokn
SKIP: genkey-softhsm
SKIP: session-softokn
SKIP: session-softhsm-proxy
SKIP: tls-softokn
SKIP: tls-softhsm-proxy
============================================================================
Testsuite summary for pkcs11-provider 0.1
============================================================================
# TOTAL: 14
# PASS: 0
# SKIP: 14
# XFAIL: 0
# FAIL: 0
# XPASS: 0
# ERROR: 0
To Reproduce
Steps to reproduce the behavior:
Expected behavior
make check works when building in the source but not when built outside the source
Operating environment (please complete the following information):
Additional context
Adding set -xv
to test
wrapperfailure to source
testvars` shows each test fails with 77 like this one:
TEST_PARAMS=(${1//-/ })
+ TEST_PARAMS=(${1//-/ })
TEST_NAME=$(basename ${TEST_PARAMS[0]})
++ basename ../../pkcs11
+ TEST_NAME=pkcs11
TEST_PATH=$(dirname ${TEST_PARAMS[0]})
++ dirname ../../pkcs11
+ TEST_PATH=../..
TOKEN_DRIVER=${TEST_PARAMS[1]}
+ TOKEN_DRIVER=provider/tests/basic
if [ -f "./tmp.${TOKEN_DRIVER}/testvars" ]; then
source ./tmp.${TOKEN_DRIVER}/testvars
else
exit 77 # token not configured, skip
fi
+ '[' -f ./tmp.provider/tests/basic/testvars ']'
+ exit 77
SKIP basic-softokn (exit status: 77)
Most of PKCS#11 tokens support RNGs. We need to use these RNGs as a part of openssl RNG support (probably just seeding and/or separate RNG)
Describe the feature
The open_ex interface provide more information to the open interface including a UI method that carries information for the the store.
Describe the bug
openssl ca commad with yubikey tokens fails.
To Reproduce
Steps to reproduce the behavior:
mkdir demoCA
mkdir demoCA/newcerts demoCA/private
echo "01" > ./demoCA/serial
touch ./demoCA/index.txt
#Creating CA cert:
openssl req -x509 -new -nodes -key "pkcs11:type=private;id=%02" -days 1024 -out ./demoCA/cacert.pem -sha256
#Creating certificate from csr:
openssl ca -out newcert.pem -days 3650 -in tmp.csr -keyfile "pkcs11:type=private;id=%02" -md sha256
error:
CA certificate and CA private key do not match
4087EFF7627F0000:error:05800075:x509 certificate routines:X509_check_private_key:unknown key type:../crypto/x509/x509_cmp.c:411:
Expected behavior
newcert.pem file
Operating environment (please complete the following information):
Token and application used (please complete the following information):
Device: Yubikey5
PKCS11 Driver version: [e.g. 1.3.4]
Application openssl
Version 3.0.5
Additional context
The provider-keymgmt API supports key generation. This ticket is feature holder to implement the EC/RSA key generation support.
Describe the feature
Provide feedback to user while generating keys by using OpenSSL clabback system when possible.
Expected behavior
When the token supports it provide user feedback on key generation.
Additional context
PKCS11 support the use of CK_CALLBACK_FUNCTION to allow feedback when the token knows long operations are expected.
This could be used to pipe this feedback all the way to OpenSSL callling application by setting appropriate provider callbacks.
The RSA and ECDSA are the most common keys these days in the PKCS#11 world, but the EdDSA is becoming also more and more popular. There are HSMs supporting these keys for some time (yubiHSM) as well as some smart cards/tokens based on OpenPGP (NitroKey with the gnuk applet). It will also become more important when the FIPS 186-5 will get approved.
Describe the bug
OpenSSL has two separate phases in dealing with storemgmt.
It expects to be able to open the store w/o requiring credentials, and then passed any callback (that cna provide passwords) at time of key loading.
When I changed the code to be able to list public objects w/o requiring a login I broke the fallback code that was going to retry reading the store in the load phase when a password is available.
To Reproduce
Steps to reproduce the behavior:
Remove the PIN from the condifuration file and execute anopenssl command that requires the PIN (anything that uses a private key).
No prompting will happen and the command fails to execute with an error about not being able to open the pkcs11 uri passed in.
Expected behavior
Prompting happens and the operation completes.
Operating environment (please complete the following information):
N/A
Token and application used (please complete the following information):
N/A
Additional context
Potentially always deferring the open operation could solve this.
The PKCS#11 WG has a page with common bugs and while reading one of the pkcs11 traces from pkcs11-provider, I noticed that it does this: Calling FindObjectsInit, then the GetAttributeValue and then moving on to other objects.
C_FindObjectsInit/Find/End can't be interrupted
Interrupting the sequence C_FindObjectsInit/C_FindObjects/C_FindObjectsEnd with a different call results in inconsistent/erratic results. This makes it difficult to safely perform any general processing of multiple objects rather than just using the find-objects capability as a basic "find me this one object".
https://wiki.oasis-open.org/pkcs11/CommonBugs
Last time I was writing something like this I think I was pointed to that page by somebody more experienced that this is wrong (even though the wording is not very clear). I changed the code to run the Find, store the handles and then read the objects attributes like this:
https://github.com/OpenSC/OpenSC/blob/master/src/tests/p11test/p11test_case_common.c#L800-L875
I could not find any similar wording in the PKCS#11 specification though and I believe any sane PKCS#11 module should handle this, but if there is somebody with more experience who could add more information, it would be appreciated
Describe the bug
openssl ocsp commad with yubikey tokens fails
To Reproduce
Steps to reproduce the behavior:
[ usr_cert ]
authorityInfoAccess = OCSP;URI:http://127.0.0.1:8080
[ v3_OCSP ]
basicConstraints = CA:FALSE
keyUsage = nonRepudiation, digitalSignature, keyEncipherment
extendedKeyUsage = OCSPSigning
$ export OPENSSL_CONF=openssl.cnf
$ openssl genrsa -out rootCA.key 1024
$ openssl req -new -x509 -days 3650 -key rootCA.key -out rootCA.crt
$ openssl genrsa -out certKey.key 1024
$ openssl req -new -x509 -days 3650 -key certKey.key -out certificate.crt
$ openssl x509 -x509toreq -in certificate.crt -out CSR.csr -signkey certKey.key
$ openssl ca -batch -startdate 150813080000Z -enddate 250813090000Z -keyfile rootCA.key -cert rootCA.crt -policy policy_anything -notext -out certificate.crt -infiles CSR.csr
$ openssl req -new -key "pkcs11:type=private" -out ocspSigning.csr -sha256
$ openssl ca -keyfile rootCA.key -cert rootCA.crt -in ocspSigning.csr -out ocspSigning.crt
$ openssl ocsp -index demoCA/index.txt -port 8080 -rsigner ocspSigning.crt -rkey "pkcs11:type=private;id=%02" -CA rootCA.crt -text -out log.txt
ACCEPT 0.0.0.0:8080 PID=31356
ocsp: waiting for OCSP client connections...
$ openssl ocsp -CAfile rootCA.crt -issuer rootCA.crt -cert certificate.crt -url http://127.0.0.1:8080 -resp_text -noverify
ocsp: Received request, 1st line: POST / HTTP/1.0
Segmentation fault (core dumped)
Describe the bug
All tests fail, except for tls_softtokn
.
It appears that the tests omit to load this provider?
To Reproduce
Steps to reproduce the behavior:
make check
test-suite.log
tls-softhsm-proxy.log
tls-softokn.log
readkeys-softhsm-proxy.log
readkeys-softokn.log
session-softhsm-proxy.log
session-softokn.log
genkey-softhsm.log
genkey-softokn.log
digests-softhsm-proxy.log
digests-softokn.log
rsapss-softokn.log
hkdf-softokn.log
oaepsha2-softokn.log
eccsha2-softokn.log
certs-softhsm-proxy.log
certs-softokn.log
basic-softhsm-proxy.log
basic-softokn.log
setup-softhsm.log
setup-softokn.log
Expected behavior
All tests passed.
Operating environment (please complete the following information):
Token and application used (please complete the following information):
Additional context
Using OpenSSL master (3.2.0dev).
Here's the patch to compile pkcs11-provider
- without it (at least on MacOS) it fails to get ssize_t
type and load definition for snprintf()
function.
diff --git a/src/provider.h b/src/provider.h
index 07b66c8..23a9e35 100644
--- a/src/provider.h
+++ b/src/provider.h
@@ -10,6 +10,7 @@
#include <stdbool.h>
#include "pkcs11.h"
+#include <sys/types.h>
#include <openssl/core_dispatch.h>
#include <openssl/core_object.h>
#include <openssl/types.h>
diff --git a/src/session.c b/src/session.c
index 57e4169..0e1339e 100644
--- a/src/session.c
+++ b/src/session.c
@@ -1,8 +1,8 @@
/* Copyright (C) 2022 Simo Sorce <[email protected]>
SPDX-License-Identifier: Apache-2.0 */
-#include "provider.h"
#include <string.h>
+#include "provider.h"
/* Slot stuff */
struct p11prov_slot {
diff --git a/src/util.c b/src/util.c
index ed996cf..0aad107 100644
--- a/src/util.c
+++ b/src/util.c
@@ -1,12 +1,13 @@
/* Copyright (C) 2022 Simo Sorce <[email protected]>
SPDX-License-Identifier: Apache-2.0 */
-#include "provider.h"
#include <string.h>
#include <time.h>
-#include "platform/endian.h"
+#include <sys/types.h>
#include <openssl/bn.h>
#include <openssl/x509.h>
+#include "provider.h"
+#include "platform/endian.h"
#define FA_RETURN_VAL(x, _a, _b) \
do { \
Describe the bug
./configure
fails to locate /opt/local/lib/nss/libsoftokn3.so
.
To Reproduce
Steps to reproduce the behavior:
autoreconf -fi
./configure
checking for /opt/local/lib/nss/libsoftokn3.so... no
. . .
configure: WARNING: Softoken library missing, tests will fail!
Expected behavior
Find /opt/local/lib/nss/libsoftokn3.dylib
, because (of course) dynamic libraries on Mac have extension .dylib
rather than .so
.
Operating environment (please complete the following information):
Token and application used (currently irrelevant):
Additional context
Add any other context about the problem here.
We have fallback code to load an associated public key from a private key object.
We need to extend this to also try to use a certificate object if a public key object is not available as some tokens store only certificate and private key and depend on software to retrieve the public key from the certificate.
I would like to build and run the provider on a Windows system.
Is Windows compatibility planned in the near future?
Describe the bug
The SoftHSM supports only CKM_ECDSA
mechanism, which is raw EC signature. The pkcs11-provider fails when we try to create hashed signature using any digest+signature fails with the following logs:
[interface.gen.c:291] p11prov_CopyObject(): Calling C_CopyObject
[objects.c:168] cache_key(): Key 17:25 cached as 27:27
[signature.c:1630] p11prov_ecdsa_set_ctx_params(): ecdsa set ctx params (ctx=0x2164b50, params=(nil))
[signature.c:1454] p11prov_ecdsa_digest_sign_update(): ecdsa digest sign update (ctx=0x2164b50, data=0x451f00, datalen=65)
[interface.gen.c:153] p11prov_GetMechanismInfo(): Calling C_GetMechanismInfo
[interface.gen.c:157] p11prov_GetMechanismInfo(): Error: 0x00000070; Error returned by C_GetMechanismInfo
C_GetMechanismInfo for CKM_ECDSA_SHA512(4166) failed 112
The provider should fall back to the CKM_ECDSA
if the CKM_ECDSA_SHA*
mechanism is not available by doing the digest inside OpenSSL. This might be problematic with the certifications, but it should improve the imteroperability with mode pkcs11 providers or smart cards.
To Reproduce
Steps to reproduce the behavior:
Expected behavior
The signatures using ECDSA keys work with CKM_ECDSA mechanism only.
Operating environment (please complete the following information):
Token and application used (please complete the following information):
Additional context
This should also allow us enabling the ecdsa signature tests with softhsm pipeline. The following change:
diff --git a/tests/Makefile.am b/tests/Makefile.am
index c3c58aa..87fae4d 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -46,7 +46,7 @@ dist_check_SCRIPTS = \
test_LIST = \
basic-softokn basic-softhsm-proxy \
certs-softokn certs-softhsm-proxy \
- eccsha2-softokn \
+ eccsha2-softokn eccsha2-softhsm-proxy \
oaepsha2-softokn \
hkdf-softokn \
rsapss-softokn \
Now gives the same error here:
Error signing raw input data
Public Key operation error
804B32944F7F0000:error:40800070:pkcs11:p11prov_GetMechanismInfo:An invalid mechanism was specified to the cryptographic operation:interface.gen.c:157:Error returned by C_GetMechanismInfo
804B32944F7F0000:error:40800001:pkcs11:p11prov_sig_operate_init:reason(1):signature.c:605:Failed to open session on slot 17
OpenSSL wants DER encoded ECDSASignature, but PKCS#11 uses RAW byte sequences.
See PR #121 for a test case.
Describe the feature
Currently pkcs11 provider fails to login if a hardware token uses an onboard pinpad or some other out of band authentication method
Expected behavior
Authentication via out of band token authentication should work
Additional context
The token_login function unconditionally fails if a pin is not available w/o trying a C_Login operation. A C_login operation should be still tried if no pin is available but the token advertises
CKF_PROTECTED_AUTHENTICATION_PATH
The OpenSSL API allows to enumerate multiple objects in a store.
This can turn useful if a token stores multiple certificates with their private keys and we want to enumerate the available certificate to find out if we have a private key to match an incoming request.
This is useful in environments where a single HSM is used to serve multiple services and therefore it stores multiple keys/certificate, for example on a terminating load balancer.
Implement support for enumerating all objects on the token instead of the single shot API currently enforced.
This is both a RFE issue as well as an open question to other project contributors.
Although this attribute is no in c11 which is our minimum standard, it is supported by gcc and clang, and makes cleanup of variables a lot nicer as it avoids goto statements in most functions.
I'll leave this here for a little while and see if there is feedback.
If I do not get any major push back I might pick it up and implement later.
The OASIS PKCS#11 header files that you have included are non-free.
Please consider replacing them with the drop-in replacements from the NSS library or pkcs11.h from OpenSC.
Describe the bug
When parsing parts of the PKCS#11 URI, the function parse_attr()
allocates just the len + 1
bytes for the members (for example uri->object
, but when these are compared against the token data, the hardcoded length from the PKCS#11 specification (32B) is used in p11prov_uri_match_token()
. This leads to the buffer overruns (probably not reported because the openssl's allocs are used) and failures to match the keys.
To Reproduce
Steps to reproduce the behavior:
Expected behavior
Matching works
Operating environment (please complete the following information):
Token and application used (please complete the following information):
Additional context
The parse_attr
either needs to allocate the full length of the buffers it is checked against in p11prov_uri_match_token()
or the p11prov_uri_match_token()
needs to be adjusted to not compare behind the end of the buffer, ignoring trailing whitespace.
Please add:
pkg-config
libtool
Thanks
This is most probably not really needed, but since this is in initial state, it should be probably documented what version of C is going to be used. My vote goes to C11/C17. It's well supported to some extent by all still supported distros including RH7. I don't think there's a strong argument for C99 in 2022.
Describe the feature
We should not try to generate new keys using a particular slot if it doesn't support the necessary mechanism.
Describe the feature
PKCS11 provider may provide the same algorithms with or without FIPS property. It depends on the underlying token certification status. Algorithm properties should be manageable via config.
Expected behavior
If a private key has CKA_ALWAYS_AUTHENTICATE
set CK_TRUE
to do any crypto operations on it, a secondary C_Login(CKU_CONTEXT_SPECIFIC)
is needed in connection with the crypto operation.
On fork a lot of PKCS#11 object handles become irrelevant. We need to have a test like "login-fork-do stuff" to check whether all these handles are dealt with correctly.
In some cases (HW accelerator) we want to insure all operations done with our keys need to stay on the token.
If we allow exporting keys though, openssl will try hard to switch to the default provider to perform operations.
By allowing config that prevents export of public keys we may force computations to stay on the token.
This can't be the default because on key generation you reallu need to be able to export a public key so that it can be handed over, so this is not a cut and dry thing.
Describe the bug
The creation of a SSL server context fails, if the pkcs11-prrovider is configured.
To Reproduce
gcc -o ssl-ctx-new ssl-ctx-new.c -l ssl -l crypto
)make check
)OPENSSL_CONF=tests/openssl.cnf PKCS11_PROVIDER_DEBUG="file:p11prov-debug.log" ./ssl-ctx-new
)The SSL_CTX_new() call returns NULL, if the pkcs11-provider is configured. If the line pkcs11 = pkcs11_sect
in openssl-cnf is commented out, the creation of the context works as expected.
Expected behavior
The SSL_CTX_new() call should also work with the pkcs11-provider.
Operating environment (please complete the following information):
Token and application used (please complete the following information):
NSS Soft-Token. Not key material is involved for the creation of the SSl context.
Additional context
Attachments: the c-example and the debug log.
Once we have a version of OpenSSL that includes openssl/openssl#19648 or equivalent we should add tests that insure no regressions on CA cert manipulation.
There is already some place holder code for CKO_CERTIFICATE / CKC_X_509 support. This is a feature place holder to implement supporting X.509 certificate objects on PKCS#11 module.
Describe the bug
If the attribute requested is sensitive or otherwise and is "required" the function may return short w/o processing all data and with the wrong error code (CKR_HOST_MEMORY)
To Reproduce
N/A
Expected behavior
All available attributes should be processed and returned even on error
P11PROV_debug statements (or even P11PROV_raise for required ones) are emitted so that there is a log of which attributes were required/requested and were not returned
Operating environment (please complete the following information):
N/A
Token and application used (please complete the following information):
N/A
Additional context
N/A
Having a pin on the pkcs11_uri is useful but also not great as those strings are shared in many places.
A configuration option to specify a pin (directly) and a pin file (so permissions can be locked down) should be provided.
Describe the feature
On iterating slots, if we need a specific mechanism, we should check whether the slot supplies this mechanism.
Current strategy is to create PKCS11 sessions on the spot when needed. However is not ideal from interoperability and performance:
Additional potential PKCS11 complication to account for is that all sessions share the logged-in user and state. Thus some operations might be possible if there are sessions using from logged-in user (e.g. some thread doing crypto as normal user, but another thread wanting to create/delete objects needing SO user).
This ticket is to discuss the strategy to manage sessions and concurrent access.
Two options to consider:
Perhaps additional options brew up during discussion of the problem.
Describe the bug
configure: WARNING: The p11-kit client not found. Can not run SoftHSM tests
To Reproduce
./configure
Expected behavior
A clear and concise description of what you expected to happen.
Operating environment (please complete the following information):
Additional context
$ softhsm2-util --version
2.6.1
Currently, the debug messages are very rough and not much verbose. Getting hang of what is going on there is sometimes hard for the people who did not write the code.
From debugging other projects for years, I was very happy with the verbosity of debug messages, that contain at least a function name in them so it is easy to figure out where the message comes from. For example, now messages can look like this:
Get session on slot 18
Error: 224
Failed to get session to load keys
The way to figure out which error is the above is mostly about grepping for some strings that look like they should be fixed (sometimes already in other function or in other file) and then guessing where the code went through. The least would be the function name, similarly like this which should already give rough estimate where the message comes from (this is something like OpenSSH):
p11prov_get_session: Get session on slot 18
p11prov_session_open: Error: 224
store_fetch: Failed to get session to load keys
Even better would be to have there exact location in the file, as sometimes functions are not always in the file where I would expect them to be. Adding pid or thread ID might also come very helpful when this will be used in multi-thread/multi-process applications in the future (this is similar what we have in OpenSC now):
[1234][T1] session.c:714 p11prov_get_session: Get session on slot 18
[1234][T1] session.c:471 p11prov_session_open: Error: 224
[1234][T1] store.c:106 store_fetch: Failed to get session to load keys
Additionally, I found useful in the past log messages at the beginning of any non-trivial function that record the function was called. This makes easy to trace the program and see where it went off the path I would expect.
The last minor note would be interpretation of the CKR values or printing these values in hexadecimal, as that is easier to find in the header files. Having the following log would already give much more information than the decimal number in the previous cases:
Error: 0xD0 (TEMPLATE_INCOMPLETE)
This is mostly request for comments if somebody sees some additional information that would be useful to get logged or if you see some drawbacks of logging some of the information before I will start rewriting this code.
Describe the bug
It looks like OpenSSL adds RSA:rsaEncryption:1.2.840.113549.1.1.1
to its name table in ossl_namemap_add_names
as umber 0 then when it tries to use it, the assert in evp_fetch.c:112 fails.
It is hard to debug as OpenSSL uses many inline macros. It is also not clear when the names are added as it looks like these are added when a provider is first used.
A pkcs11-spy shows the public key is read from card by OpenSC (which is using same version of OpenSSL as provider)
(gdb) run
Starting program: /opt/ossl-dev/bin/openssl pkey -in pkcs11:type=public\;id=%01\;pin-value=123456 -pubin -pubout -out /tmp/xxx.pub
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
../openssl/crypto/evp/evp_fetch.c:112: OpenSSL internal error: Assertion failed: name_id > 0 && name_id <= METHOD_ID_NAME_MAX
Program received signal SIGABRT, Aborted.
__pthread_kill_implementation (no_tid=0, signo=6, threadid=140737352845120) at ./nptl/pthread_kill.c:44
44 ./nptl/pthread_kill.c: No such file or directory.
(gdb) where
#0 __pthread_kill_implementation (no_tid=0, signo=6, threadid=140737352845120) at ./nptl/pthread_kill.c:44
#1 __pthread_kill_internal (signo=6, threadid=140737352845120) at ./nptl/pthread_kill.c:78
#2 __GI___pthread_kill (threadid=140737352845120, signo=signo@entry=6) at ./nptl/pthread_kill.c:89
#3 0x00007ffff7442476 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
#4 0x00007ffff74287f3 in __GI_abort () at ./stdlib/abort.c:79
#5 0x00007ffff7a73533 in OPENSSL_die (message=0x7ffff7cc5608 "Assertion failed: name_id > 0 && name_id <= METHOD_ID_NAME_MAX",
file=0x7ffff7cc55e0 "../openssl/crypto/evp/evp_fetch.c", line=112) at ../openssl/crypto/cryptlib.c:260
#6 0x00007ffff7a3dca9 in ossl_assert_int (expr=0, exprstr=0x7ffff7cc5608 "Assertion failed: name_id > 0 && name_id <= METHOD_ID_NAME_MAX",
file=0x7ffff7cc55e0 "../openssl/crypto/evp/evp_fetch.c", line=112) at ../openssl/include/internal/common.h:28
#7 0x00007ffff7a3de30 in evp_method_id (name_id=0, operation_id=10) at ../openssl/crypto/evp/evp_fetch.c:112
#8 0x00007ffff7a3e5a6 in inner_evp_generic_fetch (methdata=0x7fffffffd430, prov=0x555555690470, operation_id=10,
name=0x7ffff7ebadd8 "RSA:rsaEncryption:1.2.840.113549.1.1.1", properties=0x0, new_method=0x7ffff7a4b518 <keymgmt_from_algorithm>,
up_ref_method=0x7ffff7a4bc30 <EVP_KEYMGMT_up_ref>, free_method=0x7ffff7a4bc90 <EVP_KEYMGMT_free>) at ../openssl/crypto/evp/evp_fetch.c:323
#9 0x00007ffff7a3e7a6 in evp_generic_fetch (libctx=0x0, operation_id=10, name=0x7ffff7ebadd8 "RSA:rsaEncryption:1.2.840.113549.1.1.1", properties=0x0,
new_method=0x7ffff7a4b518 <keymgmt_from_algorithm>, up_ref_method=0x7ffff7a4bc30 <EVP_KEYMGMT_up_ref>, free_method=0x7ffff7a4bc90 <EVP_KEYMGMT_free>)
at ../openssl/crypto/evp/evp_fetch.c:364
#10 0x00007ffff7a4bc2a in EVP_KEYMGMT_fetch (ctx=0x0, algorithm=0x7ffff7ebadd8 "RSA:rsaEncryption:1.2.840.113549.1.1.1", properties=0x0)
at ../openssl/crypto/evp/keymgmt_meth.c:221
#11 0x00007ffff7bc57ca in try_key_ref (data=0x7fffffffd620, ctx=0x555555693a20, provider=0x555555690a20, libctx=0x0, propq=0x0)
at ../openssl/crypto/store/store_result.c:200
#12 0x00007ffff7bc5e9d in try_key (data=0x7fffffffd620, v=0x7fffffffd7f0, ctx=0x555555693a20, provider=0x555555690a20, libctx=0x0, propq=0x0)
at ../openssl/crypto/store/store_result.c:398
#13 0x00007ffff7bc54ed in ossl_store_handle_load_result (params=0x7fffffffd710, arg=0x7fffffffd7f0) at ../openssl/crypto/store/store_result.c:134
--Type <RET> for more, q to quit, c to continue without paging--
#14 0x00007ffff7eb2073 in p11prov_store_load (pctx=0x5555556a0540, object_cb=0x7ffff7bc5207 <ossl_store_handle_load_result>, object_cbarg=0x7fffffffd7f0,
pw_cb=0x7ffff7a832a1 <ossl_pw_passphrase_callback_dec>, pw_cbarg=0x555555693a68) at ../../src/src/store.c:493
#15 0x00007ffff7bc1fcd in OSSL_STORE_load (ctx=0x555555693a20) at ../openssl/crypto/store/store_lib.c:432
#16 0x000055555561085e in load_key_certs_crls (uri=0x7fffffffe1dd "pkcs11:type=public;id=%01;pin-value=123456", format=0, maybe_stdin=1, pass=0x0,
desc=0x5555556389c2 "Public Key", ppkey=0x0, ppubkey=0x7fffffffda08, pparams=0x0, pcert=0x0, pcerts=0x0, pcrl=0x0, pcrls=0x0)
at ../openssl/apps/lib/apps.c:987
#17 0x000055555560f615 in load_pubkey (uri=0x7fffffffe1dd "pkcs11:type=public;id=%01;pin-value=123456", format=0, maybe_stdin=1, pass=0x0, e=0x0,
desc=0x5555556389c2 "Public Key") at ../openssl/apps/lib/apps.c:606
#18 0x00005555555d5d90 in pkey_main (argc=7, argv=0x7fffffffde00) at ../openssl/apps/pkey.c:216
#19 0x00005555555cd53c in do_cmd (prog=0x555555692460, argc=7, argv=0x7fffffffde00) at ../openssl/apps/openssl.c:418
#20 0x00005555555cd0c6 in main (argc=7, argv=0x7fffffffde00) at ../openssl/apps/openssl.c:298
To Reproduce
Run the test script below.
Expected behavior
A clear and concise description of what you expected to happen.
Operating environment (please complete the following information):
Token and application used (please complete the following information):
Additional context
test script:
#!/bin/sh
#set -xv
OSSL=/opt/ossl-dev
# debug pkcs11-provider
export PKCS11_PROVIDER_DEBUG="file:/tmp/pp-debug"
export OPENSSL_CONF=$OSSL/ssl/openssl-pp.cnf
# Incase we are using pkcs11-spy.so in openssl-pp.cnf:
export PKCS11SPY=$OSSL/lib/opensc-pkcs11.so
export PKCS11SPY_OUTPUT=/tmp/pkcs11-spy.log
# may be in opensc.conf
#export OPENSC_DEBUG=3
gdb --args $OSSL/bin/openssl pkey \
-in "pkcs11:type=public;id=%01;pin-value=123456" \
-pubin -pubout -out /tmp/xxx.pub
diff of changes added to openssl.cnf
+++ openssl-pp.cnf 2022-11-06 06:53:00.983719361 -0600
@@ -13,7 +13,7 @@
# defined.
HOME = .
-# Use this in order to automatically load providers.
+ # Use this in order to automatically load providers.
openssl_conf = openssl_init
# Comment out the next line to ignore configuration errors
@@ -60,6 +60,20 @@
# included fipsmodule.cnf.
# fips = fips_sect
+base = base_sect
+pkcs11 = pkcs11_sect
+
+[base_sect]
+activate = 1
+
+[pkcs11_sect]
+module = /opt/ossl-dev/lib/pkcs11_provider.so
+#pkcs11-module-path = /opt/ossl-dev/lib/opensc-pkcs11.so
+pkcs11-module-path = /opt/ossl-dev/lib/pkcs11-spy.so
+pkcs11-module-allow-export = 1
+activate = 1
+
+
# If no providers are activated explicitly, the default one is activated implicitly.
# See man 7 OSSL_PROVIDER-default for more details.
#
@@ -69,7 +83,7 @@
# OpenSSL may not work correctly which could lead to significant system
# problems including inability to remotely access the system.
[default_sect]
-# activate = 1
+ activate = 1
Originally reported in #77 as I noticed this during the copr build:
To Reproduce
Steps to reproduce the behavior:
make check
on s390x machine (with different endianness than x84_64)## Sign and Verify with provided Hash and RSA
openssl dgst -sha256 -binary -out ${TMPPDIR}/sha256.bin ${SEEDFILE}
openssl pkeyutl -sign -inkey "${PRIURI}" -in ${TMPPDIR}/sha256.bin -out ${TMPPDIR}/sha256-sig.bin
openssl pkeyutl -verify -inkey "${PUBURI}" -pubin -in ${TMPPDIR}/sha256.bin -sigfile ${TMPPDIR}/sha256-sig.bin
000003FFB0772720:error:0200008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding:crypto/rsa/rsa_pk1.c:75:
000003FFB0772720:error:02000072:rsa routines:rsa_ossl_public_decrypt:padding check failed:crypto/rsa/rsa_ossl.c:599:
Signature Verification Failure
FAIL test.sh (exit status: 1)
Expected behavior
all checks pass
Operating environment (please complete the following information):
Token and application used (please complete the following information):
Additional context
The comment in #77 (comment) shows the potential source or problem, where the endianness is changed and I did not find this mentioned in the pkcs11 specifications, but did not get back to check it.
Some infrastructure is already present but there are no keymgmt or signature operations exported.
In setups with multiple slots/tokens, openssl key commands (pkey
and pkeyutl
) fails with the following error message:
OPENSSL_CONF=ossl_ock.conf openssl pkey -in [${key}](pkcs11:object=test_rsa1k:pub;type=public -pubin -pubout -out key.pub
Could not read key of Public Key from pkcs11:object=test_rsa1k:pub;type=public
802BE023C27F0000:error:40800005:pkcs11:p11prov_session_free:General Error:session.c:304:Potential double free in the calling code
The problem occured in a setup wiht multiple slots. Ma test setup (opencryptoki) has one slot with a sw-token and another slot without a a token (CKF_TOKEN_PRESENT not set). If I remove the second (empty) slot, the openssl command terminates correctly.
/usr/local/etc/opencryptoki/opencryptoki.conf
slot 0
{
stdll = libpkcs11_sw.so
tokversion = 3.12
}
slot 1
{
stdll = libep11.so
}
ossl_ock.conf
HOME = .
# Use this in order to automatically load providers.
openssl_conf = openssl_init
config_diagnostics = 1
[openssl_init]
providers = provider_sect
[provider_sect]
default = default_sect
pkcs11 = pkcs11_sect
base = base_sect
[base_sect]
activate = 1
[default_sect]
activate = 1
[pkcs11_sect]
module = /path/to/pkcs11_provider.so
pkcs11-module-path = libopencryptoki.so.0
pkcs11-module-token-pin = file:/path/to/pinfile.txt
activate = 1
It looks like the store_load()
function iterates over all slots for finding keys. At the beginnig of each iteration, the session should be reset by calling p11prov_session_free()
for the session from the previous iteration. This causes the error later, if the store is closed (p11prov_store_close
) and all sessions of the context are free-ed.
Backtrace of the first p11prov_session_free()
:
Thread 1 "openssl" hit Breakpoint 1, p11prov_session_free (session=0x5555556b0ad0) at session.c:291
291 P11PROV_debug("Session Free %p", session);
(gdb) bt
#0 p11prov_session_free (session=0x5555556b0ad0) at session.c:291
#1 0x00007ffff7871c34 in store_load (ctx=0x555555660430, pw_cb=0x0, pw_cbarg=0x0) at store.c:245
#2 0x00007ffff7871e1d in p11prov_store_open (pctx=0x5555556596b0, uri=0x7fffffffe5d2 "pkcs11:object=test_rsa1k:prv;type=private")
at store.c:311
#3 0x00007ffff7d32c69 in OSSL_STORE_open_ex () from /lib/x86_64-linux-gnu/libcrypto.so.3
#4 0x00005555555ef44f in ?? ()
#5 0x00005555555f00e3 in ?? ()
#6 0x00005555555f062b in ?? ()
#7 0x00005555555c1d5e in ?? ()
#8 0x00005555555ba151 in ?? ()
#9 0x0000555555596351 in ?? ()
#10 0x00007ffff78ae20a in __libc_start_call_main (main=main@entry=0x555555596190, argc=argc@entry=12, argv=argv@entry=0x7fffffffe2d8)
at ../sysdeps/nptl/libc_start_call_main.h:58
#11 0x00007ffff78ae2bc in __libc_start_main_impl (main=0x555555596190, argc=12, argv=0x7fffffffe2d8, init=<optimized out>,
fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fffffffe2c8) at ../csu/libc-start.c:389
#12 0x0000555555596481 in ?? ()
Backtrace of the second p11prov_session_free()
:
Thread 1 "openssl" hit Breakpoint 1, p11prov_session_free (session=0x5555556b0ad0) at session.c:291
291 P11PROV_debug("Session Free %p", session);
(gdb) bt
#0 p11prov_session_free (session=0x5555556b0ad0) at session.c:291
#1 0x00007ffff787189b in p11prov_store_ctx_free (ctx=0x555555660430) at store.c:184
#2 0x00007ffff7872417 in p11prov_store_close (pctx=0x555555660430) at store.c:462
#3 0x00007ffff7d32886 in ?? () from /lib/x86_64-linux-gnu/libcrypto.so.3
#4 0x00007ffff7d3364e in OSSL_STORE_close () from /lib/x86_64-linux-gnu/libcrypto.so.3
#5 0x00005555555ef62f in ?? ()
#6 0x00005555555f00e3 in ?? ()
#7 0x00005555555f062b in ?? ()
#8 0x00005555555c1d5e in ?? ()
#9 0x00005555555ba151 in ?? ()
#10 0x0000555555596351 in ?? ()
#11 0x00007ffff78ae20a in __libc_start_call_main (main=main@entry=0x555555596190, argc=argc@entry=12, argv=argv@entry=0x7fffffffe2d8)
at ../sysdeps/nptl/libc_start_call_main.h:58
#12 0x00007ffff78ae2bc in __libc_start_main_impl (main=0x555555596190, argc=12, argv=0x7fffffffe2d8, init=<optimized out>,
fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fffffffe2c8) at ../csu/libc-start.c:389
#13 0x0000555555596481 in ?? ()
Before digging deeper into the session handling: has someone else tested the pkcs11-provider successfully in a multi-slot or multi-token environment?
At least in some soft token a token key is really slow because it is stored encrypted on disk, so at each use it would have to be recovered with an expensive operation.
To speed up things for application that hold a private key structure we should copy the key to a session key object (this causes the soft toke to cache the private key) on the first operation that actually sues the private key and hold both handles on the key object.
This means the key objects will need to hold a session.
Describe the bug
The PKCS#11 modules should be generally usable before login at least for listing of the keys. The NSS is taking this to the extreme, asking pin before any operation, which is OK, according to the standards, but painful for the users [1][2].
For this, the profile objects were invented in PKCS#11 3.0, but there are many PKCS#11 modules not supporting this yet a lot of software that is using OpenSSL and will be expecting this behavior [libssh].
To Reproduce
Steps to reproduce the behavior:
openssl.cnf
pkey -in $PUBURI -pubin -pubout -out ${TSTOUT}.pub
Expected behavior
The applications should be allowed to list public objects without being prompted for a PIN and without requiring the pin in the configuration file.
Operating environment (please complete the following information):
Token and application used (please complete the following information):
Additional context
The pkcs11 provider should try to list the objects on the card even without login. The CKF_LOGIN_REQUIRED
flag says only
True if there are some cryptographic functions that a user MUST be logged in to perform
so this does not mean that the user needs to be logged in to do ANY operation on the module.
An reproduced and ugly solution that solved the issue for me is available in the follwing branch (but it will probably need some more work to work in all corner cases
https://github.com/Jakuje/pkcs11-provider/tree/login (main...Jakuje:login)
Describe the feature
On initialization, the slots without tokens should be avoided
The code that extract public key info can fallback to pull that information out of a certificate if there is no corresponding public key object in the token.
Unfortunately it is not easy to add a certificate w/o pkcs12-util also covertly extracting and adding a public key object to softokn.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.