Code Monkey home page Code Monkey logo

asn1crypto's Introduction

asn1crypto

A fast, pure Python library for parsing and serializing ASN.1 structures.

GitHub Actions CI CircleCI PyPI

Sponsorship

A quick thank you to all of the sponsors who donate to support the asn1crypto project:

Corporate

Personal

  • Nothing4You
  • sthagen

Features

In addition to an ASN.1 BER/DER decoder and DER serializer, the project includes a bunch of ASN.1 structures for use with various common cryptography standards:

Standard Module Source
X.509 asn1crypto.x509 RFC 5280
CRL asn1crypto.crl RFC 5280
CSR asn1crypto.csr RFC 2986, RFC 2985
OCSP asn1crypto.ocsp RFC 6960
PKCS#12 asn1crypto.pkcs12 RFC 7292
PKCS#8 asn1crypto.keys RFC 5208
PKCS#1 v2.1 (RSA keys) asn1crypto.keys RFC 3447
DSA keys asn1crypto.keys RFC 3279
Elliptic curve keys asn1crypto.keys SECG SEC1 V2
PKCS#3 v1.4 asn1crypto.algos PKCS#3 v1.4
PKCS#5 v2.1 asn1crypto.algos PKCS#5 v2.1
CMS (and PKCS#7) asn1crypto.cms RFC 5652, RFC 2315
TSP asn1crypto.tsp RFC 3161
PDF signatures asn1crypto.pdf PDF 1.7

Why Another Python ASN.1 Library?

Python has long had the pyasn1 and pyasn1_modules available for parsing and serializing ASN.1 structures. While the project does include a comprehensive set of tools for parsing and serializing, the performance of the library can be very poor, especially when dealing with bit fields and parsing large structures such as CRLs.

After spending extensive time using pyasn1, the following issues were identified:

  1. Poor performance
  2. Verbose, non-pythonic API
  3. Out-dated and incomplete definitions in pyasn1-modules
  4. No simple way to map data to native Python data structures
  5. No mechanism for overridden universal ASN.1 types

The pyasn1 API is largely method driven, and uses extensive configuration objects and lowerCamelCase names. There were no consistent options for converting types of native Python data structures. Since the project supports out-dated versions of Python, many newer language features are unavailable for use.

Time was spent trying to profile issues with the performance, however the architecture made it hard to pin down the primary source of the poor performance. Attempts were made to improve performance by utilizing unreleased patches and delaying parsing using the Any type. Even with such changes, the performance was still unacceptably slow.

Finally, a number of structures in the cryptographic space use universal data types such as BitString and OctetString, but interpret the data as other types. For instance, signatures are really byte strings, but are encoded as BitString. Elliptic curve keys use both BitString and OctetString to represent integers. Parsing these structures as the base universal types and then re-interpreting them wastes computation.

asn1crypto uses the following techniques to improve performance, especially when extracting one or two fields from large, complex structures:

  • Delayed parsing of byte string values
  • Persistence of original ASN.1 encoded data until a value is changed
  • Lazy loading of child fields
  • Utilization of high-level Python stdlib modules

While there is no extensive performance test suite, the CRLTests.test_parse_crl test case was used to parse a 21MB CRL file on a late 2013 rMBP. asn1crypto parsed the certificate serial numbers in just under 8 seconds. With pyasn1, using definitions from pyasn1-modules, the same parsing took over 4,100 seconds.

For smaller structures the performance difference can range from a few times faster to an order of magnitude or more.

Related Crypto Libraries

asn1crypto is part of the modularcrypto family of Python packages:

Current Release

1.5.1 - changelog

Dependencies

Python 2.6, 2.7, 3.2, 3.3, 3.4, 3.5, 3.6, 3.7, 3.8, 3.9, 3.10, 3.11, 3.12 or pypy. No third-party packages required.

Installation

pip install asn1crypto

License

asn1crypto is licensed under the terms of the MIT license. See the LICENSE file for the exact license text.

Security Policy

The security policies for this project are covered in SECURITY.md.

Documentation

The documentation for asn1crypto is composed of tutorials on basic usage and links to the source for the various pre-defined type classes.

Tutorials

Reference

Continuous Integration

Various combinations of platforms and versions of Python are tested via:

Testing

Tests are written using unittest and require no third-party packages.

Depending on what type of source is available for the package, the following commands can be used to run the test suite.

Git Repository

When working within a Git working copy, or an archive of the Git repository, the full test suite is run via:

python run.py tests

To run only some tests, pass a regular expression as a parameter to tests.

python run.py tests ocsp

PyPi Source Distribution

When working within an extracted source distribution (aka .tar.gz) from PyPi, the full test suite is run via:

python setup.py test

Package

When the package has been installed via pip (or another method), the package asn1crypto_tests may be installed and invoked to run the full test suite:

pip install asn1crypto_tests
python -m asn1crypto_tests

Development

To install the package used for linting, execute:

pip install --user -r requires/lint

The following command will run the linter:

python run.py lint

Support for code coverage can be installed via:

pip install --user -r requires/coverage

Coverage is measured by running:

python run.py coverage

To change the version number of the package, run:

python run.py version {pep440_version}

To install the necessary packages for releasing a new version on PyPI, run:

pip install --user -r requires/release

Releases are created by:

  • Making a git tag in PEP 440 format

  • Running the command:

    python run.py release

Existing releases can be found at https://pypi.org/project/asn1crypto/.

CI Tasks

A task named deps exists to download and stage all necessary testing dependencies. On posix platforms, curl is used for downloads and on Windows PowerShell with Net.WebClient is used. This configuration sidesteps issues related to getting pip to work properly and messing with site-packages for the version of Python being used.

The ci task runs lint (if flake8 is available for the version of Python) and coverage (or tests if coverage is not available for the version of Python). If the current directory is a clean git working copy, the coverage data is submitted to codecov.io.

python run.py deps
python run.py ci

asn1crypto's People

Contributors

aalba6675 avatar daa avatar davidben avatar daxtens avatar dpward avatar edwardbetts avatar eukaryote avatar geitda avatar hroncok avatar isidroas avatar jdufresne avatar jjackson-llb avatar joernheissler avatar laurivosandi avatar luckylud avatar matthiasvalvekens avatar mneundorfer avatar mttcpr avatar nkostoulas avatar psagers avatar qha avatar rollandf avatar ryanguest avatar space88man avatar strawberryjuice avatar thierryba avatar tiran avatar vinas003 avatar wbond avatar wiml avatar

Stargazers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Forkers

eukaryote zed spaceone nresare anymaster dvincent1 wfranzini joernheissler rscabell athoms ofek aalba6675 darg0001 damz ralt kak-bo-che sreekanthpulagam zdw ryanguest tiran edwardbetts mttcpr erny heri16 gmg2719 plaes reox chubbymaggie jolaf skelsec luckylud edhodapp murgeye whigg wiml rollandf krash84 mr-tan4 jjackson-llb vpiserchia wideglide nkostoulas nickray shuxin yaggyhub space88man trusttoken ayushev zzazzles mskalski tzoiker antoniivanov oshixingxing commonism sakshisharmapuresoftware ossdev07 ricky-sb ladaan janin guygoldenberg lharing de117 cambridge-blockchain hroncok pypi-buildability-project coinlegit matt-hayden lucysweetwork daxtens rideyourstyle oscarcreativeshop shoukato asellappen haykbaluyan auspex-labs google-autofuzz joye1503 joyofhex digitalderrickcs rwaycachedlibs atlantisfox matthiasvalvekens cronokirby tarker genisysram mneundorfer christopherl91 shotgun-20 trendingtechnology davidben arthurfgb toonsegers evasadov rmfought decart lardinator isidroas frankfanslc infotecs binref

asn1crypto's Issues

asn1crypto raise exception parsing indefinite length

Hi,

as showed in the test case addedd with #14 asn1crypto explodes parsing a .p7m file with indefinite lengths:

ValueError: Insufficient data - 57904015269414514108800405237541285051975119636830231669355043598084451101117 bytes requested but only 57362 available

Add OpenSSL TRUSTED CERTIFICATE ASN.1 Definition

OpenSSL can add trust info to trust roots by encoding an ANS.1 structure of x509.Certificate with the following sequence added as a final field:

TrustedCertificate ::= SEQUENCE {
    trust SEQUENCE OF OBJECT IDENTIFIER OPTIONAL,
    reject [0] IMPLICIT SEQUENCE OF OBJECT IDENTIFIER OPTIONAL,
    alias UTF8String OPTIONAL,
    keyid OCTET STRING OPTIONAL,
    other [1] IMPLICIT SEQUENCE OF AlgorithmIdentifier OPTIONAL
}

This will be useful in making oscrypto.trust_list more accurate.

Support of CAdES RFC 5126

Although CMS is supported in the last standard RFC 5652, the former CAdES RFC 5126 has additional fields like the explicit policy (EPES).

External signing of CSR's and Certificates

I would like to keep my private key in an external HSM. However, there is no way to use CSRBuilder or CertificateBuilder.build without providing a private key which is then used by oscrypto. It would be great if a function could be provided (either to build() or as a property of the object) which takes the data to be signed (ie. the bytes of the serialized cert/csr) and returns just the bytes for the signature. That custom function would probably know how to hash it, but perhaps that would be an argument as well.

BTW: Love your library, first class code!

LDAP or HTTPS can be used for OCSP

It is a rare case but LDAP and HTTPS are used for OCSP. In the code, only http is considered to be used for OCSP.

Here is one of the examples.

Authority Information Access:
                OCSP - URI:http://pki.aloaha.com/CertEnroll/pki.aloaha.com_aloaha.crt
                OCSP - URI:ldap://pki.aloaha.com/CN=aloaha,CN=AIA,CN=Public%20Key%20Services,CN=Services,CN=Configuration,DC=aloaha,DC=com?cACertificate?base?objectClass=certificationAuthority

Another example for https

https://ocsp.verisign.com

asn1crypto/asn1crypto/x509.py

Line 2380 in 2068a8a

if url.lower()[0:7] == 'http://':

Create a asn1crypto-testdata pypi package + automate release

Trying to run the unittests in my environment I could not get them to work:

[cooper:~/asn1crypto-0.22.0:]$ ~/virtualenvs/setup_py_fun/bin/python3 setup.py test
running test
running egg_info
writing asn1crypto.egg-info/PKG-INFO
writing dependency_links to asn1crypto.egg-info/dependency_links.txt
writing top-level names to asn1crypto.egg-info/top_level.txt
reading manifest file 'asn1crypto.egg-info/SOURCES.txt'
writing manifest file 'asn1crypto.egg-info/SOURCES.txt'
running build_ext
tests (unittest.loader._FailedTest) ... ERROR

======================================================================
ERROR: tests (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: tests
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/unittest/loader.py", line 153, in loadTestsFromName
    module = __import__(module_name)
ModuleNotFoundError: No module named 'tests'


----------------------------------------------------------------------
Ran 1 test in 0.000s

FAILED (errors=1)
Test failed: <unittest.runner.TextTestResult run=1 errors=1 failures=0>
error: Test failed: <unittest.runner.TextTestResult run=1 errors=1 failures=0>

Playing with setup.py now.

AttributeError: 'Sequence' object has no attribute '__bytes__' / '__unicode__'

print() used on a Sequence will raise an exception.

Is this a code bug or a documentation issue?

e.g. are subclasses of Sequence meant to implement __bytes__ and __unicode__ ?

I notice that subclasses of Concat can use its implementation -- though I note that the py2 and py3 will give inconsistent printouts, so I'm not sure if you intended it to be this way or not.

Thanks.

test.py:

#!/usr/bin/env python2
# coding=utf-8
from __future__ import absolute_import, division, print_function
import asn1crypto.core

class MySequence(asn1crypto.core.Sequence):
    _fields = [
        ('abc', asn1crypto.core.PrintableString),
    ]

keyinfo_dict= {'abc': u"1234"}
ki = MySequence(keyinfo_dict)
print(repr(ki.dump()))
print(ki.debug())
print(ki)

output:

$ python 2 ./test.py 
'0\x06\x13\x041234'
  __main__.MySequence Object #140130113838608
    Header: 0x3006
      constructed universal tag 16
    Data: 0x130431323334
      Field "abc"
        asn1crypto.core.PrintableString Object #140130113838736
          Header: 0x1304
            primitive universal tag 19
          Data: 0x31323334
            Native: 1234
None
Traceback (most recent call last):
  File "./test.py", line 15, in <module>
    print(ki)
  File "/usr/local/lib/python2.7/dist-packages/asn1crypto/core.py", line 290, in __str__
    return self.__bytes__()
AttributeError: 'MySequence' object has no attribute '__bytes__'

$ python3 ./test.py 
b'0\x06\x13\x041234'
  __main__.MySequence Object #140594304391152
    Header: 0x3006
      constructed universal tag 16
    Data: 0x130431323334
      Field "abc"
        asn1crypto.core.PrintableString Object #140594304492824
          Header: 0x1304
            primitive universal tag 19
          Data: 0x31323334
            Native: 1234
None
Traceback (most recent call last):
  File "./test.py", line 15, in <module>
    print(ki)
  File "/usr/local/lib/python3.5/dist-packages/asn1crypto/core.py", line 292, in __str__
    return self.__unicode__()
AttributeError: 'MySequence' object has no attribute '__unicode__'

RSAPublicKey is backwards in order of modulus and exponent

OpenSSL doesn't like the order you have the asn1crypto.keys.RSAPublicKey object. Simply switching the order around let's things parse correctly. I overrode with my own class, like so:

`class MyRSAPublicKey(Sequence):

_fields = [
    ('public_exponent', Integer),
    ('modulus', Integer),
]`

Example parsing the RSAPublicKey with it currently:

openssl rsa -pubin -inform PEM -text -noout < test.pub
Modulus (17 bit): 65537 (0x10001)
Exponent:
00:c1:72:a6:ca:26:db:34:1e:8c:27:8d:d4:d6:0f:
cd:05:16:55:95:a1:f0:a8:55:b4:a0:28:b1:db:bf:
c5:df:f4:7b:b4:65:3f:31:5f:04:60:d8:d2:13:3a:
8a:58:93:78:68:a3:95:78:d8:72:0d:21:d5:f7:c3:
a8:ae:7d:3a:ad:71:10:79:cf:8c:32:a3:95:16:92:
6d:63:1a:a3:55:e6:69:9a:37:0a:58:b6:44:dc:fe:
be:39:12:d1:56:ef:15:99:8f:e4:ca:91:30:e8:e6:
f5:fd:59:c5:17:61:53:b4:91:8d:df:53:8a:cc:b6:
4f:7b:c2:19:83:db:76:8b:44:55:85:de:61:5e:f0:
6f:29:8b:8a:70:af:44:16:b3:2b:27:6b:d6:63:e6:
65:81:ed:19:5c:18:12:31:77:64:30:a0:26:6b:fe:
d8:c4:0e:72:f1:26:b4:50:96:d0:e9:b3:8c:c0:bf:
a5:67:d5:5f:fc:cd:be:15:b9:db:6f:4b:eb:f3:85:
bc:5d:ef:82:3f:56:09:cf:e4:3a:e6:f6:97:2b:a6:
17:70:5e:3e:76:f4:89:f7:af:ee:9e:6f:89:f6:0c:
2b:b3:3e:97:af:4a:e7:31:f8:39:3f:14:df:62:de:
01:6b:fa:80:14:9c:9e:da:5f:48:7f:e5:8f:b2:02:
01:bf

Example parsing with the order switched:

openssl rsa -pubin -inform PEM -text -noout < test.pub
Modulus (2048 bit):
00:c1:72:a6:ca:26:db:34:1e:8c:27:8d:d4:d6:0f:
cd:05:16:55:95:a1:f0:a8:55:b4:a0:28:b1:db:bf:
c5:df:f4:7b:b4:65:3f:31:5f:04:60:d8:d2:13:3a:
8a:58:93:78:68:a3:95:78:d8:72:0d:21:d5:f7:c3:
a8:ae:7d:3a:ad:71:10:79:cf:8c:32:a3:95:16:92:
6d:63:1a:a3:55:e6:69:9a:37:0a:58:b6:44:dc:fe:
be:39:12:d1:56:ef:15:99:8f:e4:ca:91:30:e8:e6:
f5:fd:59:c5:17:61:53:b4:91:8d:df:53:8a:cc:b6:
4f:7b:c2:19:83:db:76:8b:44:55:85:de:61:5e:f0:
6f:29:8b:8a:70:af:44:16:b3:2b:27:6b:d6:63:e6:
65:81:ed:19:5c:18:12:31:77:64:30:a0:26:6b:fe:
d8:c4:0e:72:f1:26:b4:50:96:d0:e9:b3:8c:c0:bf:
a5:67:d5:5f:fc:cd:be:15:b9:db:6f:4b:eb:f3:85:
bc:5d:ef:82:3f:56:09:cf:e4:3a:e6:f6:97:2b:a6:
17:70:5e:3e:76:f4:89:f7:af:ee:9e:6f:89:f6:0c:
2b:b3:3e:97:af:4a:e7:31:f8:39:3f:14:df:62:de:
01:6b:fa:80:14:9c:9e:da:5f:48:7f:e5:8f:b2:02:
01:bf
Exponent: 65537 (0x10001)

"parameters" missing when using rsa / rsassa_pkcs1v15

There's a feature in algos.SignedDigestAlgorithm to set parameters to Null when using certain algorithms. "rsassa_pkcs1v15" is missing.
While this is easily fixed, same problem occurs for KeyEncryptionAlgorithm.

I'm wondering why you're using different classes, according to rfc5652 both SignatureAlgorithmIdentifier and KeyEncryptionAlgorithmIdentifier are the same as AlgorithmIdentifier. And rfc4055 defines this "parameters MUST be present and MUST be NULL" constraint for AlgorithmIdentifier.
So shouldn't the logic for Null be inside AlgorithmIdentifier and the other two classes inherit from here?

Error encoding NameConstraints: UnicodeError: label empty or too long

Related to #33:

#!/usr/bin/env python3

from asn1crypto.x509 import NameConstraints

der = bytes.fromhex(
    '303ea03c300582032e6772300582032e6575300682042e656475300682042e6f7267300581032e6772300581032e6575300681042e656475300681042e6f7267'
)
cs = NameConstraints.load(der)
cs.dump(force=True)

Traceback (most recent call last):
  File "/usr/lib/python3.5/encodings/idna.py", line 165, in encode
    raise UnicodeError("label empty or too long")
UnicodeError: label empty or too long

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "./name.py", line 9, in <module>
    cs.dump(force=True)
  File "/home/joern/env/lib/python3.5/site-packages/asn1crypto-0.21.1-py3.5.egg/asn1crypto/core.py", line 3512, in dump
  File "/home/joern/env/lib/python3.5/site-packages/asn1crypto-0.21.1-py3.5.egg/asn1crypto/core.py", line 3081, in _set_contents
  File "/home/joern/env/lib/python3.5/site-packages/asn1crypto-0.21.1-py3.5.egg/asn1crypto/core.py", line 3975, in dump
  File "/home/joern/env/lib/python3.5/site-packages/asn1crypto-0.21.1-py3.5.egg/asn1crypto/core.py", line 3842, in _set_contents
  File "/home/joern/env/lib/python3.5/site-packages/asn1crypto-0.21.1-py3.5.egg/asn1crypto/core.py", line 3512, in dump
  File "/home/joern/env/lib/python3.5/site-packages/asn1crypto-0.21.1-py3.5.egg/asn1crypto/core.py", line 3081, in _set_contents
  File "/home/joern/env/lib/python3.5/site-packages/asn1crypto-0.21.1-py3.5.egg/asn1crypto/core.py", line 1107, in dump
  File "/home/joern/env/lib/python3.5/site-packages/asn1crypto-0.21.1-py3.5.egg/asn1crypto/core.py", line 1483, in dump
  File "/home/joern/env/lib/python3.5/site-packages/asn1crypto-0.21.1-py3.5.egg/asn1crypto/core.py", line 1556, in set
UnicodeError: encoding with 'idna' codec failed (UnicodeError: label empty or too long)

  0  62: SEQUENCE {
  2  60: . [0] {
  4   5: . . SEQUENCE {
  6   3: . . . [2] 2E 67 72                   .gr
       : . . . }
 11   5: . . SEQUENCE {
 13   3: . . . [2] 2E 65 75                   .eu
       : . . . }
 18   6: . . SEQUENCE {
 20   4: . . . [2] '.edu'
       : . . . }
 26   6: . . SEQUENCE {
 28   4: . . . [2] '.org'
       : . . . }
 34   5: . . SEQUENCE {
 36   3: . . . [1] 2E 67 72                   .gr
       : . . . }
 41   5: . . SEQUENCE {
 43   3: . . . [1] 2E 65 75                   .eu
       : . . . }
 48   6: . . SEQUENCE {
 50   4: . . . [1] '.edu'
       : . . . }
 56   6: . . SEQUENCE {
 58   4: . . . [1] '.org'
       : . . . }
       : . . }
       : . }

Performance comparison for asn1 decode of SNMP

This is not really an issue - it's more by way of informational/comparison, given the focus on performance.

We use the ancient https://github.com/jpwarren/libsnmp for a local/bespoke high-throughput SNMP poller. I've recently tried using pysnmp, which had lower performance, apparently due to pyasn1 - see etingof/pysnmp#44

I recently came across asn1crypto and decided to try and build a simple SNMP encoder/decoder and compare performance. It seems that asn1crypto is faster than pysnmp/pyasn1 but still slower than the libsnmp code.

This is in many ways an unfair comparison as libsnmp is a very constrained application and can avoid being completely general - but I thought it was worth raising here in case it reveals any opportunities for optimisation.

The test script I use can be found here:

https://gist.github.com/philmayers/67b9300d8fb7282481a1a6af5ed45818

Some quick results from my box

cpython2.7

timeit for libsnmp 1000 iterations per-call 0.366773ms
timeit for pysnmp 1000 iterations per-call 1.913801ms
timeit for asn1crypto 1000 iterations per-call 0.784892ms

pypy 5.4.0 (much larger iteraction count to let JIT kick in)

timeit for libsnmp 100000 iterations per-call 0.026978ms
timeit for pysnmp 100000 iterations per-call 0.192198ms
timeit for asn1crypto 100000 iterations per-call 0.066650ms

Feel free to close this if the info is not of interest - as noted, comparing a very constrained libsnmp to the much more general encoding in asn1crypto is possibly not reasonable/useful.

Thanks for your work on the project - the code is great quality and I really like the API

Trouble parsing OCSP request from StrongSwan

Hi,

I am having trouble parsing OCSP request from StrongSwan. Base64 encoding of the DER request in question is:

MIGVMIGSMFEwTzBNMAkGBSsOAwIaBQAEFBD07/WvCa6V3cvOq9pdq278smVFBBSfIwRKCe7nPWojuAb164jF62AHcAIUTbNhvLwhJrngY6sYzQTi2dSkHMeiPTA7MB0GCSsGAQUFBzABAgQQhaRGOT27I73QpRF4j8XuUjAaBgkrBgEFBQcwAQQEDTALBgkrBgEFBQcwAQE=

Attempting to parse extensions or run .native on the whole thing results in:

ValueError: Insufficient data - 1424303782342441662876496716754552 bytes requested but only 16 available
    while parsing asn1crypto.core.ParsableOctetString
    while parsing asn1crypto.ocsp.TBSRequestExtension
    while parsing asn1crypto.ocsp.TBSRequestExtensions
    while parsing asn1crypto.ocsp.TBSRequest
    while parsing asn1crypto.ocsp.OCSPRequest

Skipping extensions or not using .native on the whole thing works as expected.

traceback!!

from asn1crypto.keys import PublicKeyInfo
Traceback (most recent call last):
File "", line 1, in
File "C:\Python27\lib\site-packages\asn1crypto\keys.py", line 22, in
from ._elliptic_curve import (
File "C:\Python27\lib\site-packages\asn1crypto_elliptic_curve.py", line 51, in
from ._int import inverse_mod
File "C:\Python27\lib\site-packages\asn1crypto_int.py", line 56, in
from ._perf._big_num_ctypes import libcrypto
File "C:\Python27\lib\site-packages\asn1crypto_perf_big_num_ctypes.py", line 31, in
libcrypto_path = find_library('crypto')
File "C:\Python27\lib\ctypes\util.py", line 53, in find_library
fname = os.path.join(directory, name)
File "C:\Python27\lib\ntpath.py", line 85, in join
result_path = result_path + p_path
UnicodeDecodeError: 'ascii' codec can't decode byte 0xd1 in position 1: ordinal not in range(128)

Various exception messages have em-dash which can lead to odd errors

I was getting an error:

*** UnicodeEncodeError: 'ascii' codec can't encode character u'\u2013' in position 108: ordinal not in range(128)

...which turned out to be from asn1crypto/core.py line 3099, trying to re-raise the message:

Can not set a native python value to responder_id, which has the choice type of asn1crypto.ocsp.ResponderId – value must be an instance of Asn1Value

I'm aware that python and unicode at the terminal has a somewhat polarising effect on discussions, but can I suggest that the em-dash in the various exceptions in core.py might be safely replaced with ascii hyphen?

Modify CMS structure

Hi,
Is it possible to take existing CMS structure with 3 recipients, remove one of the recipients and have a new CMS structure with 2 recipients?

BR,
Alexey

Handle TeletexString as iso-8859-1

So I came across a pkcs7 signature that has a x509 common_name that includes a { and was generated my m2crypto and asn1crypto chokes on it.

M2Crypto simply forwards asn1 encoding to openssl and I found https://wiki.openssl.org/index.php/Manual:Req(1)#BUGS which states that OpenSSL encodes TeletexString as ISO-8859-1 (Latin 1). Then I found https://github.com/etingof/pyasn1/blob/master/pyasn1/type/char.py#L241-L250 which kinda verifies that and wondered if we could implement a fallback in asn1crypto that falls back to latin-1 if we run into a decoding problem? Or, maybe even kick out T61 and just use latin-1?

See mozilla/signing-clients#24 the signature in question and a traceback.

Cannot set OID spec field with SetOf Any

I may be misunderstading oid spec but...why doesn't the following code work when the field is SetOf Anys but determined by OID spec to be SetOf BMPString.

(Or how do you actually create Attribute objects by setting fields directly instead of load'ing from existing data?)

Reproducer

# from pkcs12.py
class Attribute(Sequence):
    _fields = [
        ('type', AttributeType),
        ('values', SetOf, {'spec': Any}),
    ]

    _oid_pair = ('type', 'values')
    _oid_specs = {
        'friendly_name': SetOfBMPString,
        'local_key_id': SetOfOctetString,
        'microsoft_csp_name': SetOfBMPString,
    }

class Attributes(SetOf):
    _child_spec = Attribute

>>>from asn1crypto import pkcs12, core
>>>x = pkcs12.Attribute()
>>>x['type'] = 'friendly_name'
>>>## I think x['values'] should now be interpreted as SetOfBMPString, no?
>>>x['values'] = [ 'hello', 'world']
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/venv3/lib/python3.5/site-packages/asn1crypto/core.py", line 2982, in __setitem__
    new_value = self._make_value(field_name, field_spec, value_spec, field_params, value)
  File "/venv3/lib/python3.5/site-packages/asn1crypto/core.py", line 3237, in _make_value
    wrapper = field_spec(value=new_value.dump(), **field_params)
  File "/venv3/lib/python3.5/site-packages/asn1crypto/core.py", line 3582, in __init__
    raise e
  File "/venv3/lib/python3.5/site-packages/asn1crypto/core.py", line 3573, in __init__
    self.__setitem__(index, child)
  File "/v6/venv3/lib/python3.5/site-packages/asn1crypto/core.py", line 3733, in __setitem__
    new_value = self._make_value(value)
  File "/venv3/lib/python3.5/site-packages/asn1crypto/core.py", line 3658, in _make_value
    type_name(self)
ValueError: Can not set a native python value to asn1crypto.core.SetOf where the _child_spec is Any – value must be an instance of Asn1Value
    while constructing asn1crypto.core.SetOf

Error parsing X509

Hey guys
I've found a problem parsing a certificate with oscrypto. It looks like it's a asn1crypto error.

Certificate: HTTPS certificate of https://www.admin.ch

from oscrypto.asymmetric import load_certificate
cert = load_certificate('/home/severin/Downloads/admin.ch.crt')
cert.asn1.native

cert.asn1.native raises AttributeError: 'int' object has no attribute 'decode'
Stacktrace:

Traceback (most recent call last):
File "/home/osboxes/PycharmProjects/StrongMan/strongMan/tests/certificates/test_container.py", line 368, in test_load_cert
cert.asn1.native
File "/usr/local/lib/python3.5/dist-packages/asn1crypto/core.py", line 2846, in native
self._native[name] = child.native
File "/usr/local/lib/python3.5/dist-packages/asn1crypto/core.py", line 2846, in native
self._native[name] = child.native
File "/usr/local/lib/python3.5/dist-packages/asn1crypto/core.py", line 3308, in native
self._native = [child.native for child in self]
File "/usr/local/lib/python3.5/dist-packages/asn1crypto/core.py", line 3308, in
self._native = [child.native for child in self]
File "/usr/local/lib/python3.5/dist-packages/asn1crypto/core.py", line 2846, in native
self._native[name] = child.native
File "/usr/local/lib/python3.5/dist-packages/asn1crypto/core.py", line 1804, in native
self._native = self._parsed[0].native
File "/usr/local/lib/python3.5/dist-packages/asn1crypto/core.py", line 3308, in native
self._native = [child.native for child in self]
File "/usr/local/lib/python3.5/dist-packages/asn1crypto/core.py", line 3308, in
self._native = [child.native for child in self]
File "/usr/local/lib/python3.5/dist-packages/asn1crypto/core.py", line 2846, in native
self._native[name] = child.native
File "/usr/local/lib/python3.5/dist-packages/asn1crypto/core.py", line 829, in native
return self.chosen.native
File "/usr/local/lib/python3.5/dist-packages/asn1crypto/core.py", line 3308, in native
self._native = [child.native for child in self]
File "/usr/local/lib/python3.5/dist-packages/asn1crypto/core.py", line 3308, in
self._native = [child.native for child in self]
File "/usr/local/lib/python3.5/dist-packages/asn1crypto/core.py", line 829, in native
return self.chosen.native
File "/usr/local/lib/python3.5/dist-packages/asn1crypto/core.py", line 1128, in native
self._native = self.unicode()
File "/usr/local/lib/python3.5/dist-packages/asn1crypto/x509.py", line 150, in unicode
return uri_to_iri(self.contents)
File "/usr/local/lib/python3.5/dist-packages/asn1crypto/_iri.py", line 135, in uri_to_iri
port = port.decode('ascii')
AttributeError: 'int' object has no attribute 'decode'

accented dir + unicode str mismatch

Met issue when upgrading paramiko 1.15 to 2.2.1 (Windows10 + Python 2.7.1)

In the sequence the following : "from asn1crypto.algos import DSASignature" entails Unicode Error
The identified cause :
in the $PATH we got the " C:\Users\dom\Vic clé USB\Downloads\cmder_mini\bin;" Dir.
when the instruction libcrypto_path = find_library('crypto') is executed we have the previous Dir (str) + 'crypto' coded as unicode involving the error in 'fname = os.path.join(directory, name)'

Wishing not to modify the code the bypass was to change directory's name

Traceback (most recent call last):
File "", line 1, in
File "C:\python271\lib\site-packages\asn1crypto\algos.py", line 23, in
from ._int import fill_width
File "C:\python271\lib\site-packages\asn1crypto_int.py", line 56, in
from ._perf._big_num_ctypes import libcrypto
File "C:\python271\lib\site-packages\asn1crypto_perf_big_num_ctypes.py", line 31, in
libcrypto_path = find_library('crypto')
File "C:\python271\lib\ctypes\util.py", line 54, in find_library
fname = os.path.join(directory, name)
File "C:\python271\lib\ntpath.py", line 85, in join
result_path = result_path + p_path
File "C:\python271\lib\encodings\utf_8.py", line 16, in decode
return codecs.utf_8_decode(input, errors, True)
UnicodeDecodeError: 'utf8' codec can't decode byte 0xe9 in position 17: invalid continuation byte

CMS Enveloped Data creation failure

I am trying to create a CMS Enveloped Data using asn1crypto as a substitution of the Google's certificate-transparency source code I forked on my project https://github.com/balena/python-smime. I had the idea of using asn1crypto after having some trouble when parsing indefinite-length attributes generated by certain E-Mail managers (like Thunderbird).

While asn1crypto is great for parsing all kinds of CMS data, I am getting some trouble when generating them. I have an unit test where the PKCS7 output is passed as input to command-line OpenSSL in order to make sure the implementation is compatible. But all I get is an error from OpenSSL when parsing ASN.1 tags (I don't know exactly which one).

There is a single function responsible for returning a CMS ContentInfo structure as follows:

def __get_enveloped_data(pubkey_cipher, sym_cipher, x509_cert,
                         encrypted_key, iv, encrypted_content):
    return cms.ContentInfo({
        'contentType': cms.ContentType(u'enveloped_data'),
        'content': cms.EnvelopedData({
            'version': u'v0',
            'recipient_infos': cms.RecipientInfos([
                cms.RecipientInfo(
                    name='ktri',
                    value=cms.KeyTransRecipientInfo({
                        'version': u'v0',
                        'rid': cms.RecipientIdentifier(
                            name='issuer_and_serial_number',
                            value=__get_issuer_and_serial_number(x509_cert)
                        ),
                        'key_encryption_algorithm': cms.KeyEncryptionAlgorithm({
                            'algorithm': pubkey_cipher.oid,
                            'parameters': core.Null()
                        }),
                        'encrypted_key': encrypted_key
                    })
                )
            ]),
            'encrypted_content_info': cms.EncryptedContentInfo({
                'content_type': cms.ContentType(u'data'),
                'content_encryption_algorithm': cms.EncryptionAlgorithm({
                    'algorithm': sym_cipher.oid,
                    'parameters': iv
                }),
                'encrypted_content': cms.OctetString(
                    encrypted_content, tag=0, tag_type='implicit')
            })
        }, tag=0, tag_type='explicit')
    })

The object returned by the above function is encoded using:

encoded_content = __encode_in_base64(enveloped_data.dump())

And the __encode_in_base64 is just a pretty printer function to convert the DER output into BASE64.

When I execute the OpenSSL function:

$ openssl smime -decrypt -in tmp -inkey private_key.pem

All I get is the following:

Error reading S/MIME message
140702704453280:error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1338:
140702704453280:error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:852:
140702704453280:error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:772:Field=type, Type=PKCS7
140702704453280:error:0D0D106E:asn1 encoding routines:B64_READ_ASN1:decode error:asn_mime.c:193:
140702704453280:error:0D0D40CB:asn1 encoding routines:SMIME_read_ASN1:asn1 parse error:asn_mime.c:528:

Do you have any idea what could be the problem? Is there any caveat when encoding asn1crypto structures like the above?

Cannot create cms.SignedData

I'm trying to create a CMS structure (ContentInfo containing SignedData), but don't know how. The following fails. My mistake or a bug?

#!/usr/bin/python3
from asn1crypto import cms
cms.SignedData({
    'version': 'v1',
    'encap_content_info': {
        'content_type': 'data',
        'content': b'Hello',
    }
})

Error:

Traceback (most recent call last):
  File "sd.py", line 7, in <module>
    'content': b'Hello',
  File "/usr/lib/python3/dist-packages/asn1crypto/core.py", line 2646, in __init__
    raise e
  File "/usr/lib/python3/dist-packages/asn1crypto/core.py", line 2641, in __init__
    self.__setitem__(key, value[key])
  File "/usr/lib/python3/dist-packages/asn1crypto/core.py", line 2786, in __setitem__
    new_value = self._make_value(field_name, field_spec, value_spec, field_params, value)
  File "/usr/lib/python3/dist-packages/asn1crypto/core.py", line 2998, in _make_value
    is_any = issubclass(field_spec, Any)
TypeError: issubclass() arg 1 must be a class
    while constructing asn1crypto.cms.SignedData

field_spec is None in core.py:2998.
I assume it comes from SignedData._fields where it says None for encap_content_info.

Work Around Python Limitation of Year 0

ASN.1 supports year 0 to 9999, however Python only supports 1 to 9999.

I think it would be nice to add some comparison functionality to core.GeneralizedTime so that comparison for the sake of validation can be done without converting to a datetime.datetime object and raising a ValueError.

Possibly also add __str__()/__unicode__() to core.GeneralizedTime to return an ISO 8601 formatted string since we can easily pull that off, even for year 0.

SCEP support?

Hello, first of all thanks for usable Python modules. It made it reasonably simple to implement SCEP stuff in few evenings. Feel free to copy-paste and generalize, this should make your life significanlty easier:

laurivosandi/certidude@5ae872e

I'll be uploading SCEP client side code in next days. The server side seems to be usable with SSCEP at least.

Truncated data raises an IndexError, rather than a ValueError

I didn't see this documented, so I infer that it's probably a bug:

>>>> from asn1crypto.algos import DSASignature
>>>> DSASignature.load("\x30")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/alex_gaynor/.virtualenvs/cryptography/site-packages/asn1crypto/core.py", line 229, in load
    value, _ = _parse_build(encoded_data, spec=spec, spec_params=kwargs, strict=strict)
  File "/Users/alex_gaynor/.virtualenvs/cryptography/site-packages/asn1crypto/core.py", line 4851, in _parse_build
    info, new_pointer = _parse(encoded_data, encoded_len, pointer)
  File "/Users/alex_gaynor/.virtualenvs/cryptography/site-packages/asn1crypto/parser.py", line 151, in _parse
    length_octet = ord(encoded_data[pointer]) if _PY2 else encoded_data[pointer]
IndexError: string index out of range
>>>> DSASignature.load(b"\x30\x03\x02\x00\x02", strict=True).native
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/alex_gaynor/.virtualenvs/cryptography/site-packages/asn1crypto/core.py", line 3434, in native
    self._parse_children(recurse=True)
  File "/Users/alex_gaynor/.virtualenvs/cryptography/site-packages/asn1crypto/core.py", line 3281, in _parse_children
    parts, child_pointer = _parse(self._contents, contents_length, pointer=child_pointer)
  File "/Users/alex_gaynor/.virtualenvs/cryptography/site-packages/asn1crypto/parser.py", line 151, in _parse
    length_octet = ord(encoded_data[pointer]) if _PY2 else encoded_data[pointer]
IndexError: string index out of range

More incorrect cRLDistributionPoints encodings

I'm really sorry, but I found more (possible) issues :-)
I took a list of 157 (trusted) root certs, decoded + reencoded them with asn1crypto.

5 / 157 certs failed.

3 of them had issues with incorrect cRLDistributionPoints encoding (Similar but probably not same as #32)

#!/usr/bin/env python3

import traceback
import requests
from asn1crypto import pem, x509

certs = requests.get('https://mkcert.org/generate/').content
failed = 0

for i, (type_name, headers, der_bytes) in enumerate(pem.unarmor(certs, multiple=True)):
    if type_name != 'CERTIFICATE':
        print('Skipping ' + type_name)
    cert = x509.Certificate.load(der_bytes)
    try:
        dumped = cert.dump(force=True)
        if cert.dump(force=True) != der_bytes:
            with open('%03d.dump' % i, 'wb') as fp:
                fp.write(dumped)
            raise Exception('Input does not match output')
    except Exception as e:
        failed += 1
        with open('%03d.orig' % i, 'wb') as fp:
            fp.write(der_bytes)
        with open('%03d.exc' % i, 'wt') as fp:
            traceback.print_exc(file=fp)

print('%d / %d certs failed' % (failed, i + 1))

And just the broken cRLDistributionPoints:

#!/usr/bin/env python3

from asn1crypto.x509 import CRLDistributionPoints
from base64 import b64decode

crldp = [
'''
MIG9MIG6oIG3oIG0hiFodHRwOi8vd3d3LmUtc3ppZ25vLmh1L1Jvb3RDQS5jcmyGgY5sZGFwOi8v
bGRhcC5lLXN6aWduby5odS9DTj1NaWNyb3NlYyUyMGUtU3ppZ25vJTIwUm9vdCUyMENBLE9VPWUt
U3ppZ25vJTIwQ0EsTz1NaWNyb3NlYyUyMEx0ZC4sTD1CdWRhcGVzdCxDPUhVP2NlcnRpZmljYXRl
UmV2b2NhdGlvbkxpc3Q7YmluYXJ5
''', '''
MIHIMIGAoH6gfIZ6bGRhcDovL2RpcmVjdG9yeS5kLXRydXN0Lm5ldC9DTj1ELVRSVVNUJTIwUm9v
dCUyMENsYXNzJTIwMyUyMENBJTIwMiUyMDIwMDksTz1ELVRydXN0JTIwR21iSCxDPURFP2NlcnRp
ZmljYXRlcmV2b2NhdGlvbmxpc3QwQ6BBoD+GPWh0dHA6Ly93d3cuZC10cnVzdC5uZXQvY3JsL2Qt
dHJ1c3Rfcm9vdF9jbGFzc18zX2NhXzJfMjAwOS5jcmw=
''', '''
MIHSMIGHoIGEoIGBhn9sZGFwOi8vZGlyZWN0b3J5LmQtdHJ1c3QubmV0L0NOPUQtVFJVU1QlMjBS
b290JTIwQ2xhc3MlMjAzJTIwQ0ElMjAyJTIwRVYlMjAyMDA5LE89RC1UcnVzdCUyMEdtYkgsQz1E
RT9jZXJ0aWZpY2F0ZXJldm9jYXRpb25saXN0MEagRKBChkBodHRwOi8vd3d3LmQtdHJ1c3QubmV0
L2NybC9kLXRydXN0X3Jvb3RfY2xhc3NfM19jYV8yX2V2XzIwMDkuY3Js
''',
]

for i, dp in enumerate(crldp):
    der = b64decode(dp)

    p = CRLDistributionPoints.load(der)
    with open('dp-%d-orig.der' % i, 'wb') as fp:
        fp.write(der)
    with open('dp-%d-dump.der' % i, 'wb') as fp:
        fp.write(p.dump(force=True))

Assumptions about self signed state

I noticed you are currently using key ids to make assumptions about self signed state in asn1crypto/x509.

asn1crypto/asn1crypto/x509.py

Line 2518 in 16f4ce0

self._self_signed = 'yes'

I would consider replacing that 'yes' with 'maybe' because (1) key IDs are not always correct, and, (2) users would likely assume that self_signed == True means the signature was verified. It'd be safer to bite the CPU bullet and check the signature.

Note that this suggestion is loosely coupled to wbond/oscrypto#21

If you change those yeses to maybes without broadening the oscrypto SignatureError exception cases or otherwise handling those mismatch errors, this code in CertValidator will emit exceptions much more regularly:

    # In the case of "maybe", we have to check the signature
    signature_algo = cert['signature_algorithm'].signature_algo
    hash_algo = cert['signature_algorithm'].hash_algo

    if signature_algo == 'rsassa_pkcs1v15':
        verify_func = asymmetric.rsa_pkcs1v15_verify
    elif signature_algo == 'dsa':
        verify_func = asymmetric.dsa_verify
    elif signature_algo == 'ecdsa':
        verify_func = asymmetric.ecdsa_verify
    else:
        raise PathValidationError(pretty_message(
            '''
            Unable to verify the signature of the certificate since it uses
            the unsupported algorithm %s
            ''',
            signature_algo
        ))

    try:
        key_object = asymmetric.load_certificate(cert)
        verify_func(key_object, cert['signature_value'].native, cert['tbs_certificate'].dump(), hash_algo)
        return True

    except (oscrypto.errors.SignatureError):
        return False

Got errors while parsing

Here is my code:

# -*- coding: utf-8 -*-

'''
Created on Aug 17, 2017

@author: huytn
'''
from asn1crypto import pem, x509
from certvalidator import CertificateValidator, ValidationContext, validate

if __name__ == '__main__':
    trust_roots = []
    with open('ca4.crt', 'rb') as f:
        ders_bytes = f.read()
        if pem.detect(ders_bytes):
            for _, _, der_bytes in pem.unarmor(ders_bytes, multiple=True):
                trust_roots.append(der_bytes)
        else:
            trust_roots.append(ders_bytes)
        f.close()
    
    with open('cert.pem', 'rb') as f:
        end_entity_cert = f.read()
        if pem.detect(end_entity_cert):
            _, _, end_entity_cert = pem.unarmor(end_entity_cert)
        f.close()
    
    context = ValidationContext(trust_roots=trust_roots, allow_fetching=True, ocsp_fetch_params={'hash_algo':u'sha256'}, revocation_mode='require')
    validator = CertificateValidator(end_entity_cert, validation_context=context)
    valid_path = validator.validate_usage(validator._certificate.key_usage_value.native, set(validator._certificate.extended_key_usage_value.native))
    result = validate.validate_path(context, valid_path)

ca4.crt file includes root and CA, below is output

Traceback (most recent call last):
  File "test_ocsp.py", line 32, in <module>
    valid_path = validator.validate_usage(validator._certificate.key_usage_value.native, set(validator._certificate.extended_key_usage_value.native))
  File "/Users/huytn/.pyenv/versions/2.7.13/lib/python2.7/site-packages/certvalidator/__init__.py", line 193, in validate_usage
    self._validate_path()
  File "/Users/huytn/.pyenv/versions/2.7.13/lib/python2.7/site-packages/certvalidator/__init__.py", line 121, in _validate_path
    validate_path(self._context, candidate_path)
  File "/Users/huytn/.pyenv/versions/2.7.13/lib/python2.7/site-packages/certvalidator/validate.py", line 50, in validate_path
    return _validate_path(validation_context, path)
  File "/Users/huytn/.pyenv/versions/2.7.13/lib/python2.7/site-packages/certvalidator/validate.py", line 386, in _validate_path
    end_entity_name_override=end_entity_name_override
  File "/Users/huytn/.pyenv/versions/2.7.13/lib/python2.7/site-packages/certvalidator/validate.py", line 891, in verify_ocsp_response
    ocsp_responses = validation_context.retrieve_ocsps(cert, issuer)
  File "/Users/huytn/.pyenv/versions/2.7.13/lib/python2.7/site-packages/certvalidator/context.py", line 507, in retrieve_ocsps
    self._extract_ocsp_certs(ocsp_response)
  File "/Users/huytn/.pyenv/versions/2.7.13/lib/python2.7/site-packages/certvalidator/context.py", line 534, in _extract_ocsp_certs
    if self.certificate_registry.add_other_cert(other_cert):
  File "/Users/huytn/.pyenv/versions/2.7.13/lib/python2.7/site-packages/certvalidator/registry.py", line 199, in add_other_cert
    if cert.key_identifier:
  File "/Users/huytn/.pyenv/versions/2.7.13/lib/python2.7/site-packages/asn1crypto/x509.py", line 2250, in key_identifier
    if not self.key_identifier_value:
  File "/Users/huytn/.pyenv/versions/2.7.13/lib/python2.7/site-packages/asn1crypto/x509.py", line 1931, in key_identifier_value
    self._set_extensions()
  File "/Users/huytn/.pyenv/versions/2.7.13/lib/python2.7/site-packages/asn1crypto/x509.py", line 1885, in _set_extensions
    setattr(self, attribute_name, extension['extn_value'].parsed)
  File "/Users/huytn/.pyenv/versions/2.7.13/lib/python2.7/site-packages/asn1crypto/core.py", line 3205, in __getitem__
    raise e
ValueError: Insufficient data - 2 bytes requested but only 0 available
    while parsing asn1crypto.core.ParsableOctetString
    while parsing asn1crypto.x509.Extension

Wrong structure for the Name object causes invalid CSR DER

Will,

I noticed this error debugging an issue with the CSR generator library, but it seems to be caused by an error int the way the Subject distinguishing name object is represented in the ASN.1 crypto element Name

In particular, page 114 of rfc5280 (https://tools.ietf.org/html/rfc5280) states that a Name hast the following structure.

Name ::= CHOICE { -- only one possibility for now --
  rdnSequence  RDNSequence }

RDNSequence ::= SEQUENCE OF RelativeDistinguishedName_

DistinguishedName ::=   RDNSequence

RelativeDistinguishedName ::= SET SIZE (1..MAX) OF AttributeTypeAndValue

....

Attribute               ::= SEQUENCE {
      type             AttributeType,
      values    SET OF AttributeValue }
            -- at least one value is required

AttributeType           ::= OBJECT IDENTIFIER

AttributeValue          ::= ANY -- DEFINED BY AttributeType

AttributeTypeAndValue   ::= SEQUENCE {
    type    AttributeType,
    value   AttributeValue }_

This implies that in the generated CSR we should see something like

SEQUENCE
    SET
        SEQUENCE
            OID
            VALUE
    SET
        SEQUENCE
            OID
            VALUE

The subject domain name of the CSR generated by the last version available of CSR generator has the following structure:

SEQUENCE
    SET
        SEQUENCE
            OID
            VALUE
        SEQUENCE
            OID
            VALUE

The issue that we see with this is that based on the field specified and base on the order they appear, the CSR is rejected or not. For instance the following CSR will be accepted because all the IDs are reported in the right order:

-----BEGIN CERTIFICATE REQUEST-----
MIIDGzCCAgUCAQAwgZoxgZcwCQYDVQQGEwJVUzARBgNVBAgMCkNhbGlmb3JuaWEw
FAYDVQQHDA1TYW4gRnJhbmNpc2MwMBcGA1UECgwQbXkgb3JnIDEyMzQ1Njc4OTAm
BgNVBAMMH215aG9zdDEyMzQ1Njc4OS5teWRvbWFpbjEyMy5jb20wIAYJKoZIhvcN
AQkBFhNhYmNAbXlkb21haW4xMjMuY29tMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8A
MIIBCgKCAQEArzqYKehUC4uvhh/i4sN3Q8DqPoAG62ozz9KywKcyDQ6xxx/pbIpz
UVPlxFKPdI+7w+ytbSGyuHe0rqQEBAi2MNQdLISY+9MZ0grOu/cQH7vAXIM8JqZN
DJQSw49ohhuk3rg50rLbA0QpCiQMZ+3IQqPOE11oC0GEQtJQVF9z82sCBhzy3RF3
rw5Ro7KubJiSK+OlYZzqcEC6REUfu5SpTT7kDlMNP+519kktZW7fBFDmXBt+Uz69
flgUZVob/EpHQy/XbhWVJwrYRK6MOHxhhWG/+GyJHPWX5du6G0U5D0ddVFwbi8Qj
hcMkG7/dqD70o1U2GlorOSA/WexB5tVymQIDAQABoD0wOwYJKoZIhvcNAQkOMS4w
LDAqBgNVHREEIzAhgh9teWhvc3QxMjM0NTY3ODkubXlkb21haW4xMjMuY29tMAsG
CSqGSIb3DQEBCwOCAQEAmjMU1J1JINSVc4ClMeVXyUqIt8y9gesht+xMzUePCB9t
4Ax5zjDFGlYUO1z3lgLZaE9Ov46oiMXSU0D9h/HvZZJrxsEd4r8YYSMg/NMIxMVI
IHMeab97bhhIn3g9x2vdfnf3LVOig1N1nEWbLovKQ7f17obXX+bClph3wEuVaYWm
wJOkZyZXfHAawecqdK5LWzREUQNJ4AYpzkvWk77CpFsrFs+wO3puR8DyA2RDYGNG
YnRLAfhzIPHQg1x1lvXXW8lFfxD+eEa9q0SisqZp020BhAZ3hticDdspcUp3lpSa
8Ipuiiokjf7VxYTQMw5ZSOIZqWuTxZIWXs/AgCfbmA==
-----END CERTIFICATE REQUEST-----

The following CSR will be rejected

-----BEGIN CERTIFICATE REQUEST-----
MIIC9zCCAeECAQAwdzF1MAkGA1UEBhMCVVMwEQYDVQQIDApDYWxpZm9ybmlhMBQG
A1UEBwwNU2FuIEZyYW5jaXNjMDAXBgNVBAoMEG15IG9yZyAxMjM0NTY3ODkwJgYD
VQQDDB9teWhvc3QxMjM0NTY3ODkubXlkb21haW4xMjMuY29tMIIBIjANBgkqhkiG
9w0BAQEFAAOCAQ8AMIIBCgKCAQEA1zz1e5+05DtmnXUOVUx7noYmEZ7dd9ja/zA+
YwDx3M0fU9m5UqDKTShUyX/jVCq17Eihv3goQZt4kYSsPRiHsitaA1LndQX5Bnd9
ZrbV3cTEU8qn4djas/V3vNSI8HZ2OOwff+6HHW8fTk04lkMOdWA+QQZAXZRYsix1
01vwmNTIuv9yX6L4QxV3dV5dvkgvgjVlQ1Mik348ycTJP6br1YgSdodEnS8KgzpB
2fm5eKfjmBKKMI8a49pzqSHLsfNY4i/5oUjHiwn3CIpCloUfiSkL10OFG0EfVnE8
Mx7vaz6jjgx5Chupzwy8AFsROHBvMLCBQRtherQ0pwvyHdTPYwIDAQABoD0wOwYJ
KoZIhvcNAQkOMS4wLDAqBgNVHREEIzAhgh9teWhvc3QxMjM0NTY3ODkubXlkb21h
aW4xMjMuY29tMAsGCSqGSIb3DQEBCwOCAQEAApSFaaOFd9meRqsbi7p/SLqp4ynF
me9YZPC/+VLyUMYArcpvCNJOkZNwupr66AXL+CXaLfRUOXSKBjWXtszHwuf2eQ83
Yfli8iD0Idxk6BVqZrekWPuXz7128wJ/sY6e842c0BfDhNgPLhXyaVLIkWHojh26
fwPxVt5tD6yVym6K+KAZQP4yroW7wHqnCyBAA5xnUAoL3ZKsyjwjbOn+4sEUxMqx
ZCciI63pG5jmvACXv49x58lBWrdEM6eUmDyFO/di92kmuAF8ro07uUYba5i2mqrs
Ph4UVmq+/aiF4QENEOEBw6fkZmFLqQbVOSrgMR1in0ustXCoHcO4uuDR6g==
-----END CERTIFICATE REQUEST-----

After a lot of digging in RFC/standards and some really helpful emails with the CA guys it looks like the issue here is that a CSR generated skipping the SET element for each RDNSequence produces a single SET that contains all the SEQUENCEs. When this happens the order the element and their content length matters in order to generate valid DER encoded content.

This happens because the rules to map ASN1 to DER state that a SET OF ASN1 element (PKCS uses SET OF instead of SET) has to contain a list of elements ordered by ascending value of tag (see end of section 5.15 http://luca.ntop.org/Teaching/Appunti/asn1.html). The DER encoding section also explain how to identify the discriminant tag when two elements tags are the same (30 HEX for the SET OF elements) and do the comparison (basically do the lexicographic comparison of the bytes that follow, so in this case the length field)

Looking at the asn1parse of the good CSR we see this

15:d=4  hl=2 l=   9 cons: SEQUENCE          
17:d=5  hl=2 l=   3 prim: OBJECT            :countryName
22:d=5  hl=2 l=   2 prim: PRINTABLESTRING   :US

26:d=4  hl=2 l=  17 cons: SEQUENCE          
28:d=5  hl=2 l=   3 prim: OBJECT            :stateOrProvinceName
33:d=5  hl=2 l=  10 prim: UTF8STRING        :California

45:d=4  hl=2 l=  20 cons: SEQUENCE          
47:d=5  hl=2 l=   3 prim: OBJECT            :localityName
52:d=5  hl=2 l=  13 prim: UTF8STRING        :San Francisc0

67:d=4  hl=2 l=  23 cons: SEQUENCE          
69:d=5  hl=2 l=   3 prim: OBJECT            :organizationName
74:d=5  hl=2 l=  16 prim: UTF8STRING        :my org 123456789

92:d=4  hl=2 l=  38 cons: SEQUENCE          
94:d=5  hl=2 l=   3 prim: OBJECT            :commonName
99:d=5  hl=2 l=  31 prim: UTF8STRING        :myhost123456789.mydomain123.com

the tag sequence is 9 -> 17 -> 20 -> 23 -> 38. Everything works.

Looking at the asn1parse of the bad CSR we see this

17:d=4  hl=2 l=   9 cons: SEQUENCE          
19:d=5  hl=2 l=   3 prim: OBJECT            :countryName
24:d=5  hl=2 l=   2 prim: PRINTABLESTRING   :US

28:d=4  hl=2 l=  17 cons: SEQUENCE          
30:d=5  hl=2 l=   3 prim: OBJECT            :stateOrProvinceName
35:d=5  hl=2 l=  10 prim: UTF8STRING        :California

47:d=4  hl=2 l=  20 cons: SEQUENCE          
49:d=5  hl=2 l=   3 prim: OBJECT            :localityName
54:d=5  hl=2 l=  13 prim: UTF8STRING        :San Francisc0

69:d=4  hl=2 l=  23 cons: SEQUENCE          
71:d=5  hl=2 l=   3 prim: OBJECT            :organizationName
76:d=5  hl=2 l=  16 prim: UTF8STRING        :my org 123456789

94:d=4  hl=2 l=  38 cons: SEQUENCE          
96:d=5  hl=2 l=   3 prim: OBJECT            :commonName
101:d=5  hl=2 l=  31 prim: UTF8STRING        :myhost123456789.mydomain123.com

134:d=4  hl=2 l=  32 cons: SEQUENCE          
136:d=5  hl=2 l=   9 prim: OBJECT            :emailAddress
147:d=5  hl=2 l=  19 prim: IA5STRING         :[email protected]

the tag sequence is 9 -> 17 -> 20 -> 23 -> 38 -> 32. The CA rejects the CSR.

If the assumptions above are correct I see the two following issues:

  1. The Name element should be modified so that each AttributeTypeAndValue is stored in its own SET OF

  2. The DER encoding of the SET OF elements does not follow the mandatory tag sorting rule required by the DER encoding.

Thanks

Specify default encoding for TeletexString

I need to work with some certificates created by OpenSSL which used TeletexString with Latin-1 encoding for the commonName of the subject:

asn1crypto.x509.NameTypeAndValue Object #48061904
  Header: 0x300d
    constructed universal tag 16
  Data: 0x060355040314064dfc6c6c6572
    Field "type"
      asn1crypto.x509.NameType Object #48061968
        Header: 0x0603
          primitive universal tag 6
        Data: 0x550403
          Native: common_name
    Field "value"
      asn1crypto.x509.DirectoryString Object #48061744
        Data: 0x14064dfc6c6c6572
          asn1crypto.core.TeletexString Object #48062096
            Header: 0x1406
              primitive universal tag 20
            Data: 0x4dfc6c6c6572
              Native: Mþller

The value "Mþller" should be "Müller". If I change the default encoding via asn1crypto.core.TeletexString._encoding = 'latin1' then the value is correctly decoded. But that feels very dirty. 😉
Is there an official way to change the default encoding? I couldn't find one.

Cannot build with pybuild: command 'CleanCommand' has no such option 'all'

$ pybuild
I: pybuild base:184: python3.5 setup.py clean
running clean
error: error in /home/wulf/git/oss/asn1crypto/.pybuild/pythonX.Y_3.5/.pydistutils.cfg: command 'CleanCommand' has no such option 'all'
E: pybuild pybuild:319: plugin distutils failed: exit code=1: python3.5 setup.py clean

$ cat .pybuild/pythonX.Y_3.5/.pydistutils.cfg
[clean]
all=1
[build]
build-lib=/home/wulf/git/oss/asn1crypto/.pybuild/pythonX.Y_3.5/build
[install]
install-layout=deb
install-scripts=/usr/bin
install-lib=/usr/lib/python3.5/dist-packages
[easy_install]
allow_hosts=None

I found a fix, but don't know how correct it is:
Add to setup.py:
boolean_options = ['all']
def initialize_options(self):
self.all = None

More or less copy+pasted from /usr/lib/python3.5/distutils/command/clean.py

cRLDistributionPoints extension incorrectly encoded

916defc broke the encoding of cRLDistributionPoints.

Test case:

#!/usr/bin/python3

import asn1crypto.x509
import base64
import sys

crt = """
MIIEfjCCA2agAwIBAgIIAb26AgiiG8EwDQYJKoZIhvcNAQELBQAwSTELMAkGA1UE 
BhMCVVMxEzARBgNVBAoTCkdvb2dsZSBJbmMxJTAjBgNVBAMTHEdvb2dsZSBJbnRl
cm5ldCBBdXRob3JpdHkgRzIwHhcNMTcwMjAxMTMyMTAwWhcNMTcwNDI2MTMyMTAw
WjBnMQswCQYDVQQGEwJVUzETMBEGA1UECAwKQ2FsaWZvcm5pYTEWMBQGA1UEBwwN
TW91bnRhaW4gVmlldzETMBEGA1UECgwKR29vZ2xlIEluYzEWMBQGA1UEAwwNd3d3
Lmdvb2dsZS5kZTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBALwP5je3
55At+FDsLQ0qUShcjVIdgF1EeSvQWPjNnDLZo+QJ0FVmzdAlZPponAPmK123vW3h
B2oxXSozo4NL7vvCutcbUpY5HmN73YFNehiWgauMfqlAVtTP0bUwHnf5oV/IE4ZO
fyB8DXyFPznLBYrSx0RAeQwpvDlAxBZfDqQgwXo04YYeQ6kFxH9lMkBMmhzTi504
UtkWJfxVAMiXsHubdVWOAAGR64Fl2hROgfwUjmMfnOMzxRSaX6hSndx8u+msTnM8
JmxYNGicSo6x0gt5y/DwB2+HXzk/3WF15NzVrr2JZoKv/C1vUIIxUscZUp/dHAq3
rMpvJCLYyJM9rWsCAwEAAaOCAUowggFGMB0GA1UdJQQWMBQGCCsGAQUFBwMBBggr
BgEFBQcDAjAYBgNVHREEETAPgg13d3cuZ29vZ2xlLmRlMGgGCCsGAQUFBwEBBFww
WjArBggrBgEFBQcwAoYfaHR0cDovL3BraS5nb29nbGUuY29tL0dJQUcyLmNydDAr
BggrBgEFBQcwAYYfaHR0cDovL2NsaWVudHMxLmdvb2dsZS5jb20vb2NzcDAdBgNV
HQ4EFgQUPeG3xIBw4wOiVNjQg4sTjFEBsagwDAYDVR0TAQH/BAIwADAfBgNVHSME
GDAWgBRK3QYWG7z2aLV29YG2u2IaulqBLzAhBgNVHSAEGjAYMAwGCisGAQQB1nkC
BQEwCAYGZ4EMAQICMDAGA1UdHwQpMCcwJaAjoCGGH2h0dHA6Ly9wa2kuZ29vZ2xl
LmNvbS9HSUFHMi5jcmwwDQYJKoZIhvcNAQELBQADggEBACRqqYg/mrCu+P+opO8V
92BC9aRsRA1LJS1Hr5yiWgAHllRc2ds1RY65y7MUe5yj/L6Mh7txGRUy70A0Gl86
Cbs8Ao5OxlapSxkStMpuzltYmNGC8Am1LmmmteqMqWVkY4LndJpKk+gviPJgZGnX
ifvkFe6kseMpBYYe+n2blpU/EycYTOcwx5b/ymzs1xW8pSaFn+jeFvrcPJpq6WNq
PEdeKVPppA7t6memNUVfo/6OgMluvAPDfpnSezf47rtLXzc4tiJB4mEfX7GQe3+G
yyoPEw6BNFGBqC0OTnVurqpthqp+W21EYdjgcz97AVRYrnyGhHlfwUdhbJJz9/oc
qrc=
"""

der_orig = base64.b64decode(crt)

crt = asn1crypto.x509.Certificate.load(der_orig)
der_dump = crt.dump(force=True)

print(len(der_orig), len(der_dump), der_orig == der_dump, file=sys.stderr)
sys.stdout.buffer.write(der_dump)

The new encoding is 2 bytes larger than the original one.
dumpasn1 shows on the result:

 828   50: . . . . SEQUENCE {
 830    3: . . . . . OBJECT IDENTIFIER cRLDistributionPoints (2 5 29 31)
 835   43: . . . . . OCTET STRING, encapsulates {
 837   41: . . . . . . SEQUENCE {
 839   39: . . . . . . . SEQUENCE {
 841   35: . . . . . . . . [0] {
 843   33: . . . . . . . . . [0] {
 845   33: . . . . . . . . . . [0] {
 847   31: . . . . . . . . . . . [6] 'http://pki.google.com/GIAG2.crl'
         : . . . . . . . . . . . }
Error: Inconsistent object length, 2 bytes difference.
         : . . . . . . . . . . }
Error: Inconsistent object length, 2 bytes difference.
         : . . . . . . . . . }
         : . . . . . . . . }
         : . . . . . . . }
         : . . . . . . }
         : . . . . . }
         : . . . . }

Error encoding GeneralNames: TypeError: unorderable types: NoneType() >= int()

Related to #33:

Directly above the erroring line (parser.py:235) there is "tag = tag". Would that be the issue here?

#!/usr/bin/env python3

from asn1crypto.x509 import GeneralNames

der = bytes.fromhex(
    '302E820F73757363657274652E676F622E7665A01B060560865E0202A0120C105249462D472D32303030343033362D30'
)

alt = GeneralNames.load(der)
alt.dump(force=True)

Traceback (most recent call last):
  File "alt.py", line 10, in <module>
    alt.dump(force=True)
  File "/home/joern/env/lib/python3.5/site-packages/asn1crypto-0.21.1-py3.5.egg/asn1crypto/core.py", line 3975, in dump
  File "/home/joern/env/lib/python3.5/site-packages/asn1crypto-0.21.1-py3.5.egg/asn1crypto/core.py", line 3842, in _set_contents
  File "/home/joern/env/lib/python3.5/site-packages/asn1crypto-0.21.1-py3.5.egg/asn1crypto/core.py", line 1107, in dump
  File "/home/joern/env/lib/python3.5/site-packages/asn1crypto-0.21.1-py3.5.egg/asn1crypto/core.py", line 3512, in dump
  File "/home/joern/env/lib/python3.5/site-packages/asn1crypto-0.21.1-py3.5.egg/asn1crypto/core.py", line 3081, in _set_contents
  File "/home/joern/env/lib/python3.5/site-packages/asn1crypto-0.21.1-py3.5.egg/asn1crypto/core.py", line 789, in dump
  File "/home/joern/env/lib/python3.5/site-packages/asn1crypto-0.21.1-py3.5.egg/asn1crypto/core.py", line 1485, in dump
  File "/home/joern/env/lib/python3.5/site-packages/asn1crypto-0.21.1-py3.5.egg/asn1crypto/core.py", line 534, in dump
  File "/home/joern/env/lib/python3.5/site-packages/asn1crypto-0.21.1-py3.5.egg/asn1crypto/core.py", line 524, in dump
  File "/home/joern/env/lib/python3.5/site-packages/asn1crypto-0.21.1-py3.5.egg/asn1crypto/parser.py", line 237, in _dump_header
TypeError: unorderable types: NoneType() >= int()

  0  46: SEQUENCE {
  2  15: . [2] 'suscerte.gob.ve'
 19  27: . [0] {
 21   5: . . OBJECT IDENTIFIER '2 16 862 2 2'
 28  18: . . [0] {
 30  16: . . . UTF8String 'RIF-G-20004036-0'
       : . . . }
       : . . }
       : . }

Creating certs-only SignedData

Trying to create a certs-only p7c, but all required fields in SignedData are not encoded, and it appears there is no way to add empty digest algorithms or signer info sets? In the absence of content, I believe both of these should be encoded as an empty SET.

  SignedData ::= SEQUENCE {
    version CMSVersion,
    digestAlgorithms DigestAlgorithmIdentifiers,
    encapContentInfo EncapsulatedContentInfo,
    certificates [0] IMPLICIT CertificateSet OPTIONAL,
    crls [1] IMPLICIT RevocationInfoChoices OPTIONAL,
    signerInfos SignerInfos }

i.e. using this code

    signed_data = cms.SignedData({
        'version': 'v1',
        'encap_content_info': {
            'content_type': 'data',
        },
        'certificates': cs,
    })

the two "SET {}" fields below are missing in the encoded output:


    SEQUENCE {
      INTEGER 1
      SET {}
      SEQUENCE {
        OBJECT IDENTIFIER data (1 2 840 113549 1 7 1)
        }
      [0] {
            --{ cert data here }--
        }
      SET {}
      }

Incorrect DER encoding of "Named bit lists"

Hello,
I loaded an x509.Certificate and dumped it again with force=True.
The result differs from the original.
The certificate includes the Extension "Key Usage".
The value is encoded as "03 02 05 a0" in the original, but "03 03 07 a0 00" in the dumped form.
The former is the bit string "101", the latter is "101000000". According to the standards, only the former is correct.

RFC5280 specifies that trailing zeros are removed:

Named bit lists are BIT STRINGs where the values have been assigned
names. This specification makes use of named bit lists in the
definitions for the key usage, CRL distribution points, and freshest
CRL certificate extensions, as well as the freshest CRL and issuing
distribution point CRL extensions. When DER encoding a named bit
list, trailing zeros MUST be omitted. That is, the encoded value
ends with the last named bit that is set to one.

Also see X.690 11.2.2 and X.680 21.7

CMS Example

Hi,

Can you provide one example how to generate CMS structure with 2 PFX files. I am trying to understand the code and I can't get the idea how to generate CMS

Thanks in advance

Cannot parse nested structure with application tag

Rev d7cf731 fixed simple sequence with application tag. Thanks a lot for the quick fix! asn1crypto still cannot correctly parse ASN.1 with nested explicit application tags, e.g. Kerberos AP-REQ. AP-REQ is sequence with explicit application tag 14 that contains another sequence with explicit application tag 1. In Kerberos, all tags are explicit. A tests is available at https://github.com/tiran/kkdcpasn1/blob/asn1crypto/pykkdcpasn1.py

Code

class Ticket(core.Sequence):
    """Ticket for AP-REQ and SEQUENCE OF Ticket

    Ticket ::= [APPLICATION 1] SEQUENCE {
        tkt-vno      [0] INTEGER,
        realm        [1] Realm,
        sname        [2] PrincipalName,
        enc-part     [3] EncryptedData
    }
    """
    explicit_class = APPLICATION
    explicit_tag = 1
    tag_type = TAG

    _fields = [
        ('tkt-vno', core.Integer, {'tag_type': TAG, 'tag': 0}),
        ('realm', Realm, {'tag_type': TAG, 'tag': 1}),
        ('sname', PrincipalName, {'tag_type': TAG, 'tag': 2}),
        ('enc-part', EncryptedData, {'tag_type': TAG, 'tag': 3}),
    ]

class AP_REQ(core.Sequence):
    """AP-REQ -- Application request

    Client/server authentication exchange (CS)

    AP-REQ ::= [APPLICATION 14] SEQUENCE {
        pvno         [0] INTEGER (5),
        msg-type     [1] INTEGER (14),
        ap-options   [2] APOptions,
        ticket       [3] Ticket,
        authenticator        [4] EncryptedData
    }
    """
    explicit_class = APPLICATION
    explicit_tag = 14
    tag_type = TAG

    _fields = [
        ('pvno', core.Integer, {'tag_type': TAG, 'tag': 0, 'default': 5}),
        ('msg-type', core.Integer, {'tag_type': TAG, 'tag': 1, 'default': 14}),
        ('ap-options', APOptions, {'tag_type': TAG, 'tag': 2}),
        ('ticket', Ticket, {'tag_type': TAG, 'tag': 3}),
        ('authenticator', EncryptedData, {'tag_type': TAG, 'tag': 4}),
    ]

traceback

Traceback (most recent call last):
  File "pykkdcpasn1.py", line 523, in <module>
    test()
  File "pykkdcpasn1.py", line 519, in test
    pprint(apreq.native)
  File "/tmp/venv/lib64/python3.6/site-packages/asn1crypto/core.py", line 3702, in native
    self._parse_children(recurse=True)
  File "/tmp/venv/lib64/python3.6/site-packages/asn1crypto/core.py", line 3646, in _parse_children
    raise e
  File "/tmp/venv/lib64/python3.6/site-packages/asn1crypto/core.py", line 3619, in _parse_children
    child = _build(*child)
  File "/tmp/venv/lib64/python3.6/site-packages/asn1crypto/core.py", line 4965, in _build
    value = _build(*info, spec=spec, spec_params={'no_explicit': True})
  File "/tmp/venv/lib64/python3.6/site-packages/asn1crypto/core.py", line 4994, in _build
    CLASS_NUM_TO_NAME_MAP.get(class_, class_)
ValueError: Error parsing __main__.Ticket - class should have been universal, but application was found
    while parsing __main__.AP_REQ

ASN.1 payload

    0:d=0  hl=4 l= 510 cons: appl [ 14 ]       
    4:d=1  hl=4 l= 506 cons:  SEQUENCE          
    8:d=2  hl=2 l=   3 cons:   cont [ 0 ]        
   10:d=3  hl=2 l=   1 prim:    INTEGER           :05
   13:d=2  hl=2 l=   3 cons:   cont [ 1 ]        
   15:d=3  hl=2 l=   1 prim:    INTEGER           :0E
   18:d=2  hl=2 l=   7 cons:   cont [ 2 ]        
   20:d=3  hl=2 l=   5 prim:    BIT STRING        
   27:d=2  hl=4 l= 321 cons:   cont [ 3 ]        
   31:d=3  hl=4 l= 317 cons:    appl [ 1 ]        
   35:d=4  hl=4 l= 313 cons:     SEQUENCE          
   39:d=5  hl=2 l=   3 cons:      cont [ 0 ]        
   41:d=6  hl=2 l=   1 prim:       INTEGER           :05
   44:d=5  hl=2 l=  15 cons:      cont [ 1 ]        
   46:d=6  hl=2 l=  13 prim:       GENERALSTRING     
   61:d=5  hl=2 l=  29 cons:      cont [ 2 ]        
   63:d=6  hl=2 l=  27 cons:       SEQUENCE          
   65:d=7  hl=2 l=   3 cons:        cont [ 0 ]        
   67:d=8  hl=2 l=   1 prim:         INTEGER           :01
   70:d=7  hl=2 l=  20 cons:        cont [ 1 ]        
   72:d=8  hl=2 l=  18 cons:         SEQUENCE          
   74:d=9  hl=2 l=   6 prim:          GENERALSTRING     
   82:d=9  hl=2 l=   8 prim:          GENERALSTRING     
   92:d=5  hl=4 l= 256 cons:      cont [ 3 ]        
   96:d=6  hl=3 l= 253 cons:       SEQUENCE          
   99:d=7  hl=2 l=   3 cons:        cont [ 0 ]        
  101:d=8  hl=2 l=   1 prim:         INTEGER           :12
  104:d=7  hl=2 l=   3 cons:        cont [ 1 ]        
  106:d=8  hl=2 l=   1 prim:         INTEGER           :01
  109:d=7  hl=3 l= 240 cons:        cont [ 2 ]        
  112:d=8  hl=3 l= 237 prim:         OCTET STRING      [HEX DUMP]:ECC2A53967B412D791DFF3EA3D50F765DC7ED7C7E66912C836163F5BD0A07FC679E46D36C9B38A2273C877D233AD55FAB9F3D95637F8F1DC22117723E77537C92D909064DE10B26E86C9EFDCCF37CC16D20E75849D0CBD04A56E7F69D46C6D7DB9DDDC6ACBBE37D6CF49C6CC78C57228C0E903FDC08772B030DD22DC84B5D55C66E07279E9840A3EA68F1C27BA26305D1CD05D1E3A994A4246DB8E834948EFD3788409760994AF1A72F5CF1485C49C7646A3B72C460FD62F5C373C224ECE0859F4DDEF5B77BF75A9AB220524E00A35A357722F703A218C5FD1A717ED0392F19864BF0B52921B501906A8AF48C3
  352:d=2  hl=3 l= 159 cons:   cont [ 4 ]        
  355:d=3  hl=3 l= 156 cons:    SEQUENCE          
  358:d=4  hl=2 l=   3 cons:     cont [ 0 ]        
  360:d=5  hl=2 l=   1 prim:      INTEGER           :12
  363:d=4  hl=3 l= 148 cons:     cont [ 2 ]        
  366:d=5  hl=3 l= 145 prim:      OCTET STRING      [HEX DUMP]:EAD1DF7495E86BFEC932DBA81C9A764826080BB8C359AC3627C4B2CE530217042449E9CAA3DB66031FBE4B1CF942EC9EC773A2FB2485DD4D6E1271F9FF41D26C54CE9370D6E4573156CDC051944A563056BFA1D47CECEFF3CB4AC455FE33CD03E91BA9B4DC2243CE59EF0E1865AEBDBA76E30EB2AD8DEB71409B11F86CFA6C6E488270303C0451BD86A91C139018421A07

https://lapo.it/asn1js/#6E8201FE308201FAA003020105A10302010EA20703050000000000A38201416182013D30820139A003020105A10F1B0D465245454950412E4C4F43414CA21D301BA003020101A11430121B066B61646D696E1B086368616E67657077A38201003081FDA003020112A103020101A281F00481EDECC2A53967B412D791DFF3EA3D50F765DC7ED7C7E66912C836163F5BD0A07FC679E46D36C9B38A2273C877D233AD55FAB9F3D95637F8F1DC22117723E77537C92D909064DE10B26E86C9EFDCCF37CC16D20E75849D0CBD04A56E7F69D46C6D7DB9DDDC6ACBBE37D6CF49C6CC78C57228C0E903FDC08772B030DD22DC84B5D55C66E07279E9840A3EA68F1C27BA26305D1CD05D1E3A994A4246DB8E834948EFD3788409760994AF1A72F5CF1485C49C7646A3B72C460FD62F5C373C224ECE0859F4DDEF5B77BF75A9AB220524E00A35A357722F703A218C5FD1A717ED0392F19864BF0B52921B501906A8AF48C3A4819F30819CA003020112A28194048191EAD1DF7495E86BFEC932DBA81C9A764826080BB8C359AC3627C4B2CE530217042449E9CAA3DB66031FBE4B1CF942EC9EC773A2FB2485DD4D6E1271F9FF41D26C54CE9370D6E4573156CDC051944A563056BFA1D47CECEFF3CB4AC455FE33CD03E91BA9B4DC2243CE59EF0E1865AEBDBA76E30EB2AD8DEB71409B11F86CFA6C6E488270303C0451BD86A91C139018421A07

Can't decode some "valid" x.509 certificates

Hi Will,

There's a few valid certificate that asn1crypto fails to parse (whereas OpenSSL is able to decode them).

Here is the traceback:

Traceback (most recent call last):
  File "[...]/collect_report/__init__.py", line 181, in _extract_common_name_from_certificate_chain
    subject_fields = tbs_cert['subject'].native
  File "[...]/../lib/asn1crypto/x509.py", line 1015, in native
    for rdn in self.chosen.native:
  File "[...]/../lib/asn1crypto/core.py", line 3304, in native
    self._parse_children(recurse=True)
  File "[...]/../lib/asn1crypto/core.py", line 3276, in _parse_children
    raise e
ValueError: Value [UNIVERSAL 22] did not match the class and tag of any of the alternatives in asn1crypto.x509.DirectoryString: [UNIVERSAL 12], [UNIVERSAL 30], [UNIVERSAL 19], [UNIVERSAL 28], [UNIVERSAL 20]
    while parsing asn1crypto.x509.NameTypeAndValue
    while parsing asn1crypto.x509.RelativeDistinguishedName
    while parsing asn1crypto.x509.RDNSequence

I've tried to debug the issue myself, but I'm stuck, it would be nice if you can help.

Can I send you an email with two x509 certificates that asn1crypto fails to parse?

Thanks!

Error parsing cms CompressedData

I am working on implemented a file transfer protocol called AS2 and this requires messages to be compressed, signed and encrypted as per the CMS Specifications.

I have started with the compression and am able to parse the compressed data ASN1 generated using this package.

However I am getting the below error when I try to parse data generated from another tool that implements this protocol as well.

Traceback (most recent call last):
  File "/Users/abhishekram/Documents/work/Research/pyAS2-lib/dev/tests/test_with_mecas2.py", line 16, in test_compressed_message
    in_mic_content = in_message.parse(infile.read())
  File "/Users/abhishekram/Documents/work/Research/pyAS2-lib/dev/pyas2lib/message.py", line 150, in parse
    decompress_message(mic_content)
  File "/Users/abhishekram/Documents/work/Research/pyAS2-lib/dev/pyas2lib/cms.py", line 31, in decompress_message
    return cms_content['content'].decompressed
  File "/Users/abhishekram/Documents/work/Research/pyAS2-lib/dev/venv/lib/python2.7/site-packages/asn1crypto/cms.py", line 883, in decompressed
    self._decompressed = zlib.decompress(self['encap_content_info']['content'].native)
  File "/Users/abhishekram/Documents/work/Research/pyAS2-lib/dev/venv/lib/python2.7/site-packages/asn1crypto/core.py", line 2770, in __getitem__
    raise e
ValueError: Error parsing asn1crypto.core.ParsableOctetString - method should have been primitive, but constructed was found
    while parsing asn1crypto.cms.EncapsulatedContentInfo

I am also attaching the DER encoded compressed data

ASN1 for your reference.

Typo in asn1crypto.core error message

I see this message too often so thought to report the typo

 /asn1crypto/core.py", line 779, in __init__
    type_name(value)
TypeError: value must be an instance of **Ans1Value**, not bytes

Indefinite length encoding supported on input?

v0.21.1

Encountered a p12 file using indefinite length encoding in auth_safe. The Content-Info OctetString is not aggregated correctly from the substrings. The reconstructed OctetString is too large and the native value contains the ASN.1 headers of the substrings instead of only content.

Test case

Input string in hex = '2480040d8dfff0980736af936e423acfcc04159277f7f0e479ffc7dc33b2d03d7b1a186d4472aa490000'

dummy.zip

$ openssl asn1parse -inform DER -in dummy.da2
0:d=0 hl=2 l=inf cons: OCTET STRING
2:d=1 hl=2 l= 13 prim: OCTET STRING [HEX DUMP]:8DFFF0980736AF936E423ACFCC
17:d=1 hl=2 l= 21 prim: OCTET STRING [HEX DUMP]:9277F7F0E479FFC7DC33B2D03D7B1A186D4472AA49
40:d=1 hl=2 l= 0 prim: EOC

Expected behaviour: an OctetString whose native value is a byte string of length 34

>>> data = binascii.unhexlify('2480040d8dfff0980736af936e423acfcc04159277f7f0e479ffc7dc33b2d03d7b1a186d4472aa490000')
>>> x = core.OctetString.load(data)
>>> len(x.native)
34
>>> binascii.hexlify(x.native)
b'8dfff0980736af936e423acfcc9277f7f0e479ffc7dc33b2d03d7b1a186d4472aa49'

Observed behaviour:

>>> data = binascii.unhexlify('2480040d8dfff0980736af936e423acfcc04159277f7f0e479ffc7dc33b2d03d7b1a186d4472aa490000')
>>> x = core.OctetString.load(data)
>>> len(x.native)
38
>>> binascii.hexlify(x.native)
b'040d8dfff0980736af936e423acfcc04159277f7f0e479ffc7dc33b2d03d7b1a186d4472aa49'

## note ASN.1 headers 040d and 0415 are in the native value

KeyAlgorithmIdentifierId in asn1crypto.CMS doesn't include encryption mode OIDs

When the payload being encrypted is not another cryptographic key, regular encryption is done instead of key wrapping (at least that is what openssl command line does). If you agree, please include the 'encryption mode' OIDs in the KeyEncryptionAlgorithmIdentifierId map e.g. 2.16.840.1.101.3.4.1.2 - id-aes128-CBC

cryptography import asn1crypto(asn1crypto>=0.21.0)，but asn1crypto has a bug fixed and has not new release

in file cryptography/src/cryptography/x509/extensions.py, import asn1crypto

from asn1crypto.keys import PublicKeyInfo

cryptography limit asn1crypto>=0.21.0，but the newest release version has bug fixed in：
64ab197.

asn1crypto has not release the fixed commit.

so. when i use cryptography on Python 2, the unicode string here may raise a UnicodeDecodeError as it
tries to join a bytestring path to the unicode name "crypto"

Please release a new version.

you can see here: pyca/cryptography#3868 (comment)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.