Code Monkey home page Code Monkey logo

pandect's Introduction

pandect

Clojars Project Documentation CI

pandect is a fast and easy-to-use Message Digest, Checksum, HMAC and Signature library for Clojure.

Usage

Note that - for versions 0.6.0 and up - to use functions based on the BouncyCastle crypto provider, you have to manually include a version of the org.bouncycastle/bcprov-jdk15on artifact. Additionally, you need to take special care when trying to create an uberjar (see here).

REPL

(require '[pandect.algo.sha1 :refer :all])

(sha1 "Hello World!")                      ;; => "2ef7bde608ce5404e97d5f042f95f89f1c232871"
(sha1 (.getBytes "Hello World!" "UTF-8"))  ;; => "2ef7bde608ce5404e97d5f042f95f89f1c232871"
(sha1 (File. "project.clj"))               ;; => "ff3b4565652aeb835edf2715b2a28586929ea4cc"
(sha1 (FileInputStream. "project.clj"))    ;; => "ff3b4565652aeb835edf2715b2a28586929ea4cc"

;; more variants
(sha1-bytes "Hello World!")                ;; => #<byte[] [B@5293b95>
(sha1-file "project.clj")                  ;; => "ff3b4565652aeb835edf2715b2a28586929ea4cc"
(sha1-file-bytes "project.clj")            ;; => #<byte[] [B@e2606c7>

HMAC functions take an additional key:

(sha1-hmac "Hello World!" "secret-key")       ;; => "399fc3d94f6df2213f92fcf2a8b6669279ef7d20"
(sha1-hmac-bytes "Hello World!" "secret-key") ;; => #<byte[] [B@602bd522>

Signing functions expect a java.security.PrivateKey, while verification works with java.security.PublicKey or java.security.cert.Certificate:

(sha1-rsa "Hello World!" private-key)                 ;; => "5866..."
(sha1-rsa-verify "Hello World!" "5866..." public-key) ;; => true

You can tune stream processing using pandect.buffer/with-buffer-size (default is 2KB):

(require '[pandect.buffer :refer [with-buffer-size]])

(with-open [in (io/input-stream "shootout/data/1mb.txt")]
  (with-buffer-size (* 512 1024)
    (sha256 in)))

If you want to hash a String using a specific encoding, you should create the respective byte array manually:

(sha1 "Hällo World!")                          ;; => "f19c05a67c3d0f297b62e868657cf177913ce02a"
(sha1 (.getBytes "Hällo World!" "ISO-8859-1")) ;; => "cfe670bd6845020f5754b19a3c0eee602043eb89"

The namespace pandect.core contains all available algorithms but for better startup/compile times, using algorithm-specific ones is recommended.

Supported Algorithms

See the generated documentation for the available functions and their parameters.

Checksum MDx SHA SHA-3 RIPEMD BLAKE2 Others
Adler32* MD2* SHA-1 SHA3-224 RIPEMD-128 BLAKE2b-160 SipHash-2-4+
CRC-32* MD4 SHA-256 SHA3-256 RIPEMD-160 BLAKE2b-256 SipHash-4-8+
MD5 SHA-384 SHA3-384 RIPEMD-256 BLAKE2b-384 Tiger192,3
SHA-512 SHA3-512 RIPEMD-320 BLAKE2b-512 Whirlpool

* not available as MAC
+ only available as MAC

Uberjars and BouncyCastle

The BouncyCastle JAR has been signed to prevent tampering - and JCE won't allow usage of any of its functions if that signature is not present. This poses a problem when creating an Uberjar since all needed class files will be extracted from their JARs and repackaged into a single one - removing any existing signature.

You'll see, for example, the following exceptions in that case:

java.security.NoSuchAlgorithmException: Algorithm (...) not available
java.security.NoSuchProviderException: JCE cannot authenticate the provider BC

Primarily, this means you have to prevent the BouncyCastle files ending up within the JAR which can be accomplished using the :provided profile in you project.clj:

:profiles
{:provided
 {:dependencies [[org.bouncycastle/bcprov-jdk15on "1.54"]}}

Now you can build your uberjar but you have to make sure that the BouncyCastle JAR is on your classpath when running it, i.e. assuming they are both in the same directory:

java -cp bcprov-jdk15on-1.54.jar:app-standalone.jar app.core

Before version 0.6.0, pandect accessed the BouncyCastle functionality directly (i.e. not via MessageDigest/getInstance or Mac/getInstance but using actual constructor calls), meaning these limitations did not apply. It is, thus, strongly recommended to do it right and upgrade to 0.6.0.

Benchmarks

You can run benchmarks against pandect, buddy and clj-digest using perforate:

lein bench {--table|--edn}

License

MIT License

Copyright (c) 2014-2021 Yannick Scherer

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

pandect's People

Contributors

alexanderkiel avatar jstepien avatar mavbozo avatar tcrayford avatar totakke avatar xsc avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

pandect's Issues

Regression: Improper namespacing of inline deps causes collisions

There are several dependencies that are inlined without a namespace prefix in 1.0.1 (after upgrading from 0.6.1). See all the files not starting with pandect in the file list of the 1.0.1 jar below:

Seems to be caused by 42f0302

META-INF/MANIFEST.MF
project.clj
Util.java
clojure/
clojure/lang/
clojure/lang/PersistentUnrolledVector$Card1$1.class
clojure/lang/PersistentUnrolledVector$Card2.class
clojure/lang/PersistentUnrolledVector$Card0.class
clojure/lang/PersistentUnrolledMap$Card2$1.class
clojure/lang/PersistentUnrolledVector$Card1$UnrolledChunkedSeq.class
clojure/lang/PersistentUnrolledMap$Card6$UnrolledChunkedSeq.class
clojure/lang/PersistentUnrolledMap$Card2$UnrolledChunkedSeq.class
clojure/lang/PersistentUnrolledVector$Card5$UnrolledChunkedSeq.class
clojure/lang/PersistentUnrolledMap$Card3$1.class
clojure/lang/PersistentUnrolledVector$Card4.class
clojure/lang/PersistentUnrolledVector$Card0$1.class
clojure/lang/PersistentUnrolledVector$Card6.class
clojure/lang/PersistentUnrolledMap$Card3$UnrolledChunkedSeq.class
clojure/lang/PersistentUnrolledVector$Card4$UnrolledChunkedSeq.class
clojure/lang/PersistentUnrolledVector$Card1.class
clojure/lang/PersistentUnrolledVector.java
clojure/lang/PersistentUnrolledMap$Card1$1.class
clojure/lang/PersistentUnrolledMap$Transient.class
clojure/lang/PersistentUnrolledMap.class
clojure/lang/PersistentUnrolledVector.class
clojure/lang/PersistentUnrolledVector$Card2$1.class
clojure/lang/PersistentUnrolledVector$Card3.class
clojure/lang/PersistentUnrolledVector$Card3$1.class
clojure/lang/PersistentUnrolledVector$Transient.class
clojure/lang/PersistentUnrolledVector$Card5.class
clojure/lang/PersistentUnrolledMap$Card0$1.class
clojure/lang/PersistentUnrolledMap$Card6.class
clojure/lang/PersistentUnrolledMap$Card5$1.class
clojure/lang/PersistentUnrolledMap.java
clojure/lang/PersistentUnrolledVector$Card2$UnrolledChunkedSeq.class
clojure/lang/PersistentUnrolledMap$Card5$UnrolledChunkedSeq.class
clojure/lang/PersistentUnrolledVector$Card6$1.class
clojure/lang/PersistentUnrolledMap$Card4.class
clojure/lang/PersistentUnrolledMap$Card1$UnrolledChunkedSeq.class
clojure/lang/PersistentUnrolledVector$Card6$UnrolledChunkedSeq.class
clojure/lang/PersistentUnrolledMap$Card0.class
clojure/lang/PersistentUnrolledMap$Card4$1.class
clojure/lang/PersistentUnrolledMap$Card2.class
clojure/lang/PersistentUnrolledVector$Card5$1.class
clojure/lang/PersistentUnrolledMap$Card5.class
clojure/lang/PersistentUnrolledMap$Card6$1.class
clojure/lang/PersistentUnrolledMap$Card3.class
clojure/lang/PersistentUnrolledMap$Card1.class
clojure/lang/PersistentUnrolledVector$Card3$UnrolledChunkedSeq.class
clojure/lang/PersistentUnrolledMap$Card4$UnrolledChunkedSeq.class
clojure/lang/PersistentUnrolledVector$Card4$1.class
pandect/
pandect/impl/
pandect/impl/checksum.clj
pandect/impl/message_digest.clj
pandect/impl/bouncy_castle.clj
pandect/impl/message_digest/
pandect/impl/message_digest/signature.clj
pandect/impl/message_digest/hash.clj
pandect/impl/message_digest/hmac.clj
pandect/buffer.clj
pandect/target/
pandect/target/srcdeps/
pandect/target/srcdeps/pandect/
pandect/target/srcdeps/pandect/riddley/
pandect/target/srcdeps/pandect/riddley/v0v2v0/
pandect/target/srcdeps/pandect/riddley/v0v2v0/riddley/
pandect/target/srcdeps/pandect/riddley/v0v2v0/riddley/walk.clj
pandect/target/srcdeps/pandect/riddley/v0v2v0/riddley/compiler.clj
pandect/target/srcdeps/pandect/clj_tuple/
pandect/target/srcdeps/pandect/clj_tuple/v0v2v2/
pandect/target/srcdeps/pandect/clj_tuple/v0v2v2/clj_tuple.clj
pandect/target/srcdeps/pandect/potemkin/
pandect/target/srcdeps/pandect/potemkin/v0v4v5/
pandect/target/srcdeps/pandect/potemkin/v0v4v5/potemkin/
pandect/target/srcdeps/pandect/potemkin/v0v4v5/potemkin/walk.clj
pandect/target/srcdeps/pandect/potemkin/v0v4v5/potemkin/namespaces.clj
pandect/target/srcdeps/pandect/potemkin/v0v4v5/potemkin/utils.clj
pandect/target/srcdeps/pandect/potemkin/v0v4v5/potemkin/types.clj
pandect/target/srcdeps/pandect/potemkin/v0v4v5/potemkin/template.clj
pandect/target/srcdeps/pandect/potemkin/v0v4v5/potemkin/collections.clj
pandect/target/srcdeps/pandect/potemkin/v0v4v5/potemkin/macros.clj
pandect/target/srcdeps/pandect/potemkin/v0v4v5/potemkin.clj
pandect/algo/
pandect/algo/keccak_512.clj
pandect/algo/ripemd320.clj
pandect/algo/crc32.clj
pandect/algo/sha3_224.clj
pandect/algo/siphash.clj
pandect/algo/blake2b_384.clj
pandect/algo/gost.clj
pandect/algo/sha3_384.clj
pandect/algo/ripemd256.clj
pandect/algo/siphash48.clj
pandect/algo/sha512.clj
pandect/algo/sha1.clj
pandect/algo/md5.clj
pandect/algo/md4.clj
pandect/algo/adler32.clj
pandect/algo/md2.clj
pandect/algo/sha3_256.clj
pandect/algo/blake2b_256.clj
pandect/algo/ripemd160.clj
pandect/algo/sha224.clj
pandect/algo/sha384.clj
pandect/algo/keccak_224.clj
pandect/algo/whirlpool.clj
pandect/algo/sha3_512.clj
pandect/algo/blake2b_512.clj
pandect/algo/keccak_384.clj
pandect/algo/keccak_256.clj
pandect/algo/tiger.clj
pandect/algo/ripemd128.clj
pandect/algo/blake2b_160.clj
pandect/algo/sha256.clj
pandect/core.clj
pandect/utils/
pandect/utils/convert.clj
pandect/utils/bouncy_castle_provider.clj
pandect/gen/
pandect/gen/signature_generator.clj
pandect/gen/core.clj
pandect/gen/hmac_generator.clj
pandect/gen/hash_generator.clj
pandect/gen/protocol_utils.clj
pandect/gen/function_utils.clj
pandect/codegen.clj
riddley/
riddley/Util.java
riddley/Util.class
potemkin/
potemkin/PersistentMapProxy$1.class
potemkin/PersistentMapProxy$IEquality.class
potemkin/PersistentMapProxy$MapEntry.class
potemkin/PersistentMapProxy$IMap.class
potemkin/PersistentMapProxy.class
potemkin/PersistentMapProxy.java

get string from hmac

Hello,

Sorry for posting here, I'm not sure where to ask that, and I've been trying to get something working for quite some time...

I have trouble getting the String from the sha256-hmac-bytes function.
Here is a small example :

  (let [signature (sha256-hmac-bytes "test" "my-secret")]
    (println signature)     ;; #bin Oz+h3YkA8vHXQAgBst2NxZBRpmczdV8AEoadoTluGA4=

    ;; I can see it above !!
    ;; How do I return get the string "Oz+h3YkA8vHXQAgBst2NxZBRpmczdV8AEoadoTluGA4" above ?
    ;; instead of "test.Oz+h3YkA8vHXQAgBst2NxZBRpmczdV8AEoadoTluGA4"  I get "test.[B@5db4cdc7"
    ;; Is it possible to do that ?
    (str "test." signature))

Aditionally, I tried things like :

    (println (String. signature))               ;?�݉�����ݍŐQ�g3u_����9n�
    (println (String. signature "UTF-8"))       ;?�݉�����ݍŐQ�g3u_����9n�
    (println (String. signature "ASCII"))       ;?������������Q�g3u_����9n�
    (println (String. signature "ISO8859_1"))   ;?¡Ý�òñ×�²Ý�Å�Q¦g3u_���¡9n�

Thank you for your help

java.lang.SecurityException

org.bouncycastle.crypto.digests.SHA3Digest"'s signer information does not match signer information of other classes in the same package, compiling:(pandect/core.clj:35:1)

Thanks for pandect, looks like it can be useful for my current project.

I just tried to use it and got the above error.

This happens with pandect 3.1 and 3.2;
3.0 works fine...

I really do not know enough about java libraries here, can you enlighten me how to fix this?

Thanks, Mathias

Unexpected side-effect of sha on stream

I got an unexpected side effect by calculating the sha1 a missing etag.

I had to add a call to reset below, since otherwise the stream was consumed, and an empty pictures was sent from the email server.

Can this be improved? (Using 0.6.0)

  (let [etag (if etag etag (pandect.algo.sha1/sha1 body))]
    ;; if I do a sha on a stream, I need to reset it afterwards, since otherwise, and empty image will be sent to the client
    (if (= (type body) org.httpkit.BytesInputStream) (.reset body))

AOT issue with 0.4.0?

When bundling up my app into an uberjar and executing, I get the following error: java.lang.IllegalStateException: Attempting to call unbound fn: #'pandect.algo.sha256/sha256-hmac. Downgrading to 0.3.4 fixes the issue.

I'm not sure if I should be doing something in particular to get this to work?

'SipHash-2-4' not available when AOT-compiled

This might affect some other (maybe all BouncyCastle) algorithms, too, but when pandect is part of an AOT-compiled application, Mac.getInstance("Siphash-2-4") throws an exception.

java.security.NoSuchAlgorithmException: Algorithm Siphash-2-4 not available
    at javax.crypto.Mac.getInstance(Mac.java:181)
    at pandect.algo.siphash$fn__6733.invokeStatic(siphash.clj:45)
    at pandect.algo.siphash$fn__6733.invoke(siphash.clj:20)
    at pandect.algo.siphash$fn__6704$G__6699__6711.invoke(siphash.clj:10)
    at pandect.algo.siphash$siphash.invokeStatic(siphash.clj:149)
...

Doesn't work when uberjar'ed

I deploy my webapp via uberjar, and after upgrading to 0.4.0, none of the methods (such as pandect.core/sha256-hmac) work anymore. I'm guessing it has to do with the AOT changes... please advise on how to fix. Thanks!

Slow startup

I noticed once including pandect in my project (to use sha1 fn) that the repl startup time slowed significantly, sometimes even in timing out. I isolated it to pandect by creating a minimal repl (isolated dependency graph), and finding the culprit looking like pandect, doing (require '[pandect.core]) takes upwards of 10 seconds?

I ripped it out and replaced with clj-digest as I only needed sha1 fn and everything is sane again, luckily due to same fn name and use :refer is ns only minimal changes.

Security Exception

When trying to require pandect.core in repl, I get this exception:
(require '[pandect.core :refer :all])

CompilerException java.lang.SecurityException: class "org.bouncycastle.crypto.digests.SHA3Digest"'s signer information does not match signer information of other classes in the same package, compiling:(pandect/algo/sha3_512.clj:87:8)
The version I tried is [pandect "0.4.0"] as is recommended in repos README.

P.S. I couldn't reproduce this behavior in cloned repo though.

The result of (sha1 "你好") is error.

Here, the result of hex_sha1("你好") is

440ee0853ad1e99f962b63e459ef992d7c211722

but the result of (pandect.algo.sha1/sha1 "你好") is

4bd9abaa33ea0834383e15fef9ae377f762fc03b

As far as I know,the correct of the result is 440ee0853ad1e99f962b63e459ef992d7c211722. Is there something wrong in pandect.algo.sha1/sha1 function?

Add Support for SipHash

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.