The following changes are needed in kademlia.network.Server (should be subclassed or rewritten):

• Split up client and server
• Instead of storing same data in the "closest" nodes, we want to store n different rekeys in n closest (by hash) nodes (no duplicates!)
• Instead of sha1, should use something secure (sha256? keccak?)

How to reproduce: run py.test multiple times

(py35) michwill ~/Projects/nucypher-kms $ py.test                           
===================================================================== test session starts =====================================================================
platform linux -- Python 3.5.2, pytest-3.2.1, py-1.4.34, pluggy-0.4.0
rootdir: /home/michwill/Projects/nucypher-kms, inifile: tox.ini
plugins: cov-2.5.1
collected 13 items                                                                                                                                             

tests/test_client.py ........
tests/test_default_crypto.py ..
tests/test_kmac.py ..
tests/test_network.py F

========================================================================== FAILURES ===========================================================================
_____________________________________________________________________ test_seed_only_node _____________________________________________________________________

    def test_seed_only_node():
        """
        Shows that when we set up two nodes, a "full" node and a "seed-only" node,
        that the "seed-only" node can set key-value pairs that the "full" node will store,
        but not vice-versa.
        """
        loop = asyncio.get_event_loop()
    
        # First, let's set up two servers:
        # A full node...
        full_server = Server()
        full_server.listen(8468)
        loop.run_until_complete(full_server.bootstrap([("127.0.0.1", 8468)]))
    
        # ...and a seed-only node.
        seed_only_server = NuCypherSeedOnlyDHTServer()
        seed_only_server.listen(8471)
        loop.run_until_complete(seed_only_server.bootstrap([("127.0.0.1", 8468)]))
    
        # The seed-only node is able to set a key and retrieve it again.
        key_to_store = "llamas"
        value_to_store = "tons_of_things_keyed_llamas"
        setter = seed_only_server.set(key_to_store, value_to_store)
        loop.run_until_complete(setter)
    
        # Now, the item is stored on the full server.
        full_server_stored_items = list(full_server.storage.items())
        assert len(full_server_stored_items) == 1
        assert full_server_stored_items[0][1] == value_to_store
    
        # ...but nothing is stored on the seed-only server.
        seed_only_server_stored_items = list(seed_only_server.storage.items())
>       assert len(seed_only_server_stored_items) == 0
E       AssertionError: assert 1 == 0
E        +  where 1 = len([(b'\xf2\xe2\xd8D\xb3\xe0Ma\x10\x9cM\xea\xd6\xe1!\xbf\xbd\x98\xb0\xa3', 'tons_of_things_keyed_llamas')])

tests/test_network.py:39: AssertionError
============================================================= 1 failed, 12 passed in 0.20 seconds =============================================================
(py35) michwill ~/Projects/nucypher-kms $ py.test
===================================================================== test session starts =====================================================================
platform linux -- Python 3.5.2, pytest-3.2.1, py-1.4.34, pluggy-0.4.0
rootdir: /home/michwill/Projects/nucypher-kms, inifile: tox.ini
plugins: cov-2.5.1
collected 13 items                                                                                                                                             

tests/test_client.py ........
tests/test_default_crypto.py ..
tests/test_kmac.py ..
tests/test_network.py .

================================================================== 13 passed in 5.16 seconds =================================================================

In light of @tuxxy's work on EncryptedFile, I wonder: to what degree do we make assumptions about the contents of The Secret which Bob ultimately hopes to reconstruct?

It seems that for much of our planning, we assume that it will be a symmetric key, which he can then go on to use for whatever other purposes he may have pre-arranged with Alice.

But are Alice and Bob allowed to share other types of secrets using nucypher-kms?

@michwill and I discussed a scheme in which every value in our DHT is essentially a "symlink," even if that symlink is to the same Node.

Currently, Policy.treasure_map is just a list, and Policy.update_treasure_map() appends to that list. We need logic that allows it to actually interpret a policy offer result and update itself.

Perhaps, it makes sense to also deserialize and prepare rekeys to be used when the encrypted object is "hot" in memory, rather than deserialize every time

Quoting architecture.rst:

Kademlia by default (see kademlia.network.server) saves data in multiple nodes, and also clients are servers there.

We need to split up client and server (that is, get and set methods of the client don't save data in the current node).

Currently, set calls set_digest which in turn calls self.protocol.router.findNeighbors on the Node instance.

It has been noted that we should use Apistar!

Using the protocol described in "Designing a smart contract" section of the updated white paper https://github.com/nucypher/kms-whitepaper

No "insentives logic" in this ticket yet

ZODB stores all versions of objects. It's undesirable for the rekeys: we need to be able to "hard remove" them (e.g. wipe the old rekey object with a placeholder in the file which stores the DB, and only then mark the object as removed - no rollback will be possible for this). This is a security measure

All kFrags that come from Alice will need to be Signed by her so that Ursula can check the validity of them.

When calling Client.encrypt_key with a path equivalent to None, the key is returned without being encapsulated by a list.

This method should always return a key encapsulated by a list object, even if there is only one key to return. Doing so will help standardize the Cryptographic API more.

This isn't explicitly required for now, so I'm putting it on the backburner since it will be sorta annoying to refactor the tests for this.

@tuxxy pointed this out to me today on Slack.

Per @tuxxy, we'll use keccak-256 for signatures and sha3-256 for content.

The protocol specified in Modify kademlia.protocol.KademliaProtocol should be subclassed or re-written to re-encrypt data instead of just storing it. It should provide methods:

• store_rekey
• remove_rekey
• reencrypt

Introduce quotas for re-encryption nodes; the quotas are in accordance with the stake at the node address, stored in Ethereum blockchain

All the rekeys are encrypted persistent objects, similar to how it is done in keas.kmi

Two possible approaches here:

• Encrypt Kademlia's rpcudp messages with Ursulas using public key encryption
• Don't roll your crypto and use Kademlia only to discover Ursulas, and use normals TLS after the discovery

Two possible ways to do this:

1. Stop the node from responding to ping entirely,
2. Change the server so that it reads a specific type of acknowledgment from nodes, including examining whether they are seed-only.

The client can be started as a dummy API which does nothing, then to plug in encryption/decryption primitives, and then to be actually using the kademlia-derived Server

The charm-based prototype is being developed here: https://github.com/nucypher/umbral

The simplest policy for rekeys is time limit. Nodes should have a configuration of how long they promise to store re-encryption policies for. This probably can be later written on blockchain so that clients and validators can be always aware of it

Ursula can be discovered by all other actors via a DHT via UDP (kademlia uses rcpudp for this purpose).

However, once this discovery has occurred, a handshake ensues which enabled an upgraded communications paradigm. The exact nature of this upgrade varies depending on the actor.

This Issue concerns the ability for Alice, having discovered Ursula, to craft and offer a Policy.

See: https://github.com/nucypher/nucypher-kms/pull/44/files#diff-1ef8ad9f0872558034a3d37e11f8b81dR2

When Ursula receives a kFrag, she needs to validate the signature from Alice to confirm that it is actually from her.

The secret key can be:

• Come from config file
• Derived from a passphrase typed at the start
• Come from our already working decentralized KMS

Looking at this blog post:

But there’s a way to do it with the current algorithm. When Alice delegates access to Bob, she can generate an ephemeral key sk_e and produce a re-encryption key rk_ae. Then she encrypts sk_e with Bob’s public key pk_b yielding e_b.

The proxy will be given both rk_ae and e_b. When Bob connects, the proxy will hand him e_b. Then Bob can extract sk_e out of it and use that to decrypt encrypted messages coming from the proxy.

Am I right in thinking that this portion of the story is not yet tested in tests.test_default_crypto?

Kademlia (python3.5 branch) and ZODB both use asyncio. We need to make them using the same event loop rather then conflicting

Currently, generate_challenge_pack is fake. We need to accept the kFrag, then use it to make challenges.

Is there currently a smart contract template for publishing and signing public key fingerprints? Is there an implementation of TLS using this?

We're transforming KeyRing into a class that more literally applies the metaphor of a KeyRing, and also borrows from KeyRing concepts from other cryptographic practices, such as PGP.

The touchstone methods for this functionality will be something like .add_key() and .add_keypair().

The API... is still coming together.

One possibility is .add_key(bob_pubkey_sig) - if a single argument is given, it establishes a new character on the KeyRing - in this case Bob. Characters are represented as a string, in the form {character_name}_{pubkey_sig_fingerprint}.

.add_key(bob_pubkey_sig, pubkey_encrypt=key) adds Bob's encryption key, so that KeyRing can encrypt for him.

It is possible to have an extension to Umbral that allows Ursulas to independently check whether the kFrag they receive is consistent with the others, without interaction with the rest.

This extension requires that at the end of the split re-key generation, Alice not only generates the kFrags, but also some public information (or at least known by all the Ursulas). This information allows the Ursulas to independently check whether their individual kFrag is consistent with the others; in other words, that Alice hasn't made a mistake with some Ursula (on purpose or not).

Note that this is not a correctness check on the re-encryption key, in the sense that this cannot check whether the fragmented re-encryption key is a valid re-encryption key or not.

A Policy class will create an interface for Alice to craft, inspect, and encapsulate a Policy.

The minimal Policy class has a method for generating an ID.

I've implemented umbral in nucypher-pre-python.

What needs to be done is serialization to/from bytes. I think, it makes sense to do it in a form of a decorator (so that the algorithms stay readable):

class PRE(object):
    @serializer
    def priv2pub(self, priv_key: ec.ec_element) -> ec.ec_element:
        ...

which converts bytes to the type in the typestring when bytes are passed (could be an ec element or a named tuple). Function get_type_hints can help here.

Oracles will determine who needs to pay who (based on which Ursulas are misbehaving, etc). Each of them has a table of who needs to pay who and how much.

After that, oracles bet on the correct result, and the smart contract determines the Truth by the stake-majority vote.

This method can have its own problems, so it could be changed!

The KeyRing method to encrypt text for another person using that person's public encryption key.

keyring.encrypt_for("bob_e24f005b", plaintext, sign=True)

encrypt_for() returns a tuple, where the first element is the ciphertext, and the second element is either the signature or the NO_SIGNATURE constant.

The output/input of the v1 cryptographic API needs to be frozen to prevent breaking changes in the future.

A TreasureMap is valid for an entire PolicyGroup. The TreasureMap is encrypted for Bob and signed by Alice.

Obviously, at a minimum, it includes the node IDs for all Ursulas to whom Alice bestowed a kFrag.

But what else? Some possibilities:

• The public key which each Ursula presented.
• A fingerprint of The Stake which Ursula held at the time.

Maybe we only need the former. But this will establish that Ursula is the same person (or at least, is holding the same Stake) as the person to whom Alice gave the kFrag.

We need a glossary of terms for new contributors to look at and understand the internal development lingo (Alice, Bob, Ursula, kFrag, etc).

Within the README, there needs to be rules for contribution and links to our wiki for the glossary, open projects, and label descriptions.

This will allow any contributors and interested parties to get a quick idea of what is going on and what things they can help out with.

With BBS98, a symmetric key was generated randomly. In ECIES it's a little different: you get a symmetric key (which will be a DH shared secred) generated.

Also umbral shares between two private keys, need to do the same trick as I did with bbs98 converting it to private->public sharing (or, better, make this private>public conversion universal for any algorithm).

@michwill had an idea today of figuring out a way to distribute the Treasure Map via a separate channel form the Alice-Ursula exchange. This way, Policy ids don't need to be reconstructed by Bob and can instead be completely random.

It's interesting, but I'm not sure how to put the pieces together. At least not in the tired state I'm in now.

Where does our thinking move next on this?

We need to implement HKDF-Keccak for the KDF function in our Umbral ECIES code.

To prevent spammers from attacking re-encryption nodes with bad requests and replay attacks, implement a proof of work algorithm like hashcash or equihash.