Problem

Whisper uses gossip for routing. However, this isn't documented in the spec. This makes it hard to know how to do routing, what guarantees there are, as well as to consider moving to different forms of routing.

Acceptance criteria

• Routing and gossip described in spec, along with rough behavior/(none)guarantees

Details

In the original EIP627 only data format was described. Whisper gossip has been loosely described but not in any structured way.

Compare with:

• PSS spec (same data format, Kademlia for routing)
• libp2p gossipsub simulations and various specs (https://github.com/libp2p/specs/tree/master/pubsub/gossipsub etc)

Problem

There appears to be a small mix up. Under the status-options description there are a number of field descriptions, all are present except for confirmations-enabled. In its place is:

https://github.com/vacp2p/specs/blob/master/specs/waku/waku-1.md#message-confirmations-field

This section appears to be describing Message Response packet code 0x0C

Possible Solutions

• Move and/or rename #message-confirmations-field to "Message Response"
• Write a description for confirmations-enabled

Problem

Data Sync messages should have meta data that both change how messages are handled and give extra information on messages.

Acceptance Criteria

There is a way to transmit meta data along with messages, these may be transmitted inside of the message or before. Best alternative should be investigated.

The spec provides a way for information on messages to be provided such as the previous message, which message this is in the sequence of messages along with other things.

The spec also provides a way for information to be transmitted that tells a client how to handle a message, this can include things like whether a message should be acked or not once received.

#4

Problem

Currently the alpha spec says (https://specs.vac.dev/specs/waku/waku-v2.html#protobuf):

The seqno field MAY be used to provide a linearly increasing number. See PubSub spec for more details.

There are some trade-offs involved in using this or not.

Acceptance criteria

1. Decision on what to do with seqno (MAY/SHOULD/SHOULD NOT/MUST/MUST NOT/etc)
2. Rationale in spec

Details

From Pubsub spec:

The seqno field is a 64-bit big-endian uint that is a linearly increasing number that is unique among messages originating from each given peer. No two messages on a pubsub topic from the same peer should have the same seqno value, however messages from different peers may have the same sequence number, so this number alone cannot be used to address messages. Notably the 'timecache' in use by the go implementation contains a message_id, which is constructed from the concatenation of the seqno and the from fields. This message_id is then unique among messages. It was also proposed in #116 to use a message_hash, however, it was noted: "a potential caveat with using hashes instead of seqnos: the peer won't be able to send identical messages (e.g. keepalives) within the timecache interval, as they will get rejected as duplicates."

Possible Solutions

1. Encourage use of seqno for data consistency
2. Discourage use of seqno to not leak that information
3. Something in between (don't lean on it but still support, be agnostic, etc)

Notes

Misc notes from chat log:

• seqno might be used for data consistency
• downside is this message payload is visible to peers
• could there be a modality depending on what metadata protection guarantees we want?
• currently not implemented in nim-libp2p but WIP PR
• useful for finding out what data you are missing, e.g. desktop/historical messages
• depends on how topics are used, can be used for fine tuning but inefficient
• main shortcoming is that you only know you are missing messages if you receive a message where you identify gaps (i.e you know you are missing 2 only if you receive 3, if you never receive 3, you might be missing 2/3)
• additionally unless you know the list of peers publishing on that topic, you might not know of new peers
• ergo topic + timestamp is much more efficient in our scenario most of the cases for data retrieval
• metadata: would reveal how many messages sent on a given app topic

h/t @cammellos for discussion

cc @decanus @arnetheduck @dryajov @kdeme @cammellos

These are things that can be improved in the current remote log design. No specific order. More details on each should be turned into its own issue, unless small enough to do a quick PR.

Spec

• Elaborate on more specific requirements on CAS and NS
• Tweak protobuf naming: response data return type a bit odd
• Considering NameInit RPC for NS
• Tweak protobuf naming: Better name than Pair?
• More explicit sequence number vacp2p/research#1 (Consider making more useful with metadata field, e.g. explicitly list what sequence a message is <local hash, remote hash, data, seqid> this way I can easily sync a messages prior or after a specific number)
• Additionally, consider adding page info to remote log header to know which page a seqid can be found on
• Document random joe node as CAS/NS remote log
• Document how to find initial remote log (e.g. per sync context)
• Document flow w.r.t each operation
• Next page semantics: determine requirements re overlap/adjacent and/or missing entries
• Elaborate on how to achieve 'deletion' of messages
• Document message order requirements for remote log
• Specify that NS is a global PKI namespace, implying authentication (for that specific keypair)

PoC / code

• Extend spike to do full end to end flow as opposed to isolated CAS / remote log
• Do end to end test with IPFS/IPNS, e.g.
  • feasability study https://github.com/vacp2p/research/blob/master/remote_log/ipfs-feasability.md
• Do some PoC that uses MVDS and/or status-console-client (possibly complex due to Nim/Go)

Problem

It was brought to my attention through discussions with @cammellos, @kdeme and a comment (#116 (comment)) a pull request related to mailservers that there is currently no standardized way on how we actually treat packet sizes.

Waku clients limit message sizes they can receive from peers, if they are too large. They are ignored from my understanding. This seems like a logical thing to do. However the way it is done in various implementations is different.

From my current understanding, status-go calculates the size based on the entire packet. Where as the nim implementation calculates the size by using the message size without added metadata.

Acceptance criteria

We need to standardize the way size is calculated. This probably need to be discussed to see the tradeoffs of having metadata in the size vs not. Additionally it may make sense for the size to be dynamic, or adjustable by peer. This way I can allow trusted peers to send me larger messages, this could some how be advertised to the peer so they know what the maximum message size is they can send me.

Background

Setting confirmations-enabled in status-options data packets doesn't seem to have any functional role. Currently the confirmations-enabled setting communicates to the peer that a host will send ConfirmationRequests.

See here for details on previous discussion with @cammellos

Questions

• Why does the node need to know that the peer will send ConfirmationRequests?
• Should the host refuse ConfirmationRequests if the peer doesn't set the confirmations-enabled?

Proposal

• Decide if the confirmations-enabled in status-options data packets serves a useful role
• Decide if deprecating confirmations-enabled is desirable
• Decide if expanding the utility of confirmations to communicate more useful info is desirable
• Implement the agreed changes, if any, in the specs

• Add handshake details
• Outline difference between bridging and data format
• Think about how to maintain forwards capability for waku/v0 (#41)
• elaborate on security considerations (#39)
• Light node security considerations
• Implementation notes and have something similar but lightweight to https://tools.ietf.org/html/rfc8446#appendix-C (ping client implementers)
• Implementation Matrix
• Document further differences with Whisper v6 (Kim?)
• Document recommendations for mobile nodes node
• Describe rate limit (see recently merged PR in status/specs) #40
• Port handshake paragraph from https://github.com/status-im/specs/blob/master/status-whisper-usage-spec.md#handshake
• Whisper Envelope data field (Optional) section weird and has partial ABNF

Mailserver

• Clean up vocabulary, is it mailserver and mailserver client? mailclient? mailserver node? historynode? etc

ABNF

• Run ABNF validator, ensure it parses (can map to rlp shortcut types like '[' or 'rlp list' etc
• ABNF for whisper envelope

Out of scope for waku v0 (create issue only)

• Created issue: Add waku mode in v1, it isn't in the spec yet #49
• Create issue roadmap https://github.com/vacp2p/pm/issues/5

This issue is on how to handle topic-interest versus bloom-filter between peers:

What is there now?

In https://github.com/vacp2p/specs/blob/master/waku/mailserver.md#requesting-messages for the p2p-request packet id it is written for the topic-interest parameter:

• It is optional, which means it may be omitted, may be null (assuming here empty rlp string - 0x80 hex value), or may be a (possibly empty) list.
• omitted or null = bloom filter is checked
• rlp list = bloom filter is ignored no matter the contents of the list

In https://github.com/vacp2p/specs/blob/master/waku/waku.md#abnf-specification, for the topic-interest packet id it, it must be a list, an empty list would mean nothing will arrive. Null is not allowed, considering it is not an optional parameter (obviously or the packet would be useless).

Then the handshake:

In the current scenario, a node would connect to a peer. And until they exchange the topic-interest packet they would be sending/relaying messages according to bloom filter setting. It depends on the implementation, but this could e.g. only occur when the topic-interest gets updated. If it must be send directly after a handshake, this would have to be written in the specification (but that is just silly).

Thus, in order to improve the handshake between clients I think we should add at least a topic-interest parameter to the Status packet id.
Benefits:

• No messages send which a peer does not want (until the topic-interest packet is send)
• Can immediately disconnect from peer in case it has not topics of interest for them

The minimum spec change would be to only add topic-interest as an optional parameter of the Status packet, and handle it in the same way as the P2P Request packet does:

• omitted or null (rlp = 0x80) = bloom filter is checked
• rlp list = bloom filter is ignored no matter the value of topic-interest

Other options would be:

• provide another boolean parameter, e.g. topic-node
• merge the current light-node functionality with that -> This would mean a light node will always work with topic-interest and not bloom filter - This might be not what we want and also cause confusion

While the latter two options are not really necessary right now, it could make it more clear and easy when we want to list "capabilities" through e.g. the discovery mechanism (and perhaps also just make it more clear for the general user and/or developer)

Problem

There is currently no real way of accounting what resources a node is using. aka how many messages peers are sending and receiving, therefore accounting should be added to waku nodes. Ideally, this will also allow rate limiting to be abstracted through the accounting interface.

Solution

Keep track of how much each node contributes and takes from the network based on local information. Similar to Bittorrent seed ratio.

Notes

• https://ethersphere.github.io/swarm-home/ethersphere/orange-papers/1/sw%5E3.pdf
• Bittorrent game theory / economics specs

Problem

At the very beginning, each Status client had only a single list of peers it could connect to. Mailservers and Whisper nodes were separated. As you can imagine, this is not very clever way of sharing peers' addresses but it is simple and works great if all goes fine.

However, later, we started thinking about issues that may occur and had actually occurred. Namely, many of our peers hosted in AWS or GCP were blocked in Russia and China. So, we introduced another set of server peers in Asia + implemented the old PoC Discovery V5 algorithm from geth. When we tested it, it turned out that it consumes a lot of bandwidth and also other resources like CPU and memory. So, we want with a strategy to only turn on Discovery V5 if there are not enough peers connected and we also looked at other peer discovery algorithms like Rendezvous.

All that effort brought more and more complexity, it became harder and harder to measure the source of bandwidth usage and battery draining. It also decreased our velocity.

We need a cleaner, simpler solution for this task.

Acceptance criteria

• We don't need extremely versatile and dynamic solution because we, Status, will very likely continue to be the main maintainer of the infrastructure for the Status clients. However, we need an ability to share new peers' addresses without updating the clients.
• Updating the list of peers must not consume a lot of bandwidth and resources. Ideally, it should not require starting a node so that bootnodes also are not necessary.
• It should be safe, i.e. it should be impossible to spoof the peers' addresses when obtaining them.
• It should be easy to implement and manage.
• The act of obtaining peers should be private and encrypted.

Possible Solutions

1. Node Discovery via DNS + DNS over HTTPS. This is actually only a part of the story because it suggests only sharing bootnodes this way and later use a dynamic peers discovery mechanism. In our case, this might be sufficient as end-to-end solution.

Problem

Waku adds a light node status that isn't documented in the existing Whisper docs. This makes it hard for clients to implement and reason about wrt connectivity, privacy guarantees and perf.

Acceptance criteria

• Document light node behavior in Waku 0 spec
• Add implementation status matrix per implementation, see status-im/specs#63
• Briefly on rationale and implications for privacy

Details

See https://github.com/status-im/whisper/blob/8be20fd33d56bce7a886a93101984d5a1fe101af/whisperv6/peer.go#L80 and https://github.com/status-im/whisper/blob/8be20fd33d56bce7a886a93101984d5a1fe101af/whisperv6/api.go#L194

And https://github.com/status-im/nim-eth/blob/c64a370fe7a49d8d8d287d043870ecec1251676e/eth/p2p/rlpx_protocols/whisper_protocol.nim#L780

Possible Solutions

Look at details and amend EIP627 in https://github.com/vacp2p/specs/pull/28/files

Notes

Check with @adamb and @kdeme if there are any gotchas here

Remote log for data sync

Acceptance criteria

PoC: https://github.com/vacp2p/research/tree/master/remote_log

Spec

Write-up

Problem

Currently apps such as Status use a forked version of EIP627 that isn't properly documented. This makes it difficult to write additional implementations, such as Nim, since a lot of behavior isn't documented.

Additionally, people often confuse "Whisper" with "our (Status) fork of Whisper" which is undesirable terms of being explicit about what requirements we have and what guarantees we provide.

Finally, this is a stepping stone to address bandwidth consumption in a proper Waku mode, which will happen in Waku v1.

Acceptance criteria

• EIP627 replaced with a forked, spec with rationale and replacing Whisper with Waku
• Describe RLPx subprotocol and version numbers
• Waku/0 spec in uniform format with other specs
• Reviewed by swarm and merged to specs.vac.dev

The below are optional for a initial release, and can be addressed in e.g. 0.1 spec release (still Waku version 0). At a minimum, the below should be better understood and captured:

• Describe backwards compatibility with Whisper v6 / v1 (vacp2p/research#14) - sketch in spec

Not solved, but issues created:

• Describes existing mailserver usage, see #36
• Describe existing light client usage, see #29
• Describe current implementation status in terms of capabilities (i.e. geth vs nimbus wrt light node, etc) status-im/specs#63
• Include privacy, scalability and DDoS considerations (see #39)
• Describe existing rate limiting #40
• Describe forward compatibility with waku/1 #41

Details

See:

• https://eips.ethereum.org/EIPS/eip-627
• https://github.com/status-im/specs/blob/master/status-whisper-usage-spec.md
• https://github.com/status-im/specs/blob/master/status-whisper-mailserver-spec.md

As well as implementations in status-im/whisper and status-im/nim-eth

Possible Solutions

See above for acceptance criteria.

Notes

Provide proof that over MVDS we can achieve standardizied message queues with various delivery modes such as at-least once, exactly-once or at-most once. These are concepts we can formalize similarly to AMQP

Problem

Currently in spec it says:

Waku is a privacy-preserving peer-to-peer messaging protocol

as well as:

Privacy. These applications have a desire for some privacy guarantees, such as pseudonymity, metadata protection in transit, etc.

However, we aren't very explicit about how this is achieved and precise guarantees we have. This makes it hard to take seriously as a privacy-preserving protocol.

Acceptance criteria

1. Document privacy properties achieved with protocol
2. Document what is not (e.g. GPA resistant vs Mixnet)
3. Ensure it is useful from a threat model pov

• i.e. a user should be able to see if it fits their need, compared with e.g. Tor/Mixnet/Client-server architecture, etc.

Details

To a large extent, this is about documenting privacy properties for PubSub in libp2p, and minor modifications.

Possible Solutions

Notes

This issue may consist of multiple smaller issues as well for specific properties.

This may be influenced by adaptive node model.

This should also mention encryption guarantees being provided at a separate layer (unlike Waku v1).

#158

Problem

In the Waku spec we currently use an ABNF definition. It would make sense now to standardize around one of the two formats for the rest of our specifications and decide which to take.

I decided to record this in an issue to capture the decision making.

Details

ABNF is used more commonly in compilers, however BNF is more popular in academic papers. Therefore it makes sense to chose one depending on what we are targeting.

Notes

• http://xahlee.info/parser/bnf_ebnf_abnf.html
• https://github.com/Engelberg/instaparse/blob/master/docs/ABNF.md

Problem

When a client sends a message using shh_post, the message ID is returned immediately but it does not mean that the message was pushed through the wire and reached at least one peer. For example, in go-ethereum Whisper implementation, envelopes are batched and broadcasted in a loop. The next thing is that sending an envelope might be over a stale connection and that is also unknown.

In order to provide a meaningful information to the user what happened with her message, we need a message confirmation mechanism.

Acceptance criteria

• The client receives a meaningful information when a message is pushed through the wire.
• The client receives a feedback from at least one peer that the message was received by it.

Possible Solutions

The first point in AC can be either implemented as a blocking shh_post operation or through a notification as we did in our Whisper fork.

The second point in AC requires a new message code and Waku server to send a confirmation for each message. This is also implemented in our Whisper fork.

Problem

Not all packets are currently covered by the spec.

• 11 Batch Ack
• 12 Message Response
• 123 P2P Sync Request
• 124 P2P Sync Response

Acceptance criteria

Align with the PR status-im/specs#72.

Problem

Currently you can only fetch historical messages by bloom filter in https://specs.vac.dev/wms.html

This means you are likely to fetch too much data. See https://vac.dev/fixing-whisper-with-waku

Acceptance criteria

• There's a way to fetch messages by topic, similar to https://specs.vac.dev/waku.html topic-interest

Details

See above

Possible Solutions

Considering the API is currently an RLP encoded list, what are our options for extending it as an optional feature? I suppose appending it? Then we get into the same double-optional issue. cc @adambabik

We could also do a new message code.

This might be an opportunity to switch to protobuf, as this would likely give us more flexibility. @cammellos thoughts?

also cc @decanus for any other thoughts re bandwidth usage and mailserver/remotelog/mvds usage

and cc @kdeme for general thoughts

Notes

Background

Currently a MessageResponse has a version number, but the role of the version doesn't seem evident.

See here for previous discussion with @cammellos

Questions

• In which circumstance would Message Response make a version change that wouldn’t affect the protocol version?

Proposal

• Decide if MessageResponse version numbers are necessary
• If not, remove / deprecate MessageResponse version numbers.

Problem

Currently there's a rudimentary form of rate blocking. This is not captured in the spec, which makes it hard to write an implementation of it.

Acceptance criteria

• Rate limiting captured in specs

Details

See status-im/specs#60 for current description

Possible Solutions

Document it based on above

Notes

See #144 for WIP

This should become a draft that corresponds to what is implemented and what will soon be implemented

General notes and TODOs

General

In general, we want to remove as much as possible. WakuSub should be a thin layer on top of PubSub.

A good question to ask: what is the minimal subset we have to support?

How can we defer specific properties to other protocols?

Data field

Can we assume data field stays the same? There is nothing in data field about RLPx AFAICT:
https://github.com/vacp2p/specs/blob/master/specs/waku/envelope-data-format.md

How do we deal with PoW etc?

If basically only full nodes set PoW to 0 then it be fine. But if light nodes also set this (afair they do) we have a compatibility issue. Though we also have mailservers.

Simplest might might be to basically send both envelopes, just take out the data thing:

• v1 [ expiry ttl topic data nonce ]
• v2 [ ... data ... ]

Get rid of privileged point of node for RPC?

We could have JSON RPC for now and then use alt method later. We need to make
sure the interface with Stimbus and Core/Desktop makes sense.

It'd be useful if a node in browser could use another node without giving away
private keys (WalletConnect, e.g.).

Are filters a useful abstraction?

Not sure I like this filter businesss, just deliver on a topic. If we want to do
that it can be done separately with a shim and have that specified at a
different layer. Removing coupling of routing and crypto.

Consumer spec

Consumer: https://specs.status.im/spec/10

Factoring out

Separate out into constituent pieces, keep main spec small here. What are the pieces?

• RPC/Node API
• Filter shim / crypto key compatibility
• Data field
• Waku v1 bridge

App topics and waku topics

Ensure path forward for sharding topics. See Eth2 networking spec too.

Ensure topic interest in payload (not for routing, only edge nodes). Current Waku topics.

Clarity of purpose

Make it very clear what we provide over e.g. FloodSub.

• Privacy, resource restriction (offline, topic interest, etc)
• Thin layer on top of PubSub

WakuSub type and options

• Highlight that it can be gossip enabled as option

Abstract

Currently we describe the various types of capabilities as different node types, such as bootstrap nodes, mailserver servers, mailserver clients, relay nodes, and light nodes. In some places we describe these as actual capabilities that different things a node can do. This inconsistency and lack of clarity around a node and what it can do makes it harder to do many things. Among others, to reason about and extend things related to specs, analysis, clients, simulations, running a node and explaining things to end users.

A cleaner mental model is to be consistent about it being a node with a set of capabilities. The "X node" then becomes more of a shortcut to refer to certain optimized client configurations. This is more in-line with the adaptive nodes ethos that we want to practice, as well as future enhancements we want to make (libp2p, dagger). It is conceptually simpler and more accurate. If clients implement this correctly, it also simplifies the use of different capabilities for people who want to run their own node.

Specific proposal

Use the following terminology consistently throughout the spec:

1. A node is defined by its set of capabilities.

2. For Waku, these capabilities are:

• send messages
• receive messages, including historical messages
• relay messages
• bootstrap nodes
• store historical messages and return these upon request

3. We use the following shortcuts to refer to nodes with certain set of common capabilities:

• A light node only has send and receive capability
• A full node has all capabilities (send/receive/relay/bootstrap/store)
• A bootstrap node only has capability to bootstrap nodes

4. These common configurations and their capabilities can be cleanly visualized as follows:

• Light node [x] send [x] receive [ ] relay [ ] bootstrap [ ] store
• Full node [x] send [x] receive [x] relay [x] bootstrap [x] store
• Bootstrap node [ ] send [ ] receive [ ] relay [x] bootstrap [ ] store

5. Each capability have a set of requirements. For example, to have the bootstrap capability you currently:

• MUST be connectable through a static IP
• SHOULD be long-running process

6. A client SHOULD be able to specify all (legal) set of capabilities through a simple configuration and the same executable.

7. The previously used "Waku server node", "mailserver node" and "Whisper relay node" is supplanted by simply a "full node" as in "run a full node to make the network more robust".

• This doesn't preclude nodes, e.g. in a specific cluster configuration, choosing to have fewer capabilities. In that case they'll simply be referred to as the set of their capabilities.

Next steps

• Consensus within protocol team
• Update Waku specs with more precise terminology
• Update Status client spec
• Update client configuration and docs to reflect and enable this
• Update infra configuration snad docs to reflect and enable this
• Update relevant places in core app

Notes

• Status client spec: https://github.com/status-im/specs/blob/master/status-client-spec.md

• Current Waku spec: https://specs.vac.dev/waku/waku.html

• UX and confusion around terminology: status-im/design#40

• Nimbus capabilities example: https://github.com/status-im/nim-eth/issues/102#issuecomment-543590468

• Beyond messaging: Add ETH and LES RLPx full node as capability

Background

In the status-go implementation we still use the Batch Ack packet type, which appears to be a less functional version of the Message Response packet type.

https://github.com/status-im/status-go/blob/cc9c352337e401ecab04aca18477ba6ac5606f33/waku/v1/peer.go#L382-L390

// sendConfirmation sends messageResponseCode and batchAcknowledgedCode messages.
func (p *Peer) sendConfirmation(data []byte, envelopeErrors []common.EnvelopeError) (err error) {
	batchHash := crypto.Keccak256Hash(data)
	err = p2p.Send(p.rw, messageResponseCode, NewMessagesResponse(batchHash, envelopeErrors))
	if err != nil {
		return
	}
	err = p2p.Send(p.rw, batchAcknowledgedCode, batchHash) // DEPRECATED
	return
}

Also see previous discussion on this topic

Questions

• In what circumstances would Batch Acknowledge vs Message Response be used?
• What role does Batch Ack serve, that Message Response does not?

Proposal

• Decide if Batch Ack has a purpose
• If not, decide if fully deprecating Batch Ack is desirable
• Remove / deprecate Batch Ack if decided.

Example data sync client for data sync

Acceptance criteria

PoC: https://github.com/status-im/dasy

Spec

Write-up

Problem

• Current spec for waku is 0.2, which doesn't accurately reflect changes last month.

• The changelog section isn't very clear.

• It mentions waku/1 which is confusing as that's not a thing right now.

• Lifecycle of spec isn't specific

Acceptance criteria

Waku specs 0.3 PR merged

Notes

• Make changelog in-line to simplify changes (up to date with each PR) as well as scanning for difference (easier for client implementers)

• Use KISS lifecycle style https://rfc.unprotocols.org/spec:2/COSS/ (can also come in separate PR)

Problem

There is currently no part of the specification that clearly outlines the handshake and how it occurs.

Acceptance criteria

• The handshake is documented within the waku spec.

Problem

We currently do not use multihash meaning we are bound to a single hashing algorithm.

Solution

The message_ids are described to be multihashes.

Problem

The current EIP-627 style spec is (1) verbose and (2) using a custom form of data layout. For clarity of exposition, and to pave the way for machine readable specs, a more standard formal language is desired.

Acceptance criteria

• ABNF format used consistently

• Gone through some form of validation program, e..g Panini

• Interface points with RLP are clarified (ie deferring to RLP as de facto format, etc)

• Style: still horizontal scroll overflow on Waku spec

• pow is no defined, grammar ill defined

• optionals ambiguous x2 #52

• p2p mailserver spec wrong, 1 envelope or multiple

Details

Why ABNF? It is standard for internet protocols, i.e. see https://tools.ietf.org/html/std68 and https://en.wikipedia.org/wiki/Augmented_Backus–Naur_form

It is used to describe many communication protocols, email etc.

Possible Solutions

A start exists in Waku 0 PR:

packet-format ::= "[" packet-code packet-format "]"

; Packet codes 0 - 127 are reserved for Waku protocol
packet-code ::= 0 / 1 / 2 / 3 / [ x4-127 ]

; packet-format needs to be paired with its corresponding packet-format

packet ::= "[" required-packet / [ optional-packet ] "]"

required-packet ::= 0 status / 1 messages / 2 pow-requirement / 3 bloom-filter

optional-packet ::= 126 p2p-request / 127 p2p-message

packet-format ::= "[" packet-code packet-format "]"

status ::= "[" version pow-requirement [ bloom-filter ] [ light-node ] "]"

; version is "an integer (as specified in RLP)"
version ::= integer

messages ::= *whisper-envelope

; pow is "a single floating point value of PoW. This value is the IEEE 754 binary representation of 64-bit floating point number. Values of qNAN, sNAN, INF and -INF are not allowed. Negative values are also not allowed."
pow-requirement ::= pow

; bloom filter is "a byte array"
bloom-filter ::= bytes

light-node ::= bool

p2p-request ::= whisper-envelope

p2p-message ::= whisper-envelope

; todo: pow, whisper-envelope, version, bloom-filter, etc

Notes

Problem

Certain applications like ultralight beam may see peers changing the method of which they are accessible frequently without notifying peers.

Solution

A max retries option is added which informs the MVDS node how many times a message should be retransmissioned, if it can not be sent to the peer in those retries the application is notified and can then handle this gracefully by for example trying to reach the peer through some other means or delaying until it knows that a peer is online again.

Notes

Due to how ultralight beam handles peers, this may not be that relevant assuming we send to a public key.

Initially we managed the changelog in a markdown table when things were small, in waku we then moved to a more comprehensive list. We should update all the specs to use the same standard, so probably best to use what waku uses.

Additionally, we should ensure there are repository tags for every unique spec release.

Problem

Currently the spec is in raw mode and there are some glaring omissions. We want to have a draft spec that we can share with the community and stakeholders.

Acceptance criteria

1. Draft means it corresponds to what is implemented in at least one implementation (nim-waku)a
2. All glaring omissions should be documented (some specific enhancements may be out of scope to not drag out)
3. Merged PR with more reviewers and and released version

Details

Here's a list of some issues that have to be addressed before this can be done:

Specs

• #149
• #150
• #151
• #152

Implementation

TODO.

Other?

Please comment below if there are more things missing.

Possible Solutions

Notes

Problem

Currently we treat all specs as roughly the same. Some are mere sketches, whereas others are in production. This problem will only grow as we produce more specs.

Solution

Use lightweight version of e.g. https://rfc.unprotocols.org/spec:2/COSS/ with

Raw -> Draft -> Stable -> Deprecated -> Retired -> Deleted

To clarify which state a spec is in. The document linked above outlines what each state means.

Acceptance criteria

• Agreement on this process
• Documented in this repo, e.g. LIFECYCLE.md
• All specs have a lifecycle state attached to them
• Optional: Nice color for which state

Problem

Currently the waku spec is rather larger and contains secondary parts to the base protocol that do not need to all be in one specification.

Acceptance criteria

The specification is logically split up into multiple specifications, it would make sense for things like lightnode, mailserver, whisper envelope and other things to be completely separate.

Problem

We want to be able to build on Waku and create new version with new capabilities.

Acceptance criteria

Should be in forward compatibility section in Waku spc:

• Documented clear path from waku/0 to waku/1 and waku/2
• Documented understanding of when we can avoid bridging and when we can't
• Documented understanding of if we can keep being connected with mismatching version numbers
• Documented and generalized understanding of additional packet code capabilities and ignore diff
• Sketch of how we can achieve some of the things we are likely to do (see details)
• Understanding/sketch of how protocol negotiation would work (simplified by libp2p), e..g with MVDS on top (but not for public chat etc)
• Understanding of how and if minor version makes sense

Details

Example of capabilities we want to create:

• changing version number to 1;
• moving to libp2p; changing routing to PSS style;
• remove PoW;
• replacing PoW with zkSNARKs;
• adding packet codes for rate limit / accounting for resources feedback;
• additional disconnect features
• etc

Current version logic:

    if m.protocolVersion == wakuVersion:
      debug "Waku peer", peer, wakuVersion
    else:
      raise newException(UselessPeerError, "Incompatible Waku version")

Is this what we want?

Possible Solutions

TBD.

Notes

Subprotocol for packet reliability

The Waku spec currently uses a lot of definitions and roles that are not explicitly defined, define these at the beginning of the spec.

Problem

Currently Whisper/Waku has many different capabilities and modes which impact various forms of security considerations. Right now it isn't clear what the impact is, which means the spec isn't clear about its guarantees and client users might be at risk against various threat models without being aware of it.

Acceptance criteria

Security considerations to capture in Waku spec under Security Considerations section:

Scalability / UX:

• BW usage theoretical model link
• Mailserver HA requirements or not (points to MVDS, but also not always [public chat wrong level but hint at it])
• Gossip scalability issue (points to e.g. pss routing)
• Lack of incentive centralization factor (points to e.g. accounting for resources

Privacy:

• Light node privacy
• Bloom filter privacy
• Mailserver privacy
• (waku v1) waku mode privacy
• General disclaimer on privacy not studied rigorously for Whisper; same signature (vs e.g. Sphinx/Mixnet)
• Topic hygiene (tradeoff bw)

Spam resistance:

• PoW low bad for heterogenerous devices
• Mailserver trusted direct TCP

Censorship resistance:

• Devp2p TCP port blockable (points to libp2p use)

Details

Some are captured in https://github.com/status-im/specs

Possible Solutions

Write it down in Waku specs.

Notes

Problem

https://github.com/vacp2p/specs/blob/master/wms.md doesn't use ABNF for field descriptions

Acceptance criteria

It should, like Waku spec

Problem

Currently we have a waku rpc protocol which is based off of the whisper rpc. This however is not documented anywhere.

Acceptance criteria

There exists a specification for the waku rpc that documents all commands, their responses and how they work.

Problem

Currently the alpha spec says (https://specs.vac.dev/specs/waku/waku-v2.html#protobuf):

The signature field MAY contain the signature of the message, thus providing authentication of the message. See PubSub spec for more details.

There are some trade-offs involved in using this or not.

Acceptance criteria

1. Clarity on whether we want to rely on signature field or not
2. Documented with rationale/caveats/recommendations (e.g. sig above layer?)

Details

Currently the signature field has some questionable choices:

• they are relying on protobuf serialization being consistent across peers
• they also can't add I think any field in the protobuf without breaking signature scheme, depending on the version used

However:

• doubtful protobuf will change the behavior, many people already incorrectly rely on it

See libp2p/specs#276 for upstream issue

Alternatively, to add fields without breaking signature scheme:

• pass the encoded payload and sign that one { bytes signature; bytes payload}
• then unmarshall that one and verify against already marshalled payload
• never marshal to verify the signature (indeterministic)
• and you don't care about the fields inside the signed payload (future proof)
• (the author of protobuf suggests this method as well)

h/t @cammellos for ideas

Possible Solutions

1. Use signature field as is
2. Use signature field with caveats (e.g. recommend not relying on it too much, etc)
3. Discourage usage of signature

Notes

Problem

A client might only have certain transports available. To sync messages, the clients need to know how to reach each other over which transports.

Acceptance criteria

A spec and proof of concept for a data sync client that allows:

• storing of transport information
• switching of preferred way of being contacted

Notes

See https://code.briarproject.org/briar/briar/wikis/Transport-Properties-Client

Also see protocol negotiation in libp2p.

Example: Whisper / MANET / USB sync

Problem

Currently we are using a mix of hex (RLP) values and int values for message codes.

Acceptance Criteria

We use either hex or int values consistently throughout the entire spec.

Problem

Currently https://specs.vac.dev/waku/waku.html#appendix-b-implementation-notes seems a bit out of date, and it also lists Whisper etc.

Acceptance criteria

Simplified implementation matrix merged that is (a) up to date (b) clarifies the diff between clients

Problem

Currently the Waku spec contains no mail server documentation, the implementation is not well specified enough.

Solution

Once there is some description of mailservers in status/spec, it should be added to the vac/specs.

Notes

• https://github.com/status-im/specs/blob/master/status-whisper-mailserver-spec.md

Problem

Currently in spec it says:

Waku is a privacy-preserving peer-to-peer messaging protocol for resource restricted devices.

as well as:

Resource restricted.These applications often run in constrained environments, where resources or the environment is restricted in some fashion. E.g.:

limited bandwidth, CPU, memory, disk, battery, etc
not being publicly connectable
only being intermittently connected; mostly-offline

However, we aren't very explicit about how this is achieved and precise guarantees we have. This makes it hard to take seriously as a resource restricted optimized protocol.

Acceptance criteria

1. Document what resources restrictions we deal with, and how
2. Document resource restrictions we currently don't deal with
3. Ensure it is useful from a developer/user running in a resource restricted environment POV

Details

This issue may be split up into several smaller issues. For example, historical messages (missing corresponding spec issue, cc @decanus waku-org/nwaku#38) would likely be one.

Possible Solutions

Notes

This should also mention running from a browser

Background

The status-options setting light-node would more clearly communicate its boolean type by changing the name to is-light-node. This isn't a major change and on its own I wouldn't suggest this but in the light of the other potential breaking changes, perhaps we could also consider this one.

Proposal

• Change the name of light-node to is-light-node

Problem

Waku 0 / Whisper v6 takes a lot of receive bandwidth, especially as the number of messages in the network increase.

Acceptance criteria

• Description of additional opcodes in spec

• Recommendations for clients section

• Pay attention to mailserver usage

• Issue created: PoC that shows difference in usage (possibly factor this issue out) - https://github.com/vacp2p/simulations/issues

Details

See https://discuss.status.im/t/fixing-whisper-for-great-profit/1419 for more analysis

Also:

status-im/specs#53 and status-im/specs#54

Boils down to this additional capability:

1. Modify EIP627 by adding a new packet code, e.g. Topic List [101, bytes] where bytes is a list of N topics.

As well as recommendations to clients re topic usage, nodes connected to, etc.

Additionally: mailserver usage, both API as well as node selection, etc.

Problem

Currently there is no method for knowing what version a waku node runs. In case there are changes in the future that are not fully backwards compatible there needs to be some for of protocol negotiation.

Possible Solutions

We expand the status message in such a way that waku versions are exchanged. There should also be a strategy for breaking changes, such as potentially implementing bridge nodes.