Sorry if this is the wrong place for this. I was reading through the spec for mplex, and it explicitly says it doesn't handle backpressure. When run over the JS implementation of stream muxer, I'm pretty sure the pull-stream based design will handle backpressure, but it looks like the Go implementation is just Reader/Writer/Closer, so how is backpressure handled in that case?

(some preliminary brainstorming based on a discussion with @whyrusleeping, @bigs)

Goal

Revise/update/improve libp2p specs and provide clearer usage guidance

List of proposed documents

1. Base wire formats

• multistream select
• multiplex (#96)
• yamux (https://github.com/hashicorp/yamux/blob/master/spec.md)
• secio (#106)
• Peer IDs (#100)

2. libp2p book / Usage

(some of these could be considered specs, others would be non-mandatory "suggested usage" guides)

• a doc describing exactly how these are tied together, a spec for the ‘swarm’
• document describing how dialing works.
• document describing the ‘libp2p api’. Which is not language specific, but a broader description of the capabilities of each subsystem in terms of what it does.
• Better diagrams (see https://github.com/libp2p/libp2p/issues/55)

3. Libp2p ‘builtin’ protocols

• Identify
• Ping
• pubsub
• DHT (#108)
• Relay

4. Other

• probably also something on NAT Traversal
• Also need a multiaddr spec (there may exist one, but lets make sure its "proper")

Format notes:

• Everything in markdown, organized into a gitbook. Gives us a nice website UI, or users can print it all to a single PDF.

The GossipSub spec defines a NEIGHBOR message with a peers field. It is used for

• accepting JOIN requests
• requesting connections to near neighbors
• accepting a neighbor request

But for none of these any other nodes than sender and receiver seem to be necessary. So what's the point of peers and how is it used?

In the current description of when a router joins a topic

On JOIN(topic) the router joins the topic. In order to do so, if it already has D peers from the fanout peers of a topic, then it adds them to mesh[topic], and notifies them with a GRAFT(topic) control message. Otherwise, if there are less than D peers (let this number be x) in the fanout for a topic (or the topic is not in the fanout), then it still adds them as above (if there are any), and selects the remaining number of peers (D-x) from peers.gossipsub[topic], and likewise adds them to mesh[topic] and notifies them with a GRAFT(topic) control message.

If we break this down, we have 2 cases:

• If the router contains D peers that are subscribed to a topic that are not in mesh[topic] i.e. they are in fanout[topic]

• If the router contains less than D peers that are subscribed to a topic that are in fanout[topic]

From the description that I copied over, it seems like the main difference in both cases is that in the second case, we are sampling from the D-x peers of peers.gossipsub[topic] and adding them to mesh[topic] in addition to adding the x peers of fanout[topic] (regardless of which protocol they are using) to mesh[topic]. Is this right? The second portion of the paragraph is unclear about this.

In the reference implementation, it seems like an optimization was done since both cases are very similar. Is this right?

I am a student trying to implement the bitswap module in java. My current understanding is the Kademlia DHT features are provided by the libp2p. Is there any interface that I could use the libp2p library with to make my bitswap implementation in java leverage this library ?

Question: Given that QUIC has built-in crypto, how should we associate PeerIDs with QUIC connections.

Our options are:

1. Do it ourselves. That is, establish the connection and then perform an additional handshake to authenticate it. Nobody likes this option.
2. Use our peer keys as our QUIC keys. This is reasonably simple and the closest to secio.
3. Given that, as far as I know, TLS 1.3 will require that all keys have certificates, we could generate ephemeral keys for QUIC and then sign them with our peer key. That is, there must be a certificate chain from the peer key to the QUIC key. This would allow us to (a) keep our key offline and (b) have interesting identity structures (may, eventually, tie in with DIDs).

@whyrusleeping @Kubuxu @lgierth @marten-seemann

I'm a bit confused about what the difference is between the peer-routing and discovery interfaces.
Could you elaborate on the specific functions that each interface provides, or some reading material that provides some context for the interfaces?

Thanks

In an attempt to converge all the relay spec notes and discussions, I'm listing them here, so that they can all be considered for the revamp of the relay spec. (Order represents the timeline of how theses appear, last should be considered as last 'discussed')

• first iteration of the spec - https://github.com/libp2p/specs/blob/master/relay/README.md
• question: wire protocol message format - #12
• question: relay vs tunnel - #8
• implementation notes - https://github.com/libp2p/js-libp2p-circuit/blob/master/implementation-notes.md
• Why's concerns about current implementation - https://github.com/libp2p/js-libp2p-circuit/issues/8
• Relay use cases - https://github.com/libp2p/js-libp2p-circuit/issues/6
• moar notes - https://github.com/libp2p/js-libp2p-circuit/issues/2
• moar discussion - https://github.com/libp2p/js-libp2p-circuit/issues/4
• errors table: https://gist.github.com/whyrusleeping/7665164d04e43171e6cb7b8aa6205645#file-errors-md
• wire protocol: https://gist.github.com/whyrusleeping/7665164d04e43171e6cb7b8aa6205645#file-relay-md

The gossipsub spec refers to message IDs, but doesn't actually include a message ID field in the protobuf file (and neither does the pubsub spec). Having such a field would be useful for implementing a conversion (using a From trait with Rust) between a message ID and a message, e.g. like so:

impl From<GMessage> for MsgId {
    #[inline]
    fn from(message: GMessage) -> Self {
        message.id
    }
}

For more details see message.rs (as part of libp2p/rust-libp2p#767).

cc @vyzo, @mgoelzer, @whyrusleeping

Right now dialing a node and upgrading with identify will make the node return the observed address of the dialer, plus the listening address of the listener.

However in practice, because of NAT traversal, in order to report the address it's listening on, a server needs to know how it's being perceived by the outside.
This introduces the possibility of a cyclic dependency if two nodes try to identify each other.

Separating the observed address from the listening address would make the whole system cleaner in my opinion.

As per the discussion in #32, there is a need for nodes to know what protocols their peers support so that modules (like pubsub, etc) can know whether they should dial into a protocol before they actually attempt this.

The identify protobuf already supports a repeated string field called protocols which I assume was made for this purpose.

I propose that we add a couple of lines of code to the various libp2p implementations in order to start adding protocols to the local PeerInfo of a node in order to have it passed on when a peer attempts to connect to them.

In the future, we will likely need to add extensions to the Identify protocol in order to allow peers to receive updates when a node gains or loses a protocol. (Not sure if there's something like this for multiaddrs already).

With that said, the following needs to be done to facilitate this:

• [ ] Modify js-libp2p-swarm to add the protocol to their PeerInfo object whenever a new handler is added
• [ ] Modify go-libp2p to add the protocol to their PeerInfo object whenever a new StreamHandler is set
• [ ] Make sure the identify protocols of both implementations properly pass the protocols
• [ ] Make a protocol that waits for the identity service to finish before trying to connect

Hopefully this is something the community wants to have and will be useful for coordinating between nodes with vastly different capabilities and versions in the future.

The next step to getting relays working is to figure out how to advertise addresses proxied through a relay.

Random thoughts:

1. Ignore unspecific relay advertisements. That is, ignore all advertisements of the form /p2p-circuit/ipfs/QmId (no relay specified). These are pretty much useless and could be used to trick nodes into doing a lot of work.
2. Have relays publish p2p-circuit addresses for their relayed peers. This will get us one step closer to using circuits for privacy. The downside is that advertising these addresses could be a lot of work.

I've been reading the spec and noticed the line:

Also note that as part of the pubsub api, the peer emits SUBSCRIBE and UNSUBSCRIBE control messages to all its peers whenever it joins or leaves a topic.

After briefly looking at the go-implementation, these messages don't seem to be explicitly sent in the join/leave functions.
See the Leave function for example:
https://github.com/libp2p/go-libp2p-pubsub/blob/2e1fb613f98d27ecc07f3800137daeab45ddb02e/gossipsub.go#L287

It sends a PRUNE to all peers in it's mesh. Should it also send UNSUBSCRIBE to all peers? Or have I misunderstood or missed something?

Is it the case that when a peer unsubscribes, it sends the UNSUBSCRIBE to all peers and then calls Leave?

Currently there is no way to dump the keys in order to decrypt traffic for debugging purposes.
I am currently writing https://github.com/mkg20001/libp2p-dissector so I wanted to request this here instead of creating my own unofficial hack.

This is mostly a suggestion/idea.

The relay protocol currently has two modes: HOP and STOP.

Instead of negotiating /libp2p/relay/circuit/0.1.0 then requesting a mode, it could be a good idea to have two different protocols instead. For example /libp2p/relay/circuit/0.1.0/hop and /libp2p/relay/circuit/0.1.0/stop.

The advantage is that most nodes would accept acting as a destination, but not all accept relaying. By splitting them we would know earlier on whether a node accepts relaying.

When building multi-hop relayed connections, there can be multiple modes for how exactly to dial. There are two obvious modes, but both of them are currently blind spots in the circuit relay spec.

(All kinds of other modes are also possible -- these two are just the most obvious ones.)

Hop-by-hop dialing

• How it dials, given A->R1->R2->B
  1. A connects to R1
  2. A builds A->R2 connection, via R1
  3. A builds A->B connection, via R2
• Formula
  • Relay(Relay(Relay(A, R1), R2), B)
• Properties
  • A only sends the very next hop's multiaddr as the header
  • A is in full control of the circuit
  • A doesn't rely on /hop support on R1 or R2
  • More resource-intensive the more hops there are, light clients will probably rather use deferred dialing
  • Slight overhead per hop due to multiple crypto channels and stream muxers.
• Naming
  • previously dubbed "onion", which we shouldn't use anymore due to confusion with Tor.
  • proposal: matreshka (russian doll)
  • proposal: shallot (small onion, pic)

Deferred dialing

• How it dials, given A->R1->R2->B
  1. A connects to R1
  2. R1 connects to R2
  3. R2 connects to B
  4. R1 builds R1->B connection, via R2
  5. R1 builds A->B connection, via R1->B
• Formula, Header
  • Relay(A, Relay(R1, Relay(R2, B)))
• Properties
  • Depends on R1 and R2 allowing passing /p2p-circuit addresses as headers
  • Resource-saving on A, good for light clients
  • Slightly less overhead from multiple crypto channels and stream muxers
• Naming
  • previously dubbed "telescope", which is a good name. I2P's relay construction is called the same and operates similarly, but with I2P the name isn't part of the branding (as with Tor), so it's fine to use it too.

At the beginning of the ambient peer discovery section, it starts with

With ambient peer discovery, the function is pushed outside the scope of the protocol.

Is this meant to say that floodsub doesn't depend on a particular method of peer discovery, e.g. a DHT, but depends on peers themselves to announce themselves to other peers. Then, the protocol will check if that peer is running floodsub or gossipsub and responds accordingly? Not really sure what function is pushed outside the scope of the protocol is suppose to mean.

Just chatted with @Mikerah today, and she had the impression that our specs are targeted at audiences with significant pre-existing P2P background. In the call, there was general consensus that this should not be the case; specs should be accessible to technical audiences with no specialised background.

IMO, a good archetype for accessibility could be the HTTP 1.1 spec.

Some improvement ideas:

• Glossary at the end of each spec: defining terminology or terms that the author is using but have no industry-wide consensus (e.g. ambient).
• Implementers' guide: to be clarified (@Mikerah).
• Much more pseudocode, especially for the algorithmic definitions.
• Comparative content. For example, in the current gossipsub spec, we could do with a comparison between floodsub, meshsub, randomsub, gossipsub in terms of use cases, dissemination patterns, etc.
• Diagrams; in the context of gossipsub, how do all the data structures (peers, window, etc.) relate to one another; how RPCs affect state; a control flow chart for the heartbeat algorithm, etc.

Forks off #111 to focus on the backwards compatibility issue instead of the CID/PID design space.

Summary

Peer ID calculation has changed a couple of times over the past year.

1. Initially, peer IDs were always sha2-256 multihashes of the public key.
2. From go-ipfs 0.4.16-0.4.18, we used the identity multihash to "inline" small keys (specifically, ed25519 keys) into the peer ID under the assumption that nobody was using them.
3. We reverted (2) in 0.4.19 dev as OpenBazaar was using ed25519 keys before 0.4.16.
4. We are now planning on re-introducing (2) as:
   a. Textile is also using ed25519 keys and is relying on 0.4.18 behavior.
   b. OpenBazaar is using a forked go-ipfs network. We are adding a package-level flag so they can restore pre-0.4.16 behavior in their forked build.
5. Finally, we need to agree on a migration path such that all IPFS nodes use the same method to calculate peer IDs.

PR: libp2p/go-libp2p-peer#42

Goals

This all grew out of several requirements and wants:

Requirements

• We need to provide a way to for users to select a hash function other than sha256 when computing their peer ID.
• Given a public key, peer ID calculation must be deterministic.
• There must be a case-insensitive way to express peer IDs appearing in IPNS paths (for browsers). Currently, peer IDs are always base58 encoded which is not case-insensitive.

Wants

• We'd like a way to "inline" public keys into peer IDs (if the public key is small enough to fit comfortably). This will allow encrypting a message to a peer without needing to look up their key first.
• We'd like to be able to fetch keys with bitswap.

Definitions

1. Inlining peer ID (working title...): A peer ID from which the associated
   public key can be extracted.
2. sha256 peer ID: A peer ID created using the sha256 hash function.

Note: "inlining" peer IDs look like 1..., sha256 peer IDs look like Qm....

Events

Below is an exhaustive history of this issue:

1. In the beginning, peer IDs were sha256 multihashes of peer IDs.
2. At some point, we added support for ed25519 keys in go-libp2p. The plan was
   to embed them in peer IDs
   (libp2p/go-libp2p-crypto#5). However, we punted on
   that. We never added support to go-ipfs
   (ipfs/kubo#3625).
3. OpenBazaar started using ed25519 keys anyways.
4. We (go) added a separate function for calculating inlining peer IDs (in
   libp2p/go-libp2p-peer#15) that embedded ed25519 keys. Unfortunately, that
   wasn't usable because everyone needs to compute the same peer ID.
5. In libp2p/go-libp2p-peer#30, we removed this separate function and
   switched to automatically embedding keys shorter than 42 bytes into peer IDs.
   This way, everyone would deterministically calculate the same peer ID. We did
   this by using the "identity" hash function instead of sha256.
6. Textile started using go-libp2p and go-ipfs (using ed25519 keys).
7. OpenBazaar tried to rebase onto go-ipfs 0.4.18 and discovered that peer ID
   calculation had changed.
8. After discussing the issue in libp2p/specs#111 and on a
   call, we decided to revert 5. We were under the
   impression that nobody else was using ed25519 keys given that go-ipfs doesn't
   provide a way to generate ed25519 keys.
9. Textile reached out to @whyrusleeping about a weird bug they were seeing when
   trying to connect two nodes. @whyrusleeping tried reproducing it got the
   error dial attempt failed: <peer.ID Qm*yNGz7a> --> <peer.ID 12*FrJvar> dial
   attempt failed: connected to wrong peer". This is the inverse of the issue
   OpenBazaar was having.
10. At the moment, it doesn't actually look like this is the issue Textile was
    having, they're seeing timeouts: dial attempt failed: <peer.ID 12*xWYT4W> --> <peer.ID 12*YsNMRE> dial attempt failed: context deadline exceeded and
    dial attempt failed: <peer.ID 12*xWYT4W> --> <peer.ID 12*YsNMRE> dial attempt failed: dial tcp4 13.57.23.210:4001: i/o timeout. The latter looks
    like it comes from libp2p/go-tcp-transport#24.

This issue covers the peer ID issue, not the timeout issues.

Current State

1. Most of the (go) network is using the inlining from libp2p/go-libp2p-peer#30.
2. The latest go-ipfs and go-libp2p masters are not inlining keys into peer IDs. They are now (again) inlining keys.
3. OpenBazaar is using a forked (separate) network so, for now at least, they
   don't actually need to interoperate with the rest of the network.
4. Textile is not using a forked network so they do need to interoperate. Textile is using peer IDs for identity but these could (potentially) be migrated. They are not using IPNS.
5. IPLD-DID (decentralized identity) is using the new inlined keys for IPNS. However, it's unclear if they have many/any users who would be affected.

Affected Subsystems

This covers the affected subsystems and some hacky fixes that I don't recommend.
They're only there to illustrate the issue.

Outbound Connection

When establishing an outbound connection, go-libp2p will:

1. Perform a secio handshake.
2. Derive the remote peer's ID from their public key.
3. Check if this derived peer ID is the target peer ID.

Textile is seeing this fail after updating go-libp2p because they're passing a
peer ID created using the identity hash function into go-libp2p while
go-libp2p-peer is calculating it using the sha256 multihash.

Example Fix: This could be fixed by a hack to convert a peer ID created using
the identity hash function to a sha256 one.

Inbound Connection

When receiving an inbound connection, we compute the peer's ID from their public key. That means:

1. If we're running a 0.4.18 go-ipfs node, we'll compute inlining peer IDs for ed25519 keys.
2. If we're running a 0.4.19-dev go-ipfs node, we'll compute sha256 peer IDs for ed25519 keys.

Example Fix: In practice, this "just works". That is, we use our version of the
peer's ID internally and don't really care what the other side thinks it's ID
is.

DHT

The DHT is the first place where we really care about calculating the same peer
IDs. If we don't, FindPeer breaks.

Let's say there's some peer A that wants to connect to a peer B. Let's assume
that peer A and B have this peer ID inlining enabled but the rest of the network
doesn't.

When peer A tries to connect to peer B, it'll walk the DHT looking asking nodes if they either:

1. Know of nodes closer to peer B.
2. Know how to connect to peer B. Currently (in go-ipfs), only nodes that are
   actually connected to peer B will respond.

In this case,

1. Peer A will be able to walk the DHT all the way to a node directly connected
   to peer B. That is, (1) will work.
2. That node will know peer B by a different name (ID) so it won't be able to do part (2).

Example Fix: So, if we have an inlining peer ID, we can extract the key and
compute the sha256 peer ID and try to look that up in the DHT. Unfortunately,
we can't go the other way. This fix would also be really hacky.

IPNS

I don't believe IPNS is affected but I haven't thought through it thoroughly. At
worst, we'd have to apply a fix similar to the outbound connection fix.

PubSub

Same as IPNS.

Multibase

Unfortunately, this whole issue also relates to the ask from IPFS In Web
Browsers to make IPNS (and peer IDs) use multibase. If we just have Qm...
IDs, we can avoid allocating Q as a multibase prefix and we'll be fine.
However, inlining peer IDs start with 12 and 1 is already a valid (albeit
useless) multibase prefix for unary.

The real worry is that if we allow arbitrary mulithash functions, we need to
tackle the multibase issue before we start running into collisions. That is,
hash codes such that Base58Encode([hash code]) maps to some useful multibase
prefix.

creating this issue here first for lack of a better place

Now that we have #44, I think we should start thinking about an additional interface for service discovery. The rational being, in the past rendezvous services (servers), provided a peer discovery mechanism, the proposal in #44 satisfies this requirements and can be adapted to be used as a pure peer discovery mechanism that implements interface-peer-discovery. However, we're lacking an interface to describe the new operations that rendezvous brings about, namely registering and discovering peers under a specific namespace. Defining an interface allows us to implement this functionality using alternative mechanisms, such as DHT.

A rendezvous service can satisfy this two use cases - pure peer discovery defined under interface-peer-discovery and service discovery interface-service-discovery.

Here is a very rough first draft (pseudo flow):

// interface-service-discovery

interface DiscoveryResponce {
  timestamp: int64;
  peers: PeerInfo[];
}

interface ServiceDiscovery {
  register(namespace: string, peer: PeerInfo): bool;
  discover(namespace?: string, limit?: int, since?: timestamp): DiscoveryResponce;
} 

I'm still not %100 sold on the naming, so take it with a grain of salt atm. The gist of the operations I want to convey with this interface can be summarized with:

• A peer wants to be able to register and be discoverable under a certain namespace
• A peer wants to discover other peers registered under a certain namespace

EDIT:

• marked discover params as optional (? means optional)
• added the word peer to in the past rendezvous services (servers), provided a...
• added DiscoveryResponce type to allow returning a timestamp along side the peer
• corrected DiscoveryResponce to return a list of peers and a timestamp

I am working on some P2P idea and reading lot on Libp2p as well as Kaledmia Protocol. Is there any community call or can have conf call to understand more whether Libp2p can be used for our usecase or not ?

so it's consistent with the other specs repos

Currently, there's a lot of overhead when each connection is established opening up multiple streams to figure out what services the other peer supports. This also tends to be a bit racy and really messy.

Instead, we could introduce a push-based service discovery system. On connect, a peer would announce what protocols it supports up-front (the list shouldn't be that large) and whenever a new service gets added/removed, the peer would tell its peers about this change. We could then avoid an entire round trip when opening new streams because we already know the protocols our peers support. This would also remove the need for a lot of our peer probing code (we could just have the host keep track of which connections support which protocols).

This would also make it possible to implement a pseudo message-oriented system on-top of our stream-oriented communication primitives. Why? Because most consumers of libp2p in go-ipfs are already (mostly) message oriented. That is, we have, for almost every service:

1. Something to keep track of peers and metadata about them (and keep track of the status of the connection).
2. Go routines to accept messages from peers and pipe it into a central event loop.
3. Go routines to send messages back out to peers. Note: they are sent back to peers in-order, this is the only non-message oriented part of this and why implementing this on-top of streams actually makes sense.

We shouldn't have to implement this over-and-over for every service; this is where a lot of our leaks and race conditions come from...

If we had this central service registry, we could use it to transparently manage all streams, opening them as necessary and possibly even managing associated metadata.

Having separate multistream endpoints for hop and stop allowed to quickly determine if a peer had relay capabilities, with the removal of the separate multistream endpoints this isn't possible anymore.

@vyzo and myself discussed the possibility of adding a new message type that would allow determining if a host is a relay or isn't. This is crucial functionality as it would otherwise require to resort to ugly hacks to know which hosts are relays and which aren't.

A couple I can think of:

• Send a HOP message with an invalid peer id (or somesuch) and check for an explicit CANT_SPEAK_RELAY response code, if its something else then assume it is a relay.
• A (arguably) worst way would be to simply dial every peer in the peerbook asking to relay to the desired destination, this is both slow and generates unnecessary load on the network.

In the current js implementation, I was trying over the hop endpoint on every incoming connection and adding it to an internal relay table that I would use when dialing over a relay address. This is IMO more efficient and cleaner.

This proposal tries to address this shortcomings by adding an explicit CAN_HOP message that would allow quickly asking a peer if it is a relay.

The CAN_HOP message either succeeds or fails and doesn't provide any additional information, leaving further capability discovery functionality to the already proposed ls protocol - ipfs/notes#237.

@diasdavid @whyrusleeping @lgierth

Problem:

1. Frequent data publish may lead to flooding and should be limited

2. Data should be time-signed to prevent the publish of large amounts of data pre-prepared (attack)

Suggest:

1. Published data with difficulty signature
        signature(sha256(sha256(data + time + nonce))) < 0x0000012345...
   so massive delivery requires a price

2. Time is defined as the (Time On the Chain), not the real time, is the latest block hash(filecoin, btc,....), so it can't be pre-made.

3. Implement offline message save and route (I am researching this so I don't have to work repeat...
   :-) )

thomas92911 -- (ARS.China)

I am building a torrent client, and I'm wondering if I can use libp2p to power the actual torrenting engine related to bittorrent spec. There would appear some benefit(future proof to swap out or add other types of transfers) and some downsides(starting from scratch as there is zero reference anywhere to this anywhere).

I just want to question if this is possible, feasible, and possibly encouraged. I've been poking around this code and spec for a bit, and want to actually do something with it, but I'd like to make sure I don't have the wrong idea about what use cases I can create.

In the Joining the Overlay section of the GossipSub specification a GETNODES message type is mentioned, but not how receivers should handle it. My guess is that they should respond with a NODES message containing a subset of their active and/or passive list. Is that correct? If so, what's the point of the ttl field in the NODES message?

The gossipsub spec says that the i_want function "requests transmission of messages announced in an IHAVE message" and "forwards all request messages that are present in mcache to the
requesting peer." I think it might be helpful for convenience to also return the message hashes/IDs that were not present in the mcache.

Are there any security proofs for multihash or similar constructs? I'd think the results would say roughly that, for some desirable hash function properties, multihash is not-too-unlike a hash function satisfying the weakest version of that property among satisfied by any of the hash functions with which it can be instantiated.

As stated this is false of course, there are countless protocols for which substituting multihash as if it were as hash function will break the protocol, like anything that requires uniqueness, but probably far more. It's worth writing down a general setting in which multihash is not inherently broken though, maybe including signature scheme requirements.

This issue is aimed at getting discussion started around designing a messaging layer for libp2p. The messaging layer is for encapsulating unreliable transports, like UDP, along with other use cases.

Specifically, I want to use IPFS in order to provide p2p networking for VPN traffic, which is usually implemented over UDP. It is possible to use other transports, although wasteful since TCP is usually also implemented within the networking stack the VPN encapsulates.

We discussed this a bit in ipfs/kubo#3994

From @magik6k:

There are some problems with supporting udp properly:

• Need to track connections if we want 2-way communication (this isn't that hard, maybe slightly harder if one wants this to be any fast)
• Libp2p** operates on streams, udp is message oriented. There are 2 ways to solve this:

• wrap udp packets into messages (basically size+data piped into stream)
  • IMO the wrong way to do this
• give libp2p the concept of messages
  • There are some plans for message oriented (+unreliable) transports/muxers
    • Can't find any specific issue
  • This would be ideal for udp
  • Also, useful in other parts of libp2p.

cc: @whyrusleeping @magik6k @lgierth

HI reading the docs came across this broken link.

Need to be translated into Chinese? I can come to translate.

This is my translation of ipfs: [ipfs](https: //gguoss.github.io/2017/05/28/ipfs/)

The current protocol specifies two families of 2xx and 3xx error codes for address error:

    HOP_SRC_ADDR_TOO_LONG      = 220;
    HOP_DST_ADDR_TOO_LONG      = 221;
    HOP_SRC_MULTIADDR_INVALID  = 250;
    HOP_DST_MULTIADDR_INVALID  = 251;

    STOP_SRC_ADDR_TOO_LONG     = 320;
    STOP_DST_ADDR_TOO_LONG     = 321;
    STOP_SRC_MULTIADDR_INVALID = 350;
    STOP_DST_MULTIADDR_INVALID = 351;

Why is the distinction between HOP and STOP address errors important and even desirable?
It is arguably the same error, and shouldn't matter which hop it originated from.

In theory, when a subscriber receives a message, the system should ensure the following integrity:

• the message is indeed from the sender ("from") (authentication)
• the message (data + from) hasn't been tampered (immutability)

Reading the spec, I am not sure how it expects to deliver such promises in practice, where not all users have their own node, and not all nodes are honest... :

authentication

there is an AuthMode 'KEY' that looked promising, but the publish API didn't allow us to specify a key, which means the node is probably using its default peerID, how will this work in a multi-user node, such as a gateway? What stops a faulty node from using other's peerID in the "from" field?

immutability

with or without relay, how do the subscribers know the messages they received have not been tampered, either the "from" or the "data"?

Maybe the spec left out some detail? one would think if the publisher requires to sign each message + the node to validated upon both pub() and sub(), it should be able to address both problems... I could be wrong, doesn't IPNS records deploy a similar scheme? Maybe libp2p expect such integrity to be enforced in higher (app) stack? Although I argue these issues are pretty fundamental/common thus valuable to be provided by libp2p...

On GRAFT(topic) it adds the peer to mesh[topic] if it is subscribed to the topic. If it is not subscribed, it responds with a PRUNE(topic) control message.

Similarly to the code in rust-libp2p with subscribe and unsubscribe (see below), it seems to make more sense to simply return false for grafting if it is not subscribed, and return true on success.

https://github.com/libp2p/rust-libp2p/blob/83320e0347e1feff80ebadcdf25ad9376e176164/protocols/floodsub/src/layer.rs#L75-L78:

    /// Subscribes to a topic.
    ///
    /// Returns true if the subscription worked. Returns false if we were already subscribed.
pub fn subscribe(&mut self, topic: Topic) -> bool {

https://github.com/libp2p/rust-libp2p/blob/83320e0347e1feff80ebadcdf25ad9376e176164/protocols/floodsub/src/layer.rs#L100-L105:

    /// Unsubscribes from a topic.
    ///
    /// Note that this only requires a `TopicHash` and not a full `Topic`.
    ///
    /// Returns true if we were subscribed to this topic.
pub fn unsubscribe(&mut self, topic: impl AsRef<TopicHash>) -> bool {

Context: implementing gossipsub for Rust at libp2p/rust-libp2p#767.

Cc @vyzo.

Currently in go-libp2p, when we encounter a p2p circuit multiaddr we treat the entire multiaddr as part of the transport (including the /ipfs/QmId part).

Basically, we considered /p2p-circuit/ipfs/QmId to be short for /p2p-circuit/ipfs/QmId/ipfs/QmId (where the first is the target for the circuit relay and the second is the peer) which would resolve to /p2p-circuit/ip4/.../tcp/.../ipfs/QmId. However, according to the p2p-circuit multiaddr definition, this is incorrect; the /p2p-circuit/ multiaddr takes no arguments.

We did this because we (used to) only pass the "transport" part to the transport. We didn't pass the peer ID.

However, now that we've refactored go transports, we do pass in the peer ID.

After reconsidering, I realized that treating /p2p-circuit/ as the full transport is probably more correct. For example, when we encounter /ipfs/QmId by itself in the swarm, we don't say that the "transport" part is, /ipfs/QmId, we say that no transport-level address has been specified and we look it up. In this case, the /p2p-circuit/ transport has been (completely) specified but the transport of the connection after the the circuit part has not (it's not /ipfs/QmId).

I bring this up because:

1. It's a wart that I've never fully been able to get over.
2. We implement this as a special-case hack in go-ipfs-addr.
3. We don't have this hack in go-libp2p-peerstore.InfoFromP2pAddr. At the very least, we need to be consistent.

Proposed solution: Stop doing this. That is, treat /p2p-circuit/ as a full transport address (like /ip4/xxx/tcp/yyy). This will simplify a bunch of code and remove a nasty hack.

Downside: We have to be careful not to break published PeerInfo records. That is, if an old go-ipfs node (not sure about js) sees:

PeerInfo {
  ID: "QmId",
  Addrs: ["/p2p-circuit/"],
}

It won't be able to use this p2p circuit address.

Note: removing the hack in go-ipfs-addr won't introduce this problem as long as the go-libp2p-circuit transport continues to return the full /p2p-circuit/ipfs/QmId addresses.

@dryajov how does js-libp2p handle this? If I give js-libp2p an address of the form /p2p-circuit/ipfs/QmId, what kind of PeerInfo struct would I get back out?

Just wanted to verify a small discrepancy I noticed in the spec compared to the go-implementation.
The spec states that gossip is emitted to peers that not part of the mesh like so:

for each topic in mesh+fanout:
  let mids be mcache.window[topic]
  if mids is not empty:
    select D peers from peers.gossipsub[topic]
    for each peer not in mesh[topic]
      emit IHAVE(mids)

shift the mcache

In the go implementation (see these two lines):
https://github.com/libp2p/go-libp2p-pubsub/blob/d4589956d289b383aa66807840b5baa3f6916f9f/gossipsub.go#L428
https://github.com/libp2p/go-libp2p-pubsub/blob/d4589956d289b383aa66807840b5baa3f6916f9f/gossipsub.go#L464
Peers are chosen such that they are not in the mesh (the first line), then peers are chosen such that they are not in the fanout (second line).

Should the spec read?:

for each topic in mesh+fanout:
let mids be mcache.window[topic]
if mids is not empty:
select D peers from peers.gossipsub[topic]
for each peer not in mesh[topic] or fanout[topic]
emit IHAVE(mids)

shift the mcache

@lgierth @diasdavid @dignifiedquire

The current spec describes the multistream interaction as <src><dst><data>, further defining it as:

After getting a stream to the relay node from its libp2p swarm, the dialing transport writes the header to the stream. The relaying node reads the header, gets a stream to the destination node, then writes the header to the destination stream and shortcircuits the two streams.

In order to make sure that this is unambiguous an example might be required, such as:

• Peer A connects to relay (Relay) and writes the destination address as a header
  • The header is in the form of /ipfs/relay/circuit/1.0.0\n/<valid multiaddr of dest node>
• The Relay reads the header and the address of Peer B from the stream
  • Relay then connects to Peer B and writes a header in the form of /ipfs/relay/circuit/1.0.0\n/<valid multiaddr of source node>
  • Peer B stores the origin/source address to be able to later dial back (is this the case?)
• Once connected to both peers, the Relay short circuits the two streams

This is what I currently have implemented as well, but I still feel that I might be reading the spec incorrectly, so just making sure this is the intended flow. Adding an example like this to the spec might also help future implementors. Please feel free to correct were things are off.

We have two versions of pubsub router now, FloodSubRouter and GossipSubRouter respectively. It is clear that FloodSubRouter just floods messages. However, the current GossipSubRouter does not perform the same way as the Gossipsub spec described. It is more like performing normal gossiping on each topic. Its algorithm is quite the same as the protocol in gerbil-simsub. For me, it is quite confusing that the implementation is a lot different from the spec.

IMO the name "gossipsub" is suitable for the current implementation of GossipSubRouter in the repo go-floodsub, i.e. the gerbil-simsub's protocol. And the one described in the Gossipsub spec should be the spec for "Proximity Aware Epidemic PubSub". So it will be clearer if we have specs for "gossipsub" and "Proximity Aware Epidemic PubSub" respectively.

Is there a plan already that the current gossipsub implementation will be used, or it will be changed to "Proximity Aware Epidemic PubSub" soon? It is actually discussed in libp2p/libp2p#33 (comment) and I really appreciate the answer from @whyrusleeping. Just want to confirm again, since "Proximity Aware Epidemic PubSub" seems to be more experimental, for its broadcast tree, random walks for joining, and passive view exchange. In contrast, the current "gossipsub" in go-floodsub seems more safe.

Thank you for reading!

As discussed with @dryajov, we need an error code to distinguish internal stop errors, ike timeouts when accepting the new stream.
I propose a general error code for these cases, STOP_RELAY_REFUSED.

I am trying to get a deeper understanding of gossipsub. Currently, the specification for gossipsub gives little details about what the network topology is.

This is what I have in mind about the network topology of gossipsub:
- In the meshsub step, the local topology is a strongly-connected tournament, where the in-degree = out-degree <= D
- To stabilize the mesh, peers get removed (pruned) and the mesh turns into a tree/line topology.
So, the network topology of gossipsub is a line/tree topology.

Is this correct? If not, I would like to get an understanding of the topology structure of gossipsub.

Now that we have a proper protobuf spec for the circuit-relay messages, do we care about individual addresses length? It seems like we should restrict the overall size of the protobuf message, but other than that, checking for address length doesn't seem to add any value anymore. @vyzo proposed 4096 as an arbitrary message size, which I think is reasonable.

Given this, lets remove the address length restriction in favor of a message size.

@diasdavid @whyrusleeping @lgierth

Right now the identify protocol is /ipfs/id/1.0.0, the kademlia protocol is /ipfs/kad/1.0.0, and the ping protocol is /ipfs/ping/1.0.0.
If libp2p is to become independant from IPFS, it would be nice to change these identifiers to something like /p2p/id/1.0.0, /p2p/kad/1.0.0 and /p2p/ping/1.0.0 instead.

Maybe these are other protocols which I can't think of that have the same issue.

.proto files should be included as part of the specs, since all implementations need to use the same file for compatibility.

Today, I had a chat with some of the libp2p core engineers (@vasco-santos, @raulk and @jacobheun ) and @mgoelzer. Mike mentioned that I should create an issue with clarifications that should be added to the gossipsub specification.

Here are some clarifications that I think the gossipsub specification needs:

• Gossipsub creates an overlay network over an underlying network. Thus, it doesn't affect the underlying network's topology. The network topology of gossipsub is a strongly connected mesh (see #122 ). This is implied in the spec but I think making it more explicit might help.

• Adding a glossary for terms that aren't used in the p2p networking literature such as ambient peer discovery.

• In meshsub, when a peer selects a subset of peers to create the mesh with, which peers are under consideration? Is it the peers that the peer finds from ambient peer discovery or is the peers that have subscribed to that peer's topic(s)?

• In meshsub, clarifying how often the stabilization algorithm is run could be helpful to implementers that just want to use meshsub. This could also be an implementation detail that could instead be added to an implementers guide.

• Adding guidelines for the choice of parameters such as D, D_high and D_low. These are obviously implementation independent. It might be a good idea to add this to an implementers' guide

• The control message piggybacking section is unclear and says that one should consult the Go implementation. Is this portion of gossipsub implementation specific or can it be described without being tied to a specific implementation?

• The gossip is emitted to random subsets of peers not in the mesh, similar to randomsub, and it allows us to propagate metadata about message flow throughout the network.

The above statement is unclear. I remember when the spec was just starting, that there was a distinction between eager push and lazy push. Lazy push is when a peer sends metadata about a message to other peers and eager push is when a peer sends all of the data to other peers. Is the above statement alluding to that?

• Add more specifications to the message cache data structure. As it stands, one has to go over the Go implementation of a message cache to understand what it does.

• Clarify the different between Join and Subscribe. Similarly, for Leave and Unsubscribe.

• I think the description of the Join operation can be better written.

• Add guidelines for when to run the heartbeat procedure for stabilizing the mesh, emitting gossip and maintaining fanout.

Test suites should ideally use JSON for interoperability across virtually all languages.

Currently it appears that the test suites in https://github.com/libp2p/specs/blob/master/6-interfaces.md are in JS and Go.

There doesn't seem to be a suite/interface for NAT traversal and there isn't either for gossibsub, floodsub or multicast. Not sure off the top of my head what else is missing.

Additionally where using the same .proto protobuf file, this should make it easier for interoperability.

In the current GossipSub specification, there are details lacking for the Control Message Piggybacking section. In fact, it is advised that we look at the reference Go implementation for more details. The way that section is currently written implies, to me at least, that implementing control message piggybacking is optional to making GossipSub work. However, in the Go implementation, it seems has been implemented in such a way that it relies on control message piggybacking.

How important is it that implementers implement control message piggybacking in their GossipSub implementations? If it is indeed important to implement, it needs to be properly specified, so that we don't need to rely as much on the Go implementation. If it is not important, then we, as implementers, need to know if we should implement in our GossipSub implementation.

In the beginning, peer IDs were base58 encoded sha256 multihashes. However, being able to embed a key into a peer ID was rather tempting so we introduced a IDFromEd25519PublicKey function. This function created a different peer ID that embedded the public key.

Unfortunately,

1. Users had to remember to call this.
2. If a user called the normal IDFromPublicKey function, they'd get a different peer ID for the same key. This obviously isn't good as our security transports derive peer IDs from public keys instead of validating peer IDs (this makes it impossible to mis-validate).

Therefore, I submitted libp2p/go-libp2p-peer#30 which:

1. Removed IDFromEd25519PublicKey.
2. Changed inline peer IDs to just inline the protobuf (instead of introducing an entirely new format).
3. Auto-inlined peer ids <= 42 bytes.

Unfortunately, this last bit appears to be a problem... At the time, I was under the impression that nobody had actually gotten around to using ed25519 keys as inlining didn't really work (libp2p would still use the sha256 peer IDs). However, it turns out that OpenBazaar was using them anyways.

Second, in retrospect, this wasn't even the optimal solution. Really, we also need to support arbitrary hash functions but this only adds support for the identity hash function.

So, my proposal here is to:

1. Revert that change. I'm pretty sure OpenBazaar is the only ed25519 user in the wild so that should be safe.
2. Introduce a new field in the public/private key protobuf that explicitly specifies the hash function (defaulting to "sha256" if missing).

Open question: Should we use this as an opportunity to switch to multibase-enabled peer IDs now? One significant issue with the current embedding method is that it makes it slightly harder to switch to multibase peer IDs (cc @lidel). I've previously proposed switching to CID-based peer IDs but that may be more work than it's worth (a motivation would be transfering keys over bitswap, easier to support large quantum-safe keys, format flexibility, etc.).

One should be able to read the spec and implement it in a way that will be interoperable with other implementations, without having to read other implementations. It seems that that is not the case. For instance, the spec doesn't specify what the constants should be, although I previously gleaned them from the Go impl:

// Overlay parameters
pub const TARGET_MESH_DEGREE: u32 = 6;
pub const LOW_WM_MESH_DEGREE: u32 = 4;      // low water mark for mesh degree
pub const HIGH_WM_MESH_DEGREE: u32 = 12;    // high water mark for mesh degree
pub const GOSSIP_HIST_LEN: u32 = 6;         // length of gossip history
pub const MSG_HIST_LEN: u32 = 120;          // length of total message history
pub const SEEN_MSGS_CACHE: u32 = 120;

Similarly I have also found details on mcache to be lacking, such as what the current history window should be like.

mcache: a message cache that contains the messages for the last few heartbeat ticks.

The message cache is a data structure that stores windows of message IDs and the corresponding messages. It supports the following operations:

mcache.put(m): adds a message to the current window and the cache.
mcache.get(id): retrieves a message from the cache by its ID, if it is still present.
mcache.window(): retrieves the message IDs for messages in the current history window.
mcache.shift(): shifts the current window, discarding messages older than the history length of the cache.

The seen cache is the flow control mechanism. It tracks the message IDs of seen messages for the last two minutes. It is separate from mcache for implementation reasons in Go (the seen cache is inherited from the pubsub framework), but they could be the same data structure. Note that the two minute cache interval is non-normative; a router could use a different value, chosen to approximate the propagation delay in the overlay with some healthy margin.

Is the current window the same thing as the seen cache?

How long is a heartbeat?

One problem with messages being unsized is that an attacker could spam the network with very large data fields in messages, which would be prohibitively expensive to transmit. While enforcing an arbitrary size limit may be unreasonable for a library, perhaps a workaround is to optionally make the data field limited to some value, which can be chosen by users of the library. Otherwise, leave it to users to manage this problem, such as by having a gas limit in Ethereum, and otherwise pricing storage, computation, bandwidth and I/O.

cc @whyrusleeping, @vyzo