Code Monkey home page Code Monkey logo

dht's Introduction

dht

Go Reference

Installation

Get the library package with go get github.com/anacrolix/dht/v2, or the provided cmds with go install github.com/anacrolix/dht/v2/cmd/...@latest.

Commands

Here I'll describe what some of the provided commands in ./cmd do.

dht

Supports various commands operating on the DHT.

% go run github.com/anacrolix/dht/v2/cmd/dht --help
valid arguments at this point:
  --help|-h
  --network <string>
  --secure
  --bootstrap-addr <[]string>
  --query-resend-delay <time.Duration>
  derive-put-target
  put
  put-mutable-infohash
  get
  ping
  get-peers
  query
  ping-nodes

Downstream projects

Projects that uses this repo in novel ways.

  • cove: Torrent browser with streaming, DHT search, video transcoding and casting.
  • btlink: btlink supports DNS records stored on the DHT.

dht's People

Contributors

afjoseph avatar ajnavarro avatar anacrolix avatar askalexsharov avatar boramalper avatar dborzov avatar decentralgabe avatar dz0ny avatar eraclitux avatar etix avatar felixonmars avatar gdzx avatar ilyabrin avatar milosgajdos avatar muesli avatar shoenig avatar stapelberg avatar ucwong avatar yarikk avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

dht's Issues

A lot of connections

With bootstrap and when connecting to the torrent, there are about 1000 connections.
Connections are not replaced, but added, if after bootstrap connect to the torrent, there will be about ~2000 connections, etc.
My ISP blocked me access because it thinks it's a virus attack

Over time, connections are disconnected

Broken Bootstrap Address: router.silotis.us

The address router.silotis.us is broken and throwing errors since its dns is not working

The of this server have mentioned that it is temporary previously here

Please confirm the issue from your side so that i could create a PR to fix.

Multiple nodes with the same addr can exist in the table?

panic: missing addr for node in table

goroutine 12 [running]:
github.com/anacrolix/dht.(*table).dropNode(0xc420089c40, 0xc420b83580)
        /root/gopath/src/github.com/anacrolix/dht/table.go:18 +0x189
github.com/anacrolix/dht.(*Server).getNode.func1(0xc420b83580, 0xc421f183c0)
        /root/gopath/src/github.com/anacrolix/dht/server.go:401 +0x9e
github.com/anacrolix/dht.(*bucket).EachNode(0xc420089d10, 0xc423267da8, 0x9a6e89e182121c52)
        /root/gopath/src/github.com/anacrolix/dht/bucket.go:13 +0x9b
github.com/anacrolix/dht.(*Server).getNode(0xc420089c00, 0xa65200, 0xc4243e86c0, 0xfd6dc2058a36c336, 0x9a6e89e182121c52, 0x11542caa4, 0x0, 0x7f0fc3e799e0, 0x0)
        /root/gopath/src/github.com/anacrolix/dht/server.go:399 +0x19e
github.com/anacrolix/dht.(*Server).processPacket(0xc420089c00, 0xc422058000, 0x68, 0x10000, 0xa65200, 0xc4243e86c0)
        /root/gopath/src/github.com/anacrolix/dht/server.go:217 +0x406
github.com/anacrolix/dht.(*Server).serve(0xc420089c00, 0x0, 0x0)
        /root/gopath/src/github.com/anacrolix/dht/server.go:262 +0x233
github.com/anacrolix/dht.NewServer.func1(0xc4200820e0)
        /root/gopath/src/github.com/anacrolix/dht/server.go:150 +0x32
created by github.com/anacrolix/dht.NewServer
        /root/gopath/src/github.com/anacrolix/dht/server.go:149 +0x2cd

error on `go get -u`

So,I tried to run go get -u on project which depends on this dht package .

I was greeted with this error.

go get: github.com/willf/[email protected] updating to github.com/willf/[email protected]: parsing go.mod: module declares its path as: github.com/bits-and-blooms/bitset but was required as: github.com/willf/bitset

Please fix this asap or atleast tell me what should I do

bep44: mutable items

Hi,

i was trying to figure out how much of this bep was available in your package.
http://bittorrent.org/beps/bep_0044.html

I found the immutable items part was there, if i dont mistake.

About mutable items, there are few more operation to do when a message is sent / receive.
I was wondering how that would operate with your package.
inside or outside of it ?
if there was some specific ways to implement those extra functions with the lib ?

I was looking for a Put method at Server.go, but i could not find it.
Is it missing as i believe i understand ?

thanks for help!

Question: Is it possible to specify the size of the k-buckets?

The title really states it all.
My current use case: I want to run a DHT node that remembers as many good nodes as possible.
Allowing the k-buckets to be (infinitely) large results in good nodes not being forgotten once they are stored in the routing table.

Server::findNode not exposed?

Hi.
First of all, thank you for the great work you have done on this project.

I just discovered this repo a few days ago and would like to continue using it for a project.
For that I would need to send find_node queries to target addresses.
Is it possible that the findNode function in server.go is not exposed unlike Ping and Announce?
If so, how can one use the function after having bootstrapped?
I just started using go a few days ago, therefore, this may be an oversight on my part.

Thank you,
Charmaine

Does DHT work standalone?

Hey, I've been trying to get this library to work standalone without bootstrapping from the global servers but I seem to not be getting the expected behaviour.

Here's a gist which sets up a server standalone https://gist.github.com/asim/99d9978f76ec46083b6dcf4a65b58cb5 and then the equivalent can be run with ports changed and using the first address in BootstrapNodes list.

When I run this locally or across the public internet I can see udp packets flowing back and forth between the servers but nothing is ever announced and I can't actually tell if anything's working.

If I use the global servers then it all seems to work fine. Any ideas why this won't bootstrap as a standalone dht server?

Time out log print

Is it necessary to print this time out log out?

vendor/github.com/anacrolix/dht/v2.(*Server).pingQuestionableNodesInBucket.func1.1:1376]: error pinging questionable node in bucket 18: after 3 tries: transaction timed out

does it generate a lot of trafic ?

Hi,

Using this server init, and two working nodes on 9090 / 9091 ports.

func getDhtTable(since int, bootstrap []string, refresh time.Duration) (ret *dht.Server, port int, err error) {
	err = errors.New("No port found.")
	for i := since; i < since+30; i++ {
		conf := &dht.ServerConfig{
			Addr:               fmt.Sprintf("localhost:%v", i),
			NoDefaultBootstrap: !true,
			NoSecurity:         true,
			BootstrapNodes:     bootstrap,
			StoreTimeout:       refresh,
			OnQuery: func(query *krpc.Msg, source net.Addr) (propagate bool) {
				log.Printf("query %v %#v\n", source, query.Q)
				<-time.After(1 * time.Second)
				return true // true or false ? unclear doc.
			},
			OnResponsePeer: func(query krpc.Msg, source dht.Addr) {
				r := ""
				if query.R != nil {
					r = fmt.Sprintf("ret.V=%#v ret.Seq=%#v ret.K=%#v ret.Sign=%#v",
						query.R.V, query.R.Seq, query.R.K, query.R.Sign)
				}
				log.Printf("response %v err=%v %v\n", source, query.Error(), r)
			},
		}

		if srv, netErr := dht.NewServer(conf); netErr == nil {
			ret = srv
			port = i
			err = nil
			break
		}
	}
	return
}

In the OnQuery, i had to add a 1s timeout rate limiter, otherwise the table would generate a loooottttt of traffic.
A quick count with grep/wc yields 2860 5500 query per second here (so if i have two nodes, its like 5k 10k :o).

Is there anything to take care of here, as i suspected, or it s expected to be that fast under best conditions ?

The under best conditions matters, as the phenomena won t happen if any of the node is offline (it creates a sort of 10 sec blocking timeout in the stream).

thanks!

error in Marshaling krpc.id

func main() {
	var id krpc.ID
        copy(id[:],[]byte("012345678901234567890"))
        fmt.Printf("%s",bencode.MustMarshal(id))
}

I receive:
li48ei49ei50ei51ei52ei53ei54ei55ei56ei57ei48ei49ei50ei51ei52ei53ei54ei55ei56ei57ee

Shouldn't be:
20:012345678901234567890

Michele

Trouble cloning the repository

A simple git clone is taking ages:

$ git clone [email protected]:anacrolix/dht.git
Cloning into 'dht'...
remote: Counting objects: 218641, done.
remote: Compressing objects: 100% (6/6), done.
Receiving objects:  13% (30228/218641), 20.12 MiB | 39.00 KiB/s

Same problem when cloning a fork of this repository. I've tried both HTTPS and SSH. Downloading the ZIP works fine and it's tiny.

Do you have any idea what's going on? Any other repo is working fine.

License?

Hey! Is this also licensed under MPLv2.0? I couldn't find any explicit mention of the license and I'd like to make sure. =)

panic: sync: negative WaitGroup counter

Running r55080c5 I'm regularly getting the following panic. Any ideas, or should I work on a reproduction?

panic: sync: negative WaitGroup counter
goroutine 17787 [running]:
sync.(*WaitGroup).Add(0x13984c10, 0xffffffff) /usr/local/go/src/sync/waitgroup.go:73 +0x150
sync.(*WaitGroup).Done(0x13984c10) /usr/local/go/src/sync/waitgroup.go:98 +0x24
localhost/xxx/vendor/github.com/anacrolix/dht.(*Server).Bootstrap.func1.1(0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
localhost/xxx/vendor/github.com/anacrolix/dht/server.go:662 +0x13c
localhost/xxx/vendor/github.com/anacrolix/dht.(*Server).findNode.func1(0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
localhost/xxx/vendor/github.com/anacrolix/dht/server.go:630 +0x80
created by localhost/xxx/vendor/github.com/anacrolix/dht.(*Server).query.func4 localhost/xxx/vendor/github.com/anacrolix/dht/server.go:540 +0x88
`

New release

Since adding a new parameter in the s.config.ConnectionTracking.Wait breaks the backwards compatibility, I can no longer use anacrolix/torrent through with go mod.

I'm getting the following issue when I run go get -u

# github.com/anacrolix/dht
../../../../pkg/mod/github.com/anacrolix/[email protected]/server.go:623:44: not enough arguments in call to s.config.ConnectionTracking.Wait
	have (conntrack.Entry, string, number)
	want (context.Context, conntrack.Entry, string, conntrack.priority)

I think this can be fixed by simply creating a new Github release. Thanks.

invalid memory address or nil pointer dereference

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x8d8778]

goroutine 49 [running]:
github.com/anacrolix/dht/v2.(*Server).handleQuery(0xc00025e000, {0x1741be0, 0xc00c3e13c0}, {{0xc0155e7337, 0x3}, 0xc00f23dc20, {0xc0155e733c, 0x4}, {0xc0155e7342, 0x1}, ...})
        /root/go/pkg/mod/github.com/anacrolix/dht/[email protected]/server.go:582 +0x7f8
github.com/anacrolix/dht/v2.(*Server).processPacket(0xc00025e000, {0xc0007de000, 0x40d, 0x10000}, {0x1741be0?, 0xc00c3e13c0})
        /root/go/pkg/mod/github.com/anacrolix/dht/[email protected]/server.go:318 +0x350
github.com/anacrolix/dht/v2.(*Server).serve(0xc00025e000)
        /root/go/pkg/mod/github.com/anacrolix/dht/[email protected]/server.go:375 +0x15b
github.com/anacrolix/dht/v2.(*Server).serveUntilClosed(0xc00025e000)
        /root/go/pkg/mod/github.com/anacrolix/dht/[email protected]/server.go:247 +0x33
created by github.com/anacrolix/dht/v2.NewServer
        /root/go/pkg/mod/github.com/anacrolix/dht/[email protected]/server.go:242 +0x79c

image

Documentation about peer/traversal stats

Hello! Thank you for maintaining this project and publishing it as FLOSS. I've been experimenting with using it as part of the ooni project to measure mainline DHT censorship.

However, it is not clear to me precisely what the exposed stats actually mean. For example, running the provided CLI tool returns the following:


$ ./dht get-peers
{
  "Peers": [],
  "DistinctPeerIps": 0,
  "TraversalStats": {
    "NumAddrsTried": 31,
    "NumResponses": 0
  },
  "ServerStats": {
    "GoodNodes": 2,
    "Nodes": 3,
    "OutstandingTransactions": 0,
    "SuccessfulOutboundAnnouncePeerQueries": 0,
    "BadNodes": 0,
    "OutboundQueriesAttempted": 31
  }
}

In this example, we can see NumResponses is zero, but in server stats GoodNodes is 2. This does not match my current mental model.

In a test case i wrote, where i start a local dht.Server without any bootstrap nodes, then try to connect from it from another dht.Server (with only the first one as bootstrap nodes), i get the following results after an Announce: in announce.TraversalStats(): 1 peer tried, 0 responded. And announce.Peers is empty. Yet i can see using tcpdump that the first peer did respond to the query.

Trying with the default set of bootstrap nodes (13 addresses), i get 19 NumResponses (NumAddrsTried 46) and announce.Peers contains 22 entries.

It's not clear to me what NumResponses and the entries in announce.Peers are. Is the first one the number of traversed peers queried about the specific infohash, and the second the number of found peers who announce that infohash? If so, is NumResponses only counting replies from peers announcing that infohash?

To get more comprehensive results, maybe i should call get_peers recursively myself? In the meantime, i feel like it's worth explaining more about what the exposed stats/data actually means.

Thanks for your attention

invalid memory address or nil pointer dereference

^[[0mpanic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x20 pc=0x10de6c7]
/vendor/github.com/anacrolix/dht/v2/bep44/store.go:54 +0x27

get_peers is performed but no announce

if I add a log.Printf to annouce.go/beginQuery I get:
20-03-21 21:50:33 announce.go:290: doing dht announce get_peers to [address]

and not a single doing "dht announce announce_peer"

do not start a goroutine on import

I'm trying to use this as part of my project, and the BT/DHT function is not always needed, it's only some part of my application.

So the dht/v2 as a library start a goroutine in the background running forever is very weird.

I think this goroutine should be started when the BT client is created, not the library is imported, library itself shoule bt side-effect free.

use dht for own storage

From @vtolstov on December 5, 2016 13:41

I need a dht package for own storage system, and as i see godoc.org says that this package have more stars then other. Does it possible to use this package for own program that does not using torrents?

Copied from original issue: anacrolix/torrent#134

Ability to multiplex connection

Pardon my naivete of the bittorrent protocol, but is it possible currently to multiplex the connection? I see I can probably provide my own socket and pre-obtain a byte array and pass it along only if it's bencoded. Is this the recommended approach or might it be preferred to have a config option to handle fallthrough packets?

using anacrolix/dht for holepunching?

I am trying to build a p2p network and I am just asking can this dht github.com/anacrolix/dht be used for holepunshing and if so how to provide the tcp listening internal port?

Example Ping timeout

Here is my try:
go run ./cmd/dht ping router.bittorrent.com:6881 router.utorrent.com:6881

2022/01/06 14:16:01 main.go:63: public ip: "103.201.26.3"
2022/01/06 14:16:01 main.go:70: dht server on [::]:61082 with id 5d12d5586d35ce2eaf751ef3af0812935955529b
router.utorrent.com:6881: 2.003219305s: after 1 tries: transaction timed out
router.bittorrent.com:6881: 2.004312261s: after 1 tries: transaction timed out

And here is my capture from tcpdump
image

Actually I also met this question when I using python.
Does my operation right ?
Could you please give me some advice, thank you very much !

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.