tantalor93 / dnspyre Goto Github PK
View Code? Open in Web Editor NEWCLI tool for a high QPS DNS benchmark
Home Page: https://tantalor93.github.io/dnspyre/
License: MIT License
CLI tool for a high QPS DNS benchmark
Home Page: https://tantalor93.github.io/dnspyre/
License: MIT License
Describe the feature
I'd like to have dnspyre return a nonzero exit code if any requests failed. Different people might have different definitions of failure at different times (e.g. connect timeout, nxdomain, duplicate answers, wrong answers, bad certificate, etc), so the parameter might take an argument: --fail="timeout,nxdomain"
Why do you need this feature
I have a program that does binary searches. It'll run a specified program and fill in a specified parameter with a number, and based on the return code of the program, will search until it finds the number where the program just starts to fail:
bsearch -l low -h high -- command args args {} args args
This will find the largest number that can be used in place of the {},
in the range of low to high.
Options:
-l low : the low number of the range to test
-h high : the high number of the range to test
Without this feature, I have to write a wrapper script that runs dnspyre and then greps for a success or failure string.
For an example, see the --fail
argument for curl
.
Thanks!
PS: This looks like the best DNS benchmarking program out there! DNSperf / DNSRes are hard to use, Flamethrower is hard in a different way, and DNS shotgun is... weird? Dnspyre looks polished, and it has all the protocols I'm looking for.
Cannot check limit of number of files. Skipping check. Please make sure it is sufficient manually. RlimitStack is not implemented on Windows
What does this message mean, and could you provide a way to suppress this warning?
Describe the feature
I'm glad to see that dnspyre has an --ednsopt
option that lets you add arbitrary EDNS options, like Client Subnet (08).
Could you also add an option like --subnet=x.x.x.x/yy
or --ecs=x.x.x.x/yy
to make it a little easier?
Also, is there a way to specify a list of subnets to send on different queries?
Aaaand can it do IPv6 too?
Why do you need this feature
We're performance testing a version of PowerDNS authoritative server that's ECS-aware, and typing --ednsopt="08:00011600cc0fdc"
instead of --subnet=204.15.220.0/22
is a bit tedious. :-)
Describe the feature
If you want to do a tcp-based test, you can only do at most 64k simultaneous connections before you start reusing port numbers, and since a connection is defined by source+destination ip+port, there would be no way to tell the connections apart.
Many programs allow the user to specify a source ip address chosen from the ips of configured interfaces. Adding just that feature would let you start multiple copies of dnspyre
, one per source ip, but that's going to split the statistics into per-interface buckets.
What would really be useful would be to specify a list of source ips to use. In fact, the loopback interface allows you to use any ip address in the 127.0.0.0/8 range, so in that special case it would be reasonable to specify a subnet, not just a list of ips.
Why do you need this feature
64k ports is not enough for really big tests.
Describe the feature
As part of benchmark I want to be able to specify that I want to the server to do the DNSSEC validation (set DO=1)
Why do you need this feature
i want to be able to observe performance of the tested server, when it is doing DNSSEC validation
What happened
Ran pdns_server
on port 91 with *.doh-test.com TXT "wildcard.doh-test.com!"
in a doh-test.com zone.
Ran tcpdump -w /tmp/a.pcap port 91
Ran dnspyre --no-distribution --no-progress --json -t TXT --server 172.16.200.25:91 --tcp -d 10s -c 1 -t TXT @infile.txt
Infile.txt contains testit-1.doh-test.com
to testit-10000000.doh-test.com
(10 million records).
Json output:
{
"totalRequests": 46420,
"totalSuccessResponses": 42242,
"totalNegativeResponses": 0,
"totalErrorResponses": 0,
"totalIOErrors": 4178,
"TotalIDmismatch": 0,
"totalTruncatedResponses": 0,
"responseRcodes": {
"NOERROR": 42242
},
"questionTypes": {
"TXT": 46420
},
"queriesPerSecond": 3892.1,
"benchmarkDurationSeconds": 11.93,
"latencyStats": {
"minMs": 0,
"meanMs": 0,
"stdMs": 0,
"maxMs": 2,
"p99Ms": 0,
"p95Ms": 0,
"p90Ms": 0,
"p75Ms": 0,
"p50Ms": 0
}
}
Analyzed the capture with wireshark:
Conclusion 1: Dnspyre lost 4178 requests before they even got to the outgoing TCP stack.
testit-21121.test-doh.com
.Conclusion 2: Dnspyre is sending every hostname twice.
What you expected to happen
Expected 0 errors, as lost packets would have been retransmitted in the middle of a single TCP conversation.
Expected each hostname to be sent once.
dnspyre command
dnspyre -t TXT --server 172.16.200.25:91 --tcp -d 10s -c 1 -t TXT @infile.txt
Describe the feature
When running a TCP-based test (TCP/TLS/DoH/etc), it's hard to tell if connections are being reused or not. Could you add a flag to enable/disable connection reuse (aka keepalive)?
Why do you need this feature
I'm trying to get our DoH server to handle 200,000 QPS with <100ms response time for separate users. If the dnspyre clients are using keepalive, then the results may differ from what we'll see in production.
Add support for querying DNS servers over DoH
Having used dnspyre for a while it's been great, but I kept ended up either grepping the stdout output or using --csv flag going to a third or fourth pipe to pass it downstream to do something with it. However the csv flag doesn't give me the rest of the output summary, like successful vs failed request counts and the p99 etc overviews.
Would it be possible to make a mode where the stdout output is json only, and non json output goes into stderr?
I was considering running dnspyre in a loosely isolated distributive mode on kubernetes but need some way to take the response from a worker pod back to a controlling operator, and json probably makes the most sense - can capture the stdout and curl it back to the operator http server to collect results per worker.
Would it possible to add an insecure flag to ignore self signed certificates?
Load testing on an internal environment with ephemeral DNS clusters would benefit greatly from being able to accept an untrusted certificate.
Thanks
Reproduction:
docker run tantalor93/dnspyre:v2.12.1 --server 8.8.8.8 https://raw.githubusercontent.com/Tantalor93/dnspyre/master/data/1000-domains
There was an error while starting benchmark: failed to download file 'https://raw.githubusercontent.com/Tantalor93/dnspyre/master/data/1000-domains' with error 'Get "https://raw.githubusercontent.com/Tantalor93/dnspyre/master/data/1000-domains": tls: failed to verify certificate: x509: certificate signed by unknown authority'
What happened
After upgrading to dnspyre v2.19.0 or v2.21.0, I encountered an issue specific to NextDNS during DoT testing:
tls: failed to verify certificate: x509: cannot validate certificate for 45.90.30.0 because it doesn't contain any IP SANs 1 (100.00)%
Unlike other DNS services like Quad9 and 1.1.1.1, which include extensive IP SANs in their server certificates, NextDNS uses a wildcard SAN. This seems to be causing the validation failure in dnspyre from v2.19.0 onwards. This issue did not occur in earlier versions, such as v2.18.2 or v2.17.0, so it's triggered only under certain circumstances.
What you expected to happen
I expected dnspyre to validate NextDNS's certificate without issues, as it did in versions prior to v2.19.0, considering that other DNS services with detailed IP SANs in their certificates are not affected.
dnspyre command
v2.18:
$ dnspyre --version
2.18.2-linux-amd64
$ dnspyre --dot --server="test.dns.nextdns.io" google.com
Using 1 hostnames
Benchmarking test.dns.nextdns.io:853 via tls with 1 concurrent requests
Total requests: 1
DNS success codes: 1
DNS response codes:
NOERROR: 1
DNS question types:
A: 1
Time taken for tests: 172.07ms
Questions per second: 5.8
DNS timings, 1 datapoints
min: 167.77ms
mean: 171.97ms
[+/-sd]: 0s
max: 176.16ms
p99: 176.16ms
p95: 176.16ms
p90: 176.16ms
p75: 176.16ms
p50: 176.16ms
v2.19+:
$ dnspyre --version
2.19.0-linux-amd64
$ dnspyre --dot --server="test.dns.nextdns.io" google.com
Using 1 hostnames
Benchmarking 45.90.30.0:853 via tls with 1 concurrent requests
Total requests: 1
Read/Write errors: 1
Time taken for tests: 76.7ms
Questions per second: 13.0
Total Errors: 1
Top errors:
tls: failed to verify certificate: x509: cannot validate certificate for 45.90.30.0 because it doesn't contain any IP SANs 1 (100.00)%
$ dnspyre --dot --server="anycast.dns.nextdns.io" google.com
Using 1 hostnames
Benchmarking 45.90.28.0:853 via tls with 1 concurrent requests
Total requests: 1
Read/Write errors: 1
Time taken for tests: 111.21ms
Questions per second: 9.0
Total Errors: 1
Top errors:
tls: failed to verify certificate: x509: cannot validate certificate for 45.90.28.0 because it doesn't contain any IP SANs 1 (100.00)%
$ dnspyre --dot --server="firefox.dns.nextdns.io" google.com
Using 1 hostnames
Benchmarking 217.146.31.87:853 via tls with 1 concurrent requests
Total requests: 1
Read/Write errors: 1
Time taken for tests: 646.89ms
Questions per second: 1.5
Total Errors: 1
Top errors:
tls: failed to verify certificate: x509: cannot validate certificate for 217.146.31.87 because it doesn't contain any IP SANs 1 (100.00)%
cert digging command
$ openssl s_client -connect dns.quad9.net:853 -showcerts | openssl x509 -noout -text | grep -A1 'Subject Alternative Name:'
depth=2 C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert Global Root CA
verify return:1
depth=1 C = US, O = DigiCert Inc, CN = DigiCert TLS Hybrid ECC SHA384 2020 CA1
verify return:1
depth=0 C = US, ST = California, L = Berkeley, O = Quad9, CN = *.quad9.net
verify return:1
X509v3 Subject Alternative Name:
DNS:*.quad9.net, DNS:quad9.net, IP Address:9.9.9.9, IP Address:9.9.9.10, IP Address:9.9.9.11, IP Address:9.9.9.12, IP Address:9.9.9.13, IP Address:9.9.9.14, IP Address:9.9.9.15, IP Address:149.112.112.9, IP Address:149.112.112.10, IP Address:149.112.112.11, IP Address:149.112.112.12, IP Address:149.112.112.13, IP Address:149.112.112.14, IP Address:149.112.112.15, IP Address:149.112.112.112, IP Address:2620:FE:0:0:0:0:0:9, IP Address:2620:FE:0:0:0:0:0:10, IP Address:2620:FE:0:0:0:0:0:11, IP Address:2620:FE:0:0:0:0:0:12, IP Address:2620:FE:0:0:0:0:0:13, IP Address:2620:FE:0:0:0:0:0:14, IP Address:2620:FE:0:0:0:0:0:15, IP Address:2620:FE:0:0:0:0:0:FE, IP Address:2620:FE:0:0:0:0:FE:9, IP Address:2620:FE:0:0:0:0:FE:10, IP Address:2620:FE:0:0:0:0:FE:11, IP Address:2620:FE:0:0:0:0:FE:12, IP Address:2620:FE:0:0:0:0:FE:13, IP Address:2620:FE:0:0:0:0:FE:14, IP Address:2620:FE:0:0:0:0:FE:15
$ openssl s_client -connect 1.1.1.1:853 -showcerts | openssl x509 -noout -text | grep -A1 'Subject Alternative Name:'
Can't use SSL_get_servername
depth=2 C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert Global Root CA
verify return:1
depth=1 C = US, O = DigiCert Inc, CN = DigiCert TLS Hybrid ECC SHA384 2020 CA1
verify return:1
depth=0 C = US, ST = California, L = San Francisco, O = "Cloudflare, Inc.", CN = cloudflare-dns.com
verify return:1
X509v3 Subject Alternative Name:
DNS:cloudflare-dns.com, DNS:*.cloudflare-dns.com, DNS:one.one.one.one, IP Address:1.0.0.1, IP Address:1.1.1.1, IP Address:162.159.36.1, IP Address:162.159.46.1, IP Address:2606:4700:4700:0:0:0:0:1001, IP Address:2606:4700:4700:0:0:0:0:1111, IP Address:2606:4700:4700:0:0:0:0:64, IP Address:2606:4700:4700:0:0:0:0:6400
$ openssl s_client -connect test.dns.nextdns.io:853 -showcerts | openssl x509 -noout -text | grep -A1 'Subject Alternative Name:'
depth=2 C = US, ST = New Jersey, L = Jersey City, O = The USERTRUST Network, CN = USERTrust ECC Certification Authority
verify return:1
depth=1 C = AT, O = ZeroSSL, CN = ZeroSSL ECC Domain Secure Site CA
verify return:1
depth=0 CN = dns.nextdns.io
verify return:1
X509v3 Subject Alternative Name:
DNS:dns.nextdns.io, DNS:*.dns.nextdns.io
Describe the feature
We'd like to benchmark using TSIG on requests to see if it's faster that DoH or any of the other encrypted protocols.
Why do you need this feature
We've got a client who's worried about replay attacks, and requests that we support 200,000 requests per second with 100ms latency. We've achieved that with plain UDP, now we just have to get some sort of replay protection set up. DoH, DoT, DoQ and DNSSEC all provide that, but we're having problems reaching the necessary speed, and are hoping that TSIG will do the trick.
Since "Hope is Not a Plan", we're going to have to benchmark it.
The equivalent dig command is:
$ pdnsutil generate-tsig-key foobar hmac-sha512
Create new TSIG key foobar hmac-sha512 asdfasdfYpM2nUHdJtC3mDJnbBIXzUGgDU2/dwz8cDAfEQQ1O9OEON3+5aqTjQlHzXTyZwU
$ pdnsutil activate-tsig-key example.com foobar primary
$ cat > /tmp/foobar.tsig <<< 'EOF'
key "foobar" {
algorithm hmac-sha512;
secret "asdfasdfYpM2nUHdJtC3mDJnbBIXzUGgDU2/dwz8cDAfEQQ1O9OEON3+5aqTjQlHzXTyZwU";
};
EOF
$ dig +short -k /tmp/foobar.tsig txt testing.example.com @127.0.0.4
"this is a test"
"this is another test"
$ dig +short -k /tmp/snafu.tsig txt testing.example.com @127.0.0.4
;; Couldn't verify signature: expected a TSIG or SIG(0)
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.