Comments (9)
The domain entries from https://ahadns.com/dns-over-https/ have been added, this to ensure the DoH servers remain blocked, even if removed from other lists.
from piholemanual.
first, how I got the IP addresses you mentioned:
104.26.2.137 blitz.ahadns.com
104.26.2.137 blitz-setup.ahadns.com
104.26.3.137 blitz.ahadns.com
104.26.3.137 blitz-setup.ahadns.com
172.67.69.149 blitz.ahadns.com
172.67.69.149 blitz-setup.ahadns.com
second, lists where these domains appear in:
blitz.ahadns.com: 7, 11, 12, 15
blitz-setup.ahadns.com: 11, 12
7: https://raw.githubusercontent.com/wiki/curl/curl/DNS-over-HTTPS.md
11: https://raw.githubusercontent.com/dibdot/DoH-IP-blocklists/master/doh-ipv4.txt
12: https://raw.githubusercontent.com/dibdot/DoH-IP-blocklists/master/doh-ipv6.txt
15: https://raw.githubusercontent.com/jbaggs/doh-intel/master/doh.intel
only list 15 is new, since 2022-08-02 ,see NOTICE here
third, IPv6 addresses, associated with this domain:
2606:4700:20::681a:289 blitz.ahadns.com
2606:4700:20::681a:389 blitz.ahadns.com
2606:4700:20::ac43:4595 blitz.ahadns.com
2606:4700:20::681a:289 blitz-setup.ahadns.com
2606:4700:20::681a:389 blitz-setup.ahadns.com
2606:4700:20::ac43:4595 blitz-setup.ahadns.com
It is a very good idea to maintain your own exception list, this prevents the excessive growht of exceptions, using the generic list.
I will (a soon as possible) update the doc to encourage new users to create a personal exception list, this to avoid entries that don't apply to the users environment / region, and ever growing exception list(s).
The DoH IP addresses, mentioned above, will thus NOT be added to the exceptions, the exceptions lists are considered deprecated, the files will remain on GitHub, this to ensure the block rules doesn't cause problems in existing environments.
I strongly recommend, if you are using unbound (or bind, knot resolver, ...), to look into and add the RPZ (response policy zone) feature (mentioned in de DoH manual, explained in full in a separate manual). The idea is, even though you have whitelisted (exception) a specific IP address, the DNS name is still blocked. A client using a different name than the DoH name will not be hindered, getting the required resources (images in this specific case), a client trying to find the address for a DoH service, using the DNS name will be blocked.
Of course, if a client (app) is using IP addresses to get to the DoH servers, the RPZ will do nothing, the exceptions will make the DoH request succeed...
edit
I don't think adding the Cloudflare IP ranges (https://www.cloudflare.com/ips/) addresses (ranges) to the IP execption lists is a good idea, due to the amount of IP addresses that would be allowed. I very much like your approach to create customized (your own) exception lists, adding exceptions case by case.
/edit
edit2
from the current IPv4 exception list ( 37 entries) only 2 are in the cloudflare ranges (15)
from the IPv4 list ( 418 entries), 28 are in the cloudflare ranges
from the current IPv6 exception list ( 9 entries) only 2 are in the cloudflare ranges (7)
from the IPv6 list ( 281 entries), 36 are in the cloudflare ranges
tested with grepcidr (example usage https://unix.stackexchange.com/questions/274330/check-ip-is-in-range-of-whitelist-array - accepted solution)
/edit2
Your thoughts are welcome....
from piholemanual.
I've been working (thinking) almost all day to find the correct solution for this issue, and decided your suggestion (create a personal exception list) is the best solution.
I've editted my previous comment several times to get to the final version, you might want to reread it.
As explained (and also in the open NOTICE, I will consider the exceptionlists as deprecated.
Question for you:
what method do you use to define the exceptions?
options:
1: IP alias, has the advantage you can add a comment, everything in one place.
2: URL alias, requires extra files (v4 & v6) on a web server, also allows for comments (# entry on a separate line)
what method would be the best (consider NOT all users are as advanced as you are (with your great ideas!!).
Thanks for your time and effort, hoping to hear your thoughts.
from piholemanual.
Thanks for the detailed reply. It's good to know they are actual DoH servers.
Using the RPZ feature to block DNS names is a good idea. I forgot that DoH usually depends on "regular" DNS to get going. I'll be sure to block all the known hostnames for DoH servers. I use nextdns as my upstream server and I think they do this for me already with the "block bypass methods" toggle but I'll double check.
Thanks for using that grepcidr tool to figure out which IPs on your lists are in the Cloudflare ranges. I'd never heard of grepcidr before but it sounds handy. For me, on the one hand I want to block as many DoH servers as possible, but on the other hand I don't want to cause problems for my users. If it was just me I wouldn't add the full Cloudflare ranges to my exception list but I run a small business and the staff get kind of upset when "the internet" doesn't work haha. So I ended up adding the Cloudflare ranges to my exception list.
For the method I use to define the exception list, my router is a Ubiquiti Edgerouter so I just have a firewall rule that rejects outbound requests to servers on your list, but then I insert a rule just before it to allow access to IPs on my exception list. The Edgerouter allows one comment per rule so I can keep track of them that way. The way you've described it I think the IP alias sounds the best since it's simpler for those users that don't have as much experience.
Thanks again
from piholemanual.
So as I was looking into AhaDNS a bit more, I believe blitz.ahadns.com and blitz-setup.ahadns.com are just web sites that explain how to set up AhaDNS. Their actual DoH servers are set up in the format https://doh.[LOCATION].ahadns.net/dns-query . You can see the full list here https://ahadns.com/dns-over-https/ . So you might be able to remove those IPs for blitz.ahadns.com and blitz-setup.ahadns.com after all. The ahadns.net servers are all in your list so we're covered there.
As I thought about it and felt somewhat nervous adding Cloudflare's huge IP ranges to my exception list, I wondered if Cloudflare would actually proxy DoH requests. AhaDNS's instructions don't list blitz.ahadns.com or blitz-setup.ahadns.com as usable DoH servers. Do you know of a way to test a DoH server? I tried adding it to Firefox's settings but even if I put a server there that I know is invalid, DNS still works so I can't use it as a test. I know dig recently got DoH support (https://www.isc.org/blogs/bind-doh-update-2021/) but the version I have in Debian Stable isn't new enough
from piholemanual.
The DoH rules prevent me from checking https://ahadns.com/dns-over-https/ right now, I'll look into that later, and if possible, add the list (check the doc for all lists used), this to ensure the ahadns servers remain in the block list. As explained, I only parse the lists I can find, extracting domains the authors have marked as (o)DoH servers. I never test if they are actual DoH servers, only try to get the IPs and add these to the block list. New domains are added, but never removed, althoug it is possible to keep track of removed entries, by verifying the timestamp of the entries in the database (see sqlite3 queries in the doc).
There is something called "oblivious DNS over HTTPS" wich uses a proxy. The proxy knows the IP of the requestor, this to be able to send the answer, the actual DoH server recieves all requests from the proxy, thus no client identification possible. Those providers that support oDoH all claim the proxy doesn't keep logs... Don't know if cloudflare supports oDoH.
There are lots of DoH clients on GitHub, I've tried some, in order to be able to test if it's really a DoH server, none of them stand out, they all have pro / cons and don't always work. Most of the lists only provide a domain (example ahadns.com), you need the URL to test (https://ahadns.com/dns-over-https/), so it's not always clear what to test.
You can install a windows version of dig, following this guide. Since the link you provided says "as of March, 2021" and the latest downloadable windows version says "July 2022", this could possibly work (not tested yet)
I'll update the manual as soon as possible, using the alias IP as the method to create "personal" exceptions, marking the GitHub exceptionlists as deprecated. The database version will increase to v4, given the new policy (create local exception lists), the exceptions table becomes obsolete.
from piholemanual.
All done, documentation, scripts, ...
generated new lists, this to ensure everything still works...
- time required to make the RPZ list (from database): 1m49s
- time required to make the DOH lists (from source lists): 51m50s
ever considered writing a manual to implement this on Ubiquiti Edgerouter? Would increase the user base...
from piholemanual.
Unfortunately ISC stopped providing Windows builds and you have to install 9.16.31 (but 9.17.10 is when they added DoH support). I have a machine running Debian Testing that should be able to install 9.18.4 so I will give that a go next time I have access to it, or worst case I can try one of those clients floating around github.
For the Edgerouter manual, I would but Ubiquiti has discontinued their EdgeMAX line. They aren't officially discontinued but they don't release updates for them any more - 2.0.9 is the current version for Edgerouter and it was released in 2020 (although it has gotten a few basic hotfixes since then - just things like updating openssl). You can't buy them anywhere either - I have one Edgeswitch I need and it's been on "back order" for 18 months. It's a shame because they are a really good product but Ubiquiti has gotten really weird the past few years. Their UniFi and AirMAX products are still available and see regular updates, but they aren't as easy to customize as the EdgeMAX products so it would be a lot harder to use your list on them (if it's even possible).
I use the script here modified to my own situation https://github.com/WaterByWind/edgeos-bl-mgmt and there's a megathread on the community here https://community.ui.com/questions/Emerging-Threats-Blacklist/62a9549e-ddae-4631-941d-b0878b2a13e0 . It pulls your list in once every couple days along with a bunch of other lists I use and updates them in the firewall automatically. So if you have Edgerouter users that are interested I'd send them in that direction.
from piholemanual.
added a section / method (manual) to limit the number of exceptions, can be used for CIDR addresses, such as cloudflare, google, ...
from piholemanual.
Related Issues (20)
- remove 167.99.242.112 from DOHipv4 HOT 2
- https://raw.githubusercontent.com/oneoffdallas/dohservers/master/list.txt - Out of sync HOT 2
- use.fontawesome.com - ip's are blocked - Hainging browser sessions if sites uses them HOT 2
- Block NextDNS DoH HOT 2
- Block Disconnect DOH server HOT 2
- Lack of canary domains HOT 1
- NOTICE 2022-08-02: new DoH lists added && time to build your own exception list(s) HOT 2
- NextDNS.sh and AdguardTeam.sh issues HOT 2
- Regex.sh doesnt't nog work correctly if a \ is used HOT 2
- domain name of a server running AdGuard Home leaked HOT 1
- Website in PiHole Blocklist HOT 1
- Typos in Block DNS over HTTPS (DoH), using pfsense PDF HOT 1
- cdnjs.cloudflare.com breaks when using DOH ipv4 list HOT 13
- IP's not in the list HOT 2
- www.liveleak.com not working HOT 3
- Thank You HOT 4
- Thank you and dns.nextdns.io HOT 24
- Another DoH source to add HOT 3
- Hi @jpgpi250 HOT 6
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from piholemanual.