jpgpi250 / piholemanual Goto Github PK

DOHipv4.txt should have the following added (From what i can see)

188.172.221.9
103.127.29.198
217.146.10.59
45.90.28.0
45.90.30.0

I can see by the Closed issue responses the answer is going to be to create an exception. I just wanted to note this for the public.

These four Squarespace web servers have been getting blocked for a few weeks:

ext-sq.squarespace.com. 151 IN A 198.185.159.145
ext-sq.squarespace.com. 151 IN A 198.49.23.144
ext-sq.squarespace.com. 151 IN A 198.185.159.144
ext-sq.squarespace.com. 151 IN A 198.49.23.145

AFAIK that's any web server Squarespace hosts. Occasionally they must drop out since they sometimes do work, like yesterday afternoon.

https://github.com/jpgpi250/piholemanual/blame/2b479f3ede78ea0ba88511c5c46f787d63bbddd4/DOHipv4.txt#L814

Hey,
104.21.87.171 seems to be a false positive

Hi @jpgpi250

This website is blocked because it tried to access 84.17.46.50 which is actually not in your list. But when i look at the pftables it does excist.

I tried to open this webpage: https://www.berkenrhode.nl/

Maby it should be in the exceptions?

Originally posted by @Giel538 in #7 (comment)

151.139.128.10 is a CDN for Rumble video and images.

I see it was added earlier this year to DOHipv4.txt, can you remove this?

Otherwise, I have to mirror your list and script to keep this IP removed.

Thanks

Update on Nov 6, 2023 broke Letsencrypt cert validation with acme.sh

First off - THANK YOU for your great effort, excellent pdf documents , and the "Best" DoH blocklist "known to mankind" :-)

On
https://github.com/oneoffdallas/dohservers

I noticed the below list.txt is 4 month old
https://raw.githubusercontent.com/oneoffdallas/dohservers/master/list.txt

Whereas the iplist.txt is 3 month old.
https://raw.githubusercontent.com/oneoffdallas/dohservers/master/iplist.txt

Some ip addresses has been committed
oneoffdallas/dohservers@59f4d33

As i use pfSense (great pdf guide) i'm using your list , and "dallas" iplist.txt , and noticed that a lot of 45.90.30.xx entries was missing from your list.
Actually seems like he added a full 45.90.30.x/24 as single ip entries.

Maybe you could just switch to the iplist instead.

Thanx from Denmark
/Bingo

I use your DoH IP list in my OPNSense firewall, it will block 104.16.132.229 on port 443.
This breaks cdnjs.cloudflare.com, result is that websites using this will load very very slow. An example is www.rtvnoord.nl
An NSLookup of cdnjs.cloudflare.com gives:
Non-authoritative answer:
Name: cdnjs.cloudflare.com
Addresses: 2606:4700::6810:85e5
2606:4700::6810:84e5
104.16.132.229
104.16.133.229

Is it possible to remove these IP's? If not, is there documentation or some proof that these IP's provide DoH? If yes, then I have to consider if whitelisting is an option.

Read the relevant docs to find out more

• Block DNS over HTTPS (DoH), using pfsense
• Using Unbound response policy zones

New domain entries (300+)

The exception list(s) are growing, this makes the IP block lists less effective. poisonsnak has indicated he doesn't use the exception list(s) but maintains exception lists, based on case by case. This is a very good idea, the effectiveness of the block list(s) is thus increased for your specific environment.
New entries (exceptions) will NOT be added to the exception lists, the exception lists are considered deprecated, the files will remain on GitHub, this to ensure the block rules don't cause problems in existing environments. The doc will be updated as soon as possible.

Bunny CDN is in there, causing some issues on my network that took me a while to figure out ;)

84.17.46.54
84.17.46.53

The entire CIDR 84.17.46.0/23 should be removed, as well as 2400:52e0:1e01::/48

Checking whois information on the subnet you'll easily spot that this belongs to a CDN.

Hi,

thank you for your great list first of all, however I want to flag something

151.139.128.10

this IP is not a DoH server but a CDN/Reverse proxy used by many websites. Ergo, if you implement it in your firewall list it will block multiple legit websites.

Thank you for your attention :)

Was hit by this one today

use.fontawesome.com was blocked , and i hung on a newspaper site.
Adding : 188.114.96.0 and 188.114.97.0 (A bit weird they used those ip's) - To my own local "permit list" resolved the issue

host use.fontawesome.com
use.fontawesome.com is an alias for use.fontawesome.com.cdn.cloudflare.net.
use.fontawesome.com.cdn.cloudflare.net has address 188.114.96.0
use.fontawesome.com.cdn.cloudflare.net has address 188.114.97.0
use.fontawesome.com.cdn.cloudflare.net has IPv6 address 2a06:98c1:3121::
use.fontawesome.com.cdn.cloudflare.net has IPv6 address 2a06:98c1:3120::

Hi,

it could be that this IPV4 and IPv6 address shouldn't be in your DoH list.
For me it looks like a normal Cloudflare LB IP pair because the domain
"vroptiker.de" uses this IPs for HTTP/HTTPS.

vroptiker.de has address 172.67.169.38
vroptiker.de has address 104.21.95.14
vroptiker.de has IPv6 address 2606:4700:3031::6815:5f0e
vroptiker.de has IPv6 address 2606:4700:3034::ac43:a926

For now im implementing a exception for this IPs and domain on my firewall.

Best greetings

Hello,

You have falsely listed 2a0a:6040:b204::2 in your list, this IP belongs to my company @Simulhost and we require you to immediately remove the aforementioned IP.

Kindly respond to this within 72 hours and comply with out request, otherwise we will request GitHub to take your repository down with immediate effect.

The DOHipv4 list contains an IP 255.255.255.255 which is causing me some issues.
https://raw.githubusercontent.com/jpgpi250/piholemanual/master/DOHipv4.txt

This is not a valid ip for a dns server in my opinion I use it in a bogon list for example. Can you comment if this is intended?

Kind regards
Pete

Tailscale.com stopped loading on my LAN segments, and it appears that it currently resolves to 76.76.21.21, which is in the DOHipv4 blocklist

This IP is use by a *.gouv.fr domain

https://openacademie.beta.gouv.fr/

Please remove from the DOHipv4 list.

Hi. Excellent work on the most comprehensive list around.

The following source has several IPs that are not present in your consolidated list.
https://github.com/dibdot/DoH-IP-blocklists

Thanks for your consideration.

Today I noticed weboost.com hangs while loading because the images that come from assets.wilsonelectronics.com are blocked.

assets.wilsonelectronics.com resolves to Cloudflare IPs for me:

104.26.2.137
104.26.3.137
172.67.69.149

and these are all in the list but not in the exception list. So I guess Cloudflare is providing DOH as well as hosting websites on these IPs? I'm not sure how the IPs were added to the list but I guess they must be DOH servers. Cloudflare's official DOH servers are all IPs like 1.1.1.1, 1.0.0.1, etc.

There was a similar issue opened here but the user closed it and decided to allowlist them himself #12 . Also some similar discussion here #3 which led to you creating the exception lists.

How do you feel about adding the Cloudflare IP ranges to the exception lists (https://www.cloudflare.com/ips/) ? A lot of websites use Cloudflare and I guess any of them could end up hosted on one of these IPs. But if Cloudflare is also using these for DOH then you have a huge gap in your blocklist. Sucks that they strongarm you into not blocking them by hosting websites on them. They don't include their official DOH IPs (1.1.1.1 etc.) in the IP lists I linked.

Personally I don't use the exception list and just write in my own exceptions case by case but I thought I'd share and get your view on it.

Hi,
An organization we're managing needs access to the website python-visualization.github.io which IP addresses are contained in the DOHipv4.txt file.
python-visualization.github.io has address 185.199.108.153
python-visualization.github.io has address 185.199.109.153
python-visualization.github.io has address 185.199.110.153
python-visualization.github.io has address 185.199.111.153

Is it possible to remove these IP's?

Hey jpgpi250,

can I ask you why the page "daisukivn.asuscomm.com" is inside your blocklist?
Is there a reason? Did you recognized attacks from this domain?

You can keep the Domain in the blacklist if you want.
I just need to know if there are problems with my Server

Best Regards

hello, how to remove iplist from list?
because is block my blog in cloudflare

172.67.216.46
104.21.45.153
2606:4700:3032::6815:2d99
2606:4700:3034::ac43:d82e

Hi,
I have the latest version of PiHole installed and wanted to run the GoogleAds script but get the following error:

This script was written for gravity database version 17 (current version: 15)

I do not find any documentation about how to upgrade the database. Can you help here out?

www.menards.com fails to load many required resources when blocking access to IPv4 151.139.128.10 which is in the DoH block. This IPv4 appears to originate in https://raw.githubusercontent.com/dibdot/DoH-IP-blocklists/master/doh-ipv4.txt. Project page: https://github.com/dibdot/DoH-IP-blocklists

dibdot's domain list includes the domain dns-secondary.cloudnx.cloud which resolves to IPv4 151.139.128.10

$ dig cloudnx.cloud
...
;; ANSWER SECTION:
cloudnx.cloud.		300	IN	A	151.139.128.10

www.menards.com loads a lot of resources from the domain sp.menardc.com which also resolves to IPv4 151.139.128.10

$ dig sp.menardc.com
...
;; ANSWER SECTION:
sp.menardc.com.		2383	IN	CNAME	z2t7k8j7.stackpathcdn.com.
z2t7k8j7.stackpathcdn.com. 250	IN	A	151.139.128.10

dibdot's project page lacks an "issues" page.
Are you interested in whitelisting this IPv4? Removing the list from your aggregator? Submitting a merge request to dibot's project?

The various disconnect.me apps use their own DOH provider which is not included on this blocklist, such as the Disconnect Privacy DNS App and Disconnect Privacy Pro on the App Store.

My Pi-Hole shows the lookup for their server as being "doh.disconnect.app":

An nslookup gives me these ip's for that domain:

% nslookup doh.disconnect.app
Server:		10.1.1.1
Address:	10.1.1.1#53

Non-authoritative answer:
Name:	doh.disconnect.app
Address: 104.21.65.52
Name:	doh.disconnect.app
Address: 172.67.140.171

Excellent write up on blocking DoH within PFSense however I note that the RPZ domain file doesn't include the firefox/apple canary domains?

https://jpgpi250.github.io/piholemanual/doc/Block%20DOH%20with%20pfsense.pdf

Noticed a few typos...

1. prevents canary domains from working:
   local-zone: " use-application-dns.net." always_nxdomain
   local-zone: " mask.icloud.com." always_nxdomain
   local-zone: " mask-h2.icloud.com." always_nxdomain

should not have a leading space...with it the domain does resolve, in my testing:
local-zone: "use-application-dns.net." always_nxdomain
local-zone: "mask.icloud.com." always_nxdomain
local-zone: "mask-h2.icloud.com." always_nxdomain

2. responibility > responsibility, p.6
   "IP addresses for doh1.b-cdn.net needs to be included (user responibility) in the aliases"

Thanks for the write-up and list!!

In commit 9ae71db the IP 151.139.128.10 was added.

That is being used by a US regional hardware store website, menards.com, to serve images, css and js files.

sp.menardc.com. 4926 IN CNAME z2t7k8j7.stackpathcdn.com.
z2t7k8j7.stackpathcdn.com. 4926 IN A 151.139.128.10

I'm wondering if that IP could be re-evaluated as being a Dns over HTTPS server?

We are blocking using this list, and the website has been not working for about a week, so maybe stackpathcdn just picked up that IP that was previously a DOH server?

Thanks

The site www.liveleak.com is not working properly.
This ip is blocked that causes the problem: 104.16.133.229

Could you solve this?

DNSCrypt is its own protocol and doesn't need TCP to function. Therefore any DNSCrypt providers should be removed from the DoH blocklists.

Hello,

You have falsely listed 144.172.67.5 in your list, this IP is controlled by my company https://github.com/Simulhost and we require you to immediately remove the aforementioned IP.

Kindly respond to this within 72 hours and comply with out request, otherwise we will request GitHub to take your repository down with immediate effect.

Hi there,

Thank you for maintaining this DOH Ipv4 list. It is very useful :)

Are there any other lists you are aware of? Also on your page you say you only block port 443 out to these IP's. I note some DOH provider use a different port FYI (eg: dnscrypt.ca is using port 453 see https://dnscrypt.ca/)
As a result I just block any traffic to these IP's and make use of a whitelist if required.

These are the IP lists I personally currently use on firewall:

DNSManualDNSList (manual added IP's),
1.1.1.1,1.0.0.1,1.1.1.2,1.0.0.2,1.1.1.3,1.0.0.3,104.17.64.4,104.17.65.4
DNSOneoffdallasDohservers,
https://raw.githubusercontent.com/oneoffdallas/dohservers/master/iplist.txt
DNSGreatWall,
https://raw.githubusercontent.com/Sekhan/TheGreatWall/master/TheGreatWall_ipv4
DNSjpgph
https://raw.githubusercontent.com/jpgpi250/piholemanual/master/DOHipv4.txt

On Pihole I also block the following:
https://raw.githubusercontent.com/oneoffdallas/dohservers/master/list.txt
https://raw.githubusercontent.com/Sekhan/TheGreatWall/master/TheGreatWall.txt

If you are aware of any further lists it would be helpful to know.
I think blocking the DNS names of DOH providers is also useful.

Kind regards
Peter

"To overcome this, two additional lists (DOHexceptionsIPv4.txt and
DOHexceptionsIPv6.txt) are provided."
https://raw.githubusercontent.com/jpgpi250/piholemanual/master/DOHexceptionsIPv4.txt
https://raw.githubusercontent.com/jpgpi250/piholemanual/master/DOHexceptionsIPv6.txt

Hi there,

I have a server running AdGuard Home leaked. I see its domain name is listed at https://github.com/jpgpi250/piholemanual/blob/master/DOH.rpz.

May I know how you obtain this list? I do not want to publize its server name to limit the users to only myself.

Cheers.

Hi

in your script a \ in a regex file is not stored in the database.

For example (from https://github.com/mmotti/pihole-regex/blob/master/miscellaneous/amp.list)
^(.+[.-])?amp(project)?.
It becomes ^(.+[.-])?amp(project)?.

or from https://github.com/mmotti/pihole-regex/blob/master/social/facebook.list
^(.+[_.-])?(facebook|fb(cdn|sbx)?|tfbnw).[^.]+$

I discover this because I also use your script for storing a local regex file with f.i. (^|.) (cdn.dns[0-6].).*

I investigate some time but at the moment cannot find the reason. Cause is the do loop.
A manual update works correct
sudo sqlite3 /etc/pihole/gravity.db "insert or ignore into domainlist (domain, type, enabled, comment) values ('.in-addr.arpa$', 3, 1, 'test1');"

With this script you can see the output in a file that ^mads. becomes ^mads.
No database access

=== script ====
#!/usr/bin

rm /home/pi/regex-processed.txt
sudo curl https://raw.githubusercontent.com/mmotti/pihole-regex/master/regex.list -o /home/pi/regex.list
while read regex
do
echo "$regex" >> /home/pi/regex-processed.txt

done < /home/pi/regex.list

=======
Also echo "$regex" >> /home/pi/regex-processed.txt or echo $regex >> /home/pi/regex-processed.txt doesn't work. Do you have an idea?

Another point: I also have expand you script with a backup and delete before updating the database table.
In this case removed regex lines are also removed in teh databease

sudo sqlite3 /etc/pihole/gravity.db "SELECT domain FROM domainlist WHERE type="3";" >> /home/pi/regex.list.backup
sudo sqlite3 /etc/pihole/gravity.db "DELETE FROM domainlist WHERE type="3";"

Thank you for the consolidated list you provide, this is very useful !

Server dns.nextdns.io is not blocked (although listed in https://github.com/curl/curl/wiki/DNS-over-HTTPS )

Is it deliberate for some reason ?

Loaded this into pfSense today, and tested using Intra. Found these IPs not in the list that are operating as DoH resolvers:

104.16.208.90
104.17.156.85
104.17.175.85
104.17.176.85
149.112.122.10
149.112.121.10
94.130.106.88
93.177.65.183

Had to rebuild one of my pi's that I use for Pi-Hole today and went to grab the latest versions of these scripts.

Upon running them I receive the following:

sh NextDNS.sh
NextDNS.sh: 21: [[: not found
--2022-08-19 12:23:01-- https://raw.githubusercontent.com/nextdns/cname-cloaking-blocklist/master/domains
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.111.133, 185.199.110.133, 185.199.108.133, ...
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.111.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 2250 (2.2K) [text/plain]
Saving to: ‘/home/pi/domains’

/home/pi/domains 100%[=========================================================================>] 2.20K --.-KB/s in 0.001s

2022-08-19 12:23:01 (1.72 MB/s) - ‘/home/pi/domains’ saved [2250/2250]

NextDNS.sh: 34: Syntax error: "(" unexpected (expecting "fi")

and

sh AdguardTeam.sh
AdguardTeam.sh: 24: [[: not found
--2022-08-19 12:21:21-- https://raw.githubusercontent.com/AdguardTeam/cname-trackers/master/script/src/cloaked-trackers.json
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.110.133, 185.199.108.133, 185.199.109.133, ...
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.110.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 4568 (4.5K) [text/plain]
Saving to: ‘/home/pi/cloaked-trackers.json’

/home/pi/cloaked-trackers.json 100%[=========================================================================>] 4.46K --.-KB/s in 0.002s

2022-08-19 12:21:22 (1.94 MB/s) - ‘/home/pi/cloaked-trackers.json’ saved [4568/4568]

AdguardTeam.sh: 40: Syntax error: "(" unexpected (expecting "done")

Any ideas?

Hi. Thanks for the script. I copied it over (downloaded it from you).
But when I run

$ sh /usr/local/bin/NextDNS.sh
or
$ sudo sh ./usr/local/bin/NextDNS.sh

I get always the same error as mentioned above:

--2020-05-16 19:21:57--  https://raw.githubusercontent.com/nextdns/cname-cloaking-blocklist/master/domains
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.36.133
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.36.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1011 [text/plain]
Saving to: ‘/home/pi/domains’

/home/pi/domains             100%[=============================================>]    1011  --.-KB/s    in 0s

2020-05-16 19:21:57 (6.85 MB/s) - ‘/home/pi/domains’ saved [1011/1011]

./usr/local/bin/NextDNS.sh: 11: ./usr/local/bin/NextDNS.sh: Syntax error: "(" unexpected (expecting "fi")

Any idea why?
Thank you in advance!