nsacyber / http-connectivity-tester Goto Github PK

Add tests for Google Update URLs for Chrome browser. https://github.com/nsacyber/Windows-Secure-Host-Baseline/blob/master/Chrome/Scripts/Chrome.psm1 is a starting point.

Add tests for Windows Update URLs

The Windows Analytics tests could be combined into one file since the URLs that are tested are very similar. Add an option for type of the Windows Analytic service to test. Values would be 'UpgradeAnalytics','UpgradeReadiness','DeviceHealth'

Addresses and Aliases property on the connectivity object are useful, but could be better. Combine them per URL:

• URL
• DNS aliases
• IP addresses for those aliases
• Boolean to denote if the IP address is part of a known cloud provider or CDN
• If boolean is true, then the name of the provider
• If boolean is true, then the address ranges that would need to be unblocked

This feature would help those who can't do URL or URL pattern name based unblocks and can only do IP address based unblocks.

Need a function to test if an IP address is in a known range specified in CIDR notation. Could take a dependency on a library or create needed functionality in pure PowerShell.

• https://stackoverflow.com/questions/9622967/how-to-see-if-an-ip-address-belongs-inside-of-a-range-of-ips-using-cidr-notation
• https://github.com/lduchosal/ipnetwork (IPNetwork.Parse, IPAddress.Parse, ipnetwork.Contains)
• https://www.nuget.org/packages/IPNetwork2/
• https://stackoverflow.com/questions/32028166/convert-cidr-notation-into-ip-range?rq=1

Need to gather address ranges of common/popular providers or download at runtime ( -IncludeProviders option?)

Amazon AWS

• https://aws.amazon.com/blogs/aws/aws-ip-ranges-json/
• https://docs.aws.amazon.com/general/latest/gr/aws-ip-ranges.html
• https://docs.aws.amazon.com/quicksight/latest/user/regions.html
• https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/LocationsOfEdgeServers.html

Microsoft Azure

• https://www.microsoft.com/en-us/download/details.aspx?id=41653
• https://azurerange.azurewebsites.net/ and https://github.com/fchapleau/AzureRange
• https://www.danielstechblog.io/azure-services-urls-and-ip-addresses-for-firewall-or-proxy-whitelisting/
• https://docs.microsoft.com/en-us/azure/cdn/cdn-pop-list-api
• http://vijayjt.blogspot.com/2017/05/azure-public-ip-ranges-and-whitelisting.html
• https://blogs.msdn.microsoft.com/nicole_welch/2017/02/azure-ip-ranges/ (Gov and O365)

Google Compute Engine

• https://cloud.google.com/compute/docs/faq#where_can_i_find_product_name_short_ip_ranges

Apple

• 17.0.0.0/8
• https://www.richard-purves.com/2016/09/10/apple-services/
• https://www.richard-purves.com/2016/09/10/apple-services/
• https://stackoverflow.com/questions/10688852/ip-address-ranges-for-apns-servers

Akamai

• https://security.stackexchange.com/questions/38658/firewall-defined-akamai-ip-range

Cloudflare

• https://www.cloudflare.com/ips/

Support retrieving information for certificate chains that are a length greater than 1

Since I can't open an issue on goSecure repository I'm opening one here. There are missing files since the move from iadgov.github.io to nsacyber.github.io. Specifically the https://iadgov.github.io/goSecure/files/ directory is missing. Could the whole https://iadgov.github.io/goSecure/ directory be added to the repository?

The testing scripts' code doesn't redact several vulnerabilities in tested endpoint systems that actually allow us to exploit them in the wild and complete our ultimate mission as per the PRISM, Pinwale, Bullrun, and X-Keyscore programs. This is a complete failure of scope, where are our tax dollars going, other that literally down the drain?

Add down level URL tests for WDATP. Add another parameter such as -WorkspaceID to prepend to the downlevel URLs

Seeing some really long timeouts on some networks when DNS resolve fails. Resolve-DnsName has -QuickTimeout but it doesn't appear to have a set timeout. I found a post online complaining it still took 4 seconds which is fine. Invoke-WebRequest has -TimeoutSec and a value can be specified. The documentation says "Specifies how long the request can be pending before it times out. Enter a value in seconds. The default value, 0, specifies an indefinite time-out. A Domain Name System (DNS) query can take up to 15 seconds to return or time out. If your request contains a host name that requires resolution, and you set -TimeoutSec to a value greater than zero, but less than 15 seconds, it can take 15 seconds or more before a WebException is thrown, and your request times out."

Add an HTML report that could be handed to firewall admins. Top section would be a table containing Blocked, UnblockUrl, DnsAliases, IpAddresses, IpRange (see #2), and Description. Bottom section could be verbose output from the command. https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.utility/convertto-html

Move up the calls to Get-IPAddress and Get-IPAlias to before testing the URL and then only make that call if the address resolves. This should speed up tests in the case where timeouts occur in the web request calls since we won't even make the call. Related to #14.

Add tests for Mozilla Update URLs for Firefox. https://support.mozilla.org/en-US/questions/1125193#answer-882616 could be a starting point

When a particular service has a GovCloud version, then add tests for the GovCloud URLs.

Check status code for 400, 404, 503. Don't populate status message for those expected error codes. The message just ends up being distracting to end users. Could set a property "HasUnexpectedError" to signal this condition

if(-not(statusCode -in @(400, 404, 503)) {
$HashUnexpectedError = $true
$statusMessage = Get-ErrorMessage -ErrorRecord $_
}

Document the connectivity results object fields and why they are useful

Checking for chain length >= 0 rather than chain length >= 1.

Current list is 200,400,403,404,500,501,503,504. Some of those probably shouldn't be on the list. Need to go through all the tests and see what status code looks from internet that is totally open.

Add tests Adobe Reader DC update URLs. https://github.com/nsacyber/Windows-Secure-Host-Baseline/blob/master/Adobe%20Reader/Scripts/AdobeReader.psm1 is a starting point.

Some types of certificate validation failures don't always result in an obvious failure. One case appears to be when a transparent proxy is involved that performs SSL/TLS interception. Perform additional server certification validation with Test-Certificate to hopefully make this case more clear to the user.

Add functions to support proxy discovery. Look for default proxy, auto discovery/WPAD, WinINet proxy, WinHTTP proxy, etc. Add that info to the verbose output.