Code Monkey home page Code Monkey logo

http-connectivity-tester's Introduction

HTTP Connectivity Tester

Aids in discovering HTTP and HTTPS connectivity issues. Includes a PowerShell module named HttpConnectivityTester along with example PowerShell scripts to use the module.

Getting started

To get started using the tools:

  1. Download the repository as a zip file
  2. Configure PowerShell
  3. Extract the code
  4. Load the code
  5. Run the code

Downloading the repository

Download the current code to your Downloads folder. It will be saved as HTTP-Connectivity-Tester-master.zip by default.

Configuring the PowerShell environment

The PowerShell commands are meant to run from a system with at least PowerShell 4.0 and .Net 4.5 installed. PowerShell may need to be configured to run the commands.

Changing the PowerShell execution policy

Users may need to change the default PowerShell execution policy. This can be achieved in a number of different ways:

  • Open a command prompt and run powershell.exe -ExecutionPolicy Bypass and run scripts from that PowerShell session.
  • Open a command prompt and run powershell.exe -ExecutionPolicy Unrestricted and run scripts from that PowerShell session.
  • Open a PowerShell prompt and run Set-ExecutionPolicy Unrestricted -Scope Process and run scripts from the current PowerShell session.
  • Open an administrative PowerShell prompt and run Set-ExecutionPolicy Unrestricted and run scripts from any PowerShell session.

Unblocking the PowerShell scripts

Users will need to unblock the downloaded zip file since it will be marked as having been downloaded from the Internet (Mark of the Web) which PowerShell will block from executing by default. Open a PowerShell prompt and run the following commands to unblock the PowerShell code in the zip file:

  1. cd $env:USERPROFILE
  2. cd Downloads
  3. Unblock-File -Path '.\HTTP-Connectivity-Tester-master.zip'

Running the PowerShell scripts inside the zip file without unblocking the file will result in the following warning:

Security warning Run only scripts that you trust. While scripts from the internet can be useful, this script can potentially harm your computer. If you trust this script, use the Unblock-File cmdlet to allow the script to run without this warning message. Do you want to run C:\users\user\Downloads\script.ps1? [D] Do not run [R] Run once [S] Suspend [?] Help (default is "D"):

If the downloaded zip file is not unblocked before extracting it, then all the individual PowerShell files that were in the zip file will have to be unblocked. You will need to run the following command after Step 5 in the Loading the code section:

Get-ChildItem -Path '.\HTTP-Connectivity-Tester' -Recurse -Include '*.ps1','*.psm1','*.psd1' | Unblock-File -Verbose

See the Unblock-File command's documentation for more information on how to use it.

Extracting the code

  1. Right click on the zip file and select Extract All
  2. At the dialog remove HTTP-Connectivity-Tester-master from the end of the path since it will extract the files to a HTTP-Connectivity-Tester-master folder by default
  3. Click the Extract button
  4. From the previously opened PowerShell prompt, rename the HTTP-Connectivity-Tester-master folder to HTTP-Connectivity-Tester mv .\HTTP-Connectivity-Tester-master\ .\HTTP-Connectivity-Tester\

or

  1. From the previously opened PowerShell prompt, type Expand-Archive -Path .\HTTP-Connectivity-Tester-master.zip -DestinationPath .\

The Expand-Archive command is available starting with PowerShell 5.0.

Loading the code

Extract the downloaded zip file and install the HttpConnectivityTester PowerShell module.

  1. cd HTTP-Connectivity-Tester
  2. Inside the HTTP-Connectivity-Tester folder is another folder named HttpConnectivityTester which is the HttpConnectivityTester PowerShell module. Move this folder to one of the PowerShell module directories on the system. Open a PowerShell prompt and type $env:PSModulePath to see the locations where PowerShell modules can be installed. PowerShell 4.0 and later allow modules to be installed at the following paths by default: %ProgramFilesDir%\WindowsPowerShell\Modules;%SystemRoot%\System32\WindowsPowerShell\v1.0\Modules;%UserProfile%\Documents\WindowsPowerShell\Modules\
  3. mv .\HttpConnectivityTester "$env:USERPROFILE\Documents\WindowsPowerShell\Modules"
  4. Close the PowerShell prompt and open a new PowerShell prompt
  5. Go to the Examples folder cd .\Examples from the extracted download
  6. Go to the vendor specific folder cd .\Microsoft
  7. Go to the product/service specific folder cd .\WindowsTelemetry\
  8. Import the product/service specific connectivity test Import-Module -Name .\WindowsTelemetryConnectivity.psm1

Running the code

Call the main Get- command (e.g. Get-WindowsTelemetryConnectivity) after importing the product/service specific connectivity test to execute the test. The main Get- command is named after the file name. For example, Get-WindowsTelemetryConnectivity is the main Get- command for the WindowsTelemetryConnectivity.psm1 file. The main Get- command is Get-WDATPConnectivity for the WDATPConnectivity.psm1 file. The product/service specific Get- command is a wrapper around the Get-HttpConnectivity command provided by the PowerShell module.

The main Get- command for each connectivity test supports the same common options:

  • -Verbose - prints verbose output to the console
  • -PerformBlueCoatLookup - useful for looking up the rating of a URL when a BlueCoat proxy is being used. A rate limit is enforced for accessing the BlueCoat SiteReview REST API so use this option only when behind a BlueCoat proxy and use it sparingly. The script will automatically rate limit BlueCoat requests after every 10 requests and will then pause for 5 minutes.

Some Get- commands support additional unique options that can be discovered by running the built-in Get-Help command on the main Get- command (e.g. Get-Help Get-WindowsTelemetryConnectivity -Detailed).

An example for running, viewing, and saving a connectivity test:

cd .\Examples\Microsoft\WindowsTelemetry\
Import-Module -Name .\WindowsTelemetryConnectivity.psm1
$connectivity = Get-WindowsTelemetryConnectivity -Verbose
$connectivity | Format-List -Property Blocked,TestUrl,UnblockUrl,DnsAliases,IpAddresses,Description,Resolved,ActualStatusCode,ExpectedStatusCode,UnexpectedStatus
Save-HttpConnectivity -Objects $connectivity -FileName ('WindowsTelemetryConnectivity_{0:yyyyMMdd_HHmmss}' -f (Get-Date))

Interpreting results

The main Get- command returns a Connectivity object that contains more information about the connectivity test. The main properties of interest from the Connectivity object that are useful for determining if a URL or service is blocked or functional are:

  • Blocked - whether the service appears to be blocked. Value should be false.
  • TestUrl - the URL that was used to perform the test.
  • UnblockUrl - the URL to get unblocked. Can be a URL pattern rather than a literal URL.
  • DnsAliases - a list of DNS aliases for the TestUrl. Pattern based unblocks of the TestUrl may need matching unblocks of all the DNS aliases.
  • IpAddresses - a list of IP addresses corresponding to the TestUrl. Unblocking based on the listed IP addresses is not effective due to cloud providers and content delivery networks that may return many different IP addresses.
  • Description - a description of what the URL is for.
  • Resolved - whether the URL resolves its DNS entry to IP addresses or DNS aliases. Value should be true.
  • ExpectedStatusCode - the expected HTTP status code returned by the test.
  • ActualStatusCode - the actual HTTP status code returned by the test. Value will be 0 when Blocked is true or Resolved is false.
  • UnexpectedStatus - was the actual status code an unexpected value regardless of whether the actual status code was the same as the expected status code.

See Interpreting results for more information.

Saving results

The Connectivity object can be saved to a JSON file using the Save-HttpConnectivity command from the PowerShell module. The Save-HttpConnectivity command supports the following options:

  • -Verbose - prints verbose output to the console.
  • -Objects - the connectivity object, or an array of connectivity objects, to save to a JSON file.
  • -OutputPath - the path to a folder to save the JSON file to.
  • -FileName - the name of the file, minus the file extension, to save the connectivity object(s) to.

Connectivity tests

The table below documents the currently implemented connectivity tests in the Examples folder.

Vendor Product / Service
Adobe Adobe Reader Manager updates
Apple macOS updates
Google Chrome updates
Microsoft Azure Active Directory Self Service Password Reset
Microsoft Windows Analytics Update Compliance
Microsoft Windows Analytics Upgrade Readiness
Microsoft Windows Defender Advanced Threat Protection
Microsoft Windows Defender Antivirus
Microsoft Windows Defender SmartScreen
Microsoft Windows Telemetry
Microsoft Windows Update
Microsoft Azure AD Self Service Password Reset
Mozilla Firefox updates

Documentation

Additional documentation is available in the documentation folder.

License

See LICENSE.

Contributing

See CONTRIBUTING.

Disclaimer

See DISCLAIMER.

http-connectivity-tester's People

Contributors

iadgovadmin avatar iadgovuser1 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

http-connectivity-tester's Issues

Add GovCloud URLs

When a particular service has a GovCloud version, then add tests for the GovCloud URLs.

Perform additional server certificate validation

Some types of certificate validation failures don't always result in an obvious failure. One case appears to be when a transparent proxy is involved that performs SSL/TLS interception. Perform additional server certification validation with Test-Certificate to hopefully make this case more clear to the user.

Add timeouts to invoke-webrequeset, resolve-dnsname, etc

Seeing some really long timeouts on some networks when DNS resolve fails. Resolve-DnsName has -QuickTimeout but it doesn't appear to have a set timeout. I found a post online complaining it still took 4 seconds which is fine. Invoke-WebRequest has -TimeoutSec and a value can be specified. The documentation says "Specifies how long the request can be pending before it times out. Enter a value in seconds. The default value, 0, specifies an indefinite time-out. A Domain Name System (DNS) query can take up to 15 seconds to return or time out. If your request contains a host name that requires resolution, and you set -TimeoutSec to a value greater than zero, but less than 15 seconds, it can take 15 seconds or more before a WebException is thrown, and your request times out."

Consider combining Windows Analytics tests

The Windows Analytics tests could be combined into one file since the URLs that are tested are very similar. Add an option for type of the Windows Analytic service to test. Values would be 'UpgradeAnalytics','UpgradeReadiness','DeviceHealth'

Suppress expected HTTP error codes from populating status message

Check status code for 400, 404, 503. Don't populate status message for those expected error codes. The message just ends up being distracting to end users. Could set a property "HasUnexpectedError" to signal this condition

if(-not(statusCode -in @(400, 404, 503)) {
$HashUnexpectedError = $true
$statusMessage = Get-ErrorMessage -ErrorRecord $_
}

WDATP down level URL tests

Add down level URL tests for WDATP. Add another parameter such as -WorkspaceID to prepend to the downlevel URLs

Only perform tests if DNS resolves

Move up the calls to Get-IPAddress and Get-IPAlias to before testing the URL and then only make that call if the address resolves. This should speed up tests in the case where timeouts occur in the web request calls since we won't even make the call. Related to #14.

Add known cloud provider/CDN/loading balancing IP range look ups

Addresses and Aliases property on the connectivity object are useful, but could be better. Combine them per URL:

  • URL
  • DNS aliases
  • IP addresses for those aliases
  • Boolean to denote if the IP address is part of a known cloud provider or CDN
  • If boolean is true, then the name of the provider
  • If boolean is true, then the address ranges that would need to be unblocked

This feature would help those who can't do URL or URL pattern name based unblocks and can only do IP address based unblocks.

Need a function to test if an IP address is in a known range specified in CIDR notation. Could take a dependency on a library or create needed functionality in pure PowerShell.

Need to gather address ranges of common/popular providers or download at runtime ( -IncludeProviders option?)

Amazon AWS

Microsoft Azure

Google Compute Engine

Apple

Akamai

Cloudflare

Mass surveilance backdoor exploits not hidden from results

The testing scripts' code doesn't redact several vulnerabilities in tested endpoint systems that actually allow us to exploit them in the wild and complete our ultimate mission as per the PRISM, Pinwale, Bullrun, and X-Keyscore programs. This is a complete failure of scope, where are our tax dollars going, other that literally down the drain?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.