cammurray / orca Goto Github PK

View Code? Open in Web Editor NEW

273.0 20.0 43.0 686 KB

The Microsoft Defender for Office 365 Recommended Configuration Analyzer (ORCA)

PowerShell 100.00%

exchange-online orca threat-protection powershell-gallery atp mdo

orca's Introduction

ORCA

The Microsoft Defender for Office 365 Recommended Configuration Analyzer (ORCA)

Introduction

ORCA is a report that you can run in your environment which can highlight known configuration issues and improvements which can impact your experience with Microsoft Defender for Office 365 (formerly Office 365 Advanced Threat Protection).

What's in scope

Configuration Health Index
Configuration in EOP which can impact MDO
SafeLinks configuration
SafeAttachments configuration
Antiphish and antispoof policies.

What is Configuration Health Index

The configuration health index is a weighted value representing your configuration. Not all configuration is considered and some configuration is weighted higher than others. The index is represented as a percentage. How the configuration impacts the configuration health index is shown next to the recommendation in the report below as a positive or negative number. The impact to your security posture is a large consideration factor when rating the configuration.

Sounds good! How do I run it?

You will need the Exchange Online Management module first.

Exchange Online Management module is available via the PowerShell gallery:

Install for current user Install-Module -Name ExchangeOnlineManagement -Scope:CurrentUser Install for all users Install-Module -Name ExchangeOnlineManagement

Then, you'll need ORCA. We publish ORCA via the PowerShell gallery to make it easy and accessible for everyone.

To install, run PowerShell as an administrator and run the following command

Install-Module ORCA

Connect to Exchange Online and then run

Get-ORCAReport

To Run ORCA you need to have necessary permissions. The "View-Only Configuration" Exchange Online role is needed to run the tool.

For example you can use the following:

Global Reader - Azure Active Directory role
View-Only Organization Management - built in role group in Exchange Online

It's that simple! At the conclusion, your default browser will open with a report detailing the recommendations.

What if I have issues?

Log an Issue via the Issues tab in GitHub!

FAQ

Is it kept up to date?

Yes! We will push new rule definitions out by releasing a new version. Every time you run Get-ORCAReport, we check to see if you have the latest version and if not; we will prompt you to update via the console and also in the report.

What if I don't have Microsoft Defender for Office 365?

There is definitely a lot of value in running your Office 365 environment with Microsoft Defender for Office 365, for instance:

Automatic incident response capabilities
Attack simulation capabilities
Behavioural analysis (sandboxing) of malware
Time of click protection for malicious URLs
Advanced anti-phishing controls

However, there are also some checks within ORCA that are not Microsoft Defender for Office 365 specific which can impact the operation of Microsoft Defender for Office 365 and security within an Office 365 tenant. ORCA can still be ran on tenants without Microsoft Defender for Office 365, albeit with reduced qty. of checks..

How do I connect to a delegated organization?

If you have permissions to another organization, pass the -DelegatedOrganization flag to connect to it. If you are already connected to another organization, we will first disconnect you before connecting to the other org.

Outputs

We now support outputting to different formats (which can be useful for different purposes, such as tracking trending over time).

If the following isn't good enough for your purpose, don't despair, the output is fully modular. You can create your own output type by creating your own "output-name.ps1" file in the outputs directory. You'll need to create a class that extends the "ORCAOutput" class, and you'll need to implement an override for the RunOutput function. Easiest way is just to copy one of the existing ps1's to make your own (if you need to, that is..)

Running specifying an alternative output

To run specifying an alternative output module, instead of using the Get-ORCAReport command, use the Invoke-ORCA command.

Example outputting to JSON

Invoke-ORCA -Output JSON

Supported outputs

HTML

HTML is the output format that you'll get when you run Get-ORCAReport.

Example 1 - this is the equivalent of running Get-ORCAReport:

Invoke-ORCA -Output HTML

Example 2 - output to HTML but don't load the HTML

Invoke-ORCA -Output HTML -OutputOptions @{HTML=@{DisplayReport=$False}}

Supported Params

DisplayReport, Optional, Boolean - load the report at the conclusion of running ORCA
OutputDirectory, Optional, String - path to store the outputted html file, default is an appdata directory created automatically
EmbedConfiguration, Optional, Boolean - allows you to embed your configuration in to the HTML file for easy sharing/snapshotting

JSON

File with JSON formatted results.

Example:

Invoke-ORCA -Output JSON

CSV

Output flatted in to two CSV files, one for an overview, one for detail on each config item.

Example:

Invoke-ORCA -Output CSV

Supported Params

OutputDirectory, Optional, String - path to store the outputted json file, default is an appdata directory created automatically

CosmosDB

Useful for storing your results, trending, or displaying in a interface (PowerBI example coming soon)

We require the unofficial CosmosDB module 'CosmosDB' for this. You can find this on the PowerShell Gallery

The key that we will use is 'id' in the CosmosDB. Make sure you specify this as your key when you create your collection.

Example - To output in to MyCosmosAccount database MyCosmosDB, the default collection will be ORCA

Invoke-ORCA -Output Cosmos -OutputOptions @{Cosmos=@{Account='MyCosmosAccount';Database='MyCosmosDB';Key='YourKeyvalue'}}

Example - To output in to MyCosmosAccount database MyCosmosDB, in to a collection called MyORCA

Invoke-ORCA -Output Cosmos -OutputOptions @{Cosmos=@{Account='MyCosmosAccount';Database='MyCosmosDB';Key='YourKeyvalue==';Collection='MyORCA'}}

Supported Params

Account, Required, String - The Cosmos DB account that the database is found in
Database, Required, String - The Cosmos DB name
Key, Required, String - One of the keys for this Cosmos DB account
Collection, Optional, String - The collection to output in to, by default this will be ORCA

Embedding Configuration in HTML files (Optional)

Embedding

To embed configuration in to the ORCA HTML output use the EmbedConfiguration switch to Get-ORCAReport, example Get-ORCAReport -EmbedConfiguration or add EmbedConfiguration to Invoke ORCA if running manually Invoke-ORCA -Output HTML -OutputOptions @{HTML=@{EmbedConfiguration=$True}}

When embedding your configuration to the HTML file, note that the HTML file becomes confidential. It may contain sensitive information at this point, and only share it with trusted parties.

Reading embedded configuration

If you have a HTML ORCA report (generated after version 2.3) that has embedded configuration, run:

Get-ORCAReportEmbeddedConfig -File filename.html to read the configuration in to a hashtable.

Store that in to a PowerShell variable by running $Config = Get-ORCAReportEmbeddedConfig -File filename.html

Explore the data available by outputting the $Config variable, or enter a specific bit of config by using the key, e.g for looking at InboundConnectors, $Config['InboundConnector']

Charting

Version 2.3 introduces charting and historical information as a concept in to ORCA. When running ORCA an ORCA report, ORCA will look for previous reports in the same Output Directory for the same tenant. We will show a chart of how the recommendations progress between versions.

Additional notes

ORCA is developed by Product Managers at Microsoft, however, is not an official Microsoft utility.

For official, in-product configuration analysis, use the MDO config analyzer.

License

We're completely open source and as matter of fact we also use some open source components in our report.

We use the following components in order to generate the report

Bootstrap, MIT License - https://getbootstrap.com/docs/4.0/about/license/

The MIT License (MIT)

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Fontawesome, CC BY 4.0 License - https://fontawesome.com/license/free
ChartJS, MIT License - https://github.com/chartjs/Chart.js

The MIT License (MIT)

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

MomentJS, MIT License - https://github.com/moment/moment/

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

orca's People

Contributors

Stargazers

Watchers

Forkers

digitalarche mahybabu iliaselmatani nehaxp blue-infosec smuggz damianscoles timtheresearcher classicsecurity absoblogginlutely nehavpandey sequel-data saipraveen lmcdonald100 beaglemarauder snehakishore21 vegardodegaard hocinewalid andrewmichaellloyd conosco-tech-consulting adrischw tmjr75 mutyrtsscv567t6 kuko73 spveggal cthulhu897 pakkiraja philrice l4m4 earlcrane iramku keescx gershonmx bpbmcs sangling fancypanda2020 anbakal dvintial wullieb1 rebelinux trisdev75 incloudtecnologia

orca's Issues

ORCA future

I noticed in the Security and Compliance center under Threat Management -> Policy, there is a component called Configuration Analyzer which recommends settings that overlap with recommendations generated in the ORCA reports. Is this configuration analyzer going replace the ORCA reports? I notice there hasn't been a new version of ORCA since October. Is all development now going into the Configuration analyzer? Of does one not have anything to do with the other? Put another way, is there any road map for ORCA?

Output File name and directory

Cannot ouput the file to a custom directory or file name. Tried:

OutputFile
OutputDirectory
-ReportFileName
-Report

All of which save it in app data with the scripted name format. How do achieve custom directory/name?

Thanks.

Check for insufficient permissions

In case user has not "Exchange service admin" rights first cmdlet which fails is Get-HostedConnectionFilterPolicy.
I recommend to add a simple check like this:

if(!(Get-Command Get-HostedConnectionFilterPolicy -ErrorAction SilentlyContinue))
{
Write-Host "Exchange PowerShell module not loaded" -foregroundcolor red
break
}

False positive on DKIM / DNS Records

Getting results of "improvement." Opened Microsoft support ticket and they verified our records are correct.

Add direct links to settings in order to improve admins changes

On every control made by ORCA, it would be a good idea to add direct links (or at least an information) to easily find the settings that the administrators have to change in order to retrieve the recommanded settings.
It's a little bit hard for some administrators to find where they have to change settings.

Silent running?

Just a possible suggestion, is there a way to get an option to silence either the output to the screen and / or the final opening of the report file? Would like to run this headless (automated) and would like to get rid of any extraneous output (without cracking it open and making non-permanent changes). Thanks.

Using ORCA report in Azure Automation

Hi, I managed to get this working in Azure Automation, however it left me with several feedback points / suggestions.

I will paste the entire AzA powershell runbook below so that you can provide advice to others & test yourself, comments & suggestions are inline.

#Runbook designed for Azure Automation

#Display verbose output in test pane
$VerbosePreference = "Continue"

#Imported (required) Modules
<#
ExchagneOnlineManagement
ORCA
#>

#Get credentials for use throughout runbook. Note that the user must have "View Only Configuration" role as a minimum.
Write-Output "Getting Credential"
$Credential = Get-AutomationPSCredential -Name 'EOPAdmin' -Verbose

#Set email parameters - Note that Azure servers are in UTC, so convert the time to a local timezone
$EmailFrom = 'valid o365 [email protected]'
$EmailTo = @('[email protected]')
$EmailSubject = 'Exchange Online ORCA Report - ' + [System.TimeZoneInfo]::ConvertTimeBySystemTimeZoneId((Get-Date), 'AUS Eastern Standard Time')
$EmailBody = 'See attached file for the ORCA report
Generated using https://github.com/cammurray/orca
Report ran in Azure Automation'
$ReportPath = "$($env:LOCALAPPDATA)\Microsoft\ORCA\ORCAReport.html"

#Connect to Exchange Online - Don't let the Get-ORCAReport function attempt to connect, it needs to be the newer module & specify creds (No MFA)
Write-Output "Creating session"
Connect-ExchangeOnline -Credential $Credential

#Run the ORCA Report tools
Write-Output "Running ORCA - Expect a warning about no program to open HTML file"
#Note: The synopsis for Get-ORCAReport states to use "-Report" to specify the filename however the parameter is actually named "Output"
#Suggested updates for author ([email protected])

- Update synopsis with correct parameter names

- Update synposis / readme with required role "View Only Configuration"

- Add an option to not automatically attempt to execute the HTML file (running in Azure Automation this does not work) - Generates error "No application is associated with the specified file for this operation"

- Add a "Return" statement which is a string containing the full file path OR just return the HTML as a string so it can be used as an email body

---- Regarding the HTML itself, it does not render in Outlook or Outlook.com well if used as the message body - I believe this is due to the CSS used and the fact that it is not embedded.

---- Consider embedding the CSS and using more cross browser friendly HTML. Some tips here:

------- https://email.uplers.com/blog/email-rendering-issues-in-outlook-and-hacks/

------- https://www.emailonacid.com/blog/article/email-development/how-to-create-excellent-emails-for-outlook-com/

------- https://www.contactmonkey.com/blog/outlook-rendering-issues

------- https://litmus.com/blog/a-guide-to-rendering-differences-in-microsoft-outlook-clients

- Add an option to auto update (no user interaction) so that we don't have to manally import the updated module into our Automation account

---- Note that even with the "NoUpdate" parameter, an error is still generated if the NuGet module is not installed

------- "Find-Module : NuGet provider is required to interact with NuGet-based repositories. Please ensure that '2.8.5.201' or newer version of NuGet provider is installed."

------- This is caused by the "Invoke-ORCAVersionCheck" being run without checking if "NoUpdate" is specified

- Update HTML to include timezone info where it displays the date/time and/or allow user to specify timezone - Timezone of where the script is running can be found with "[System.TimeZoneInfo]::Local.ToString()"

#eg: (attachment)

$ReportFile = Get-ORCAReport -NoConnect -AutoUpdate -DontOpenHTML -OutputType 'FilePath'

Send-MailMessage -Attachments $ReportFile (+other SMTP parameters)

#eg: (string)

$ReportHTML = Get-ORCAReport -NoConnect -AutoUpdate -DontOpenHTML -OutputType 'HTMLString'

Send-MailMessage -Body $ReportHTML (+other SMTP parameters)

#Run the report
Get-ORCAReport -NoConnect -NoUpdate -Output $ReportPath

#If using the HTML as the email body, convert the system.object (array of strings) into a single string first
#Not using this as it doesn't render well as yet
#$FileContent = Get-Content -Path $ReportPath
#$EmailBody = Out-String -InputObject $FileContent

#For some reason, you have to get the credentials again or sending the email fails - Maybe a quirk of AZ Automation?
Write-Output "Getting Credential"
$Credential = Get-AutomationPSCredential -Name 'EOPAdmin' -Verbose

#Send an email with the report attached
Write-Output "Sending Email"
Send-MailMessage -Attachments $ReportPath
-Body $EmailBody -BodyAsHtml:$True
-DeliveryNotificationOption @('OnFailure') -From $EmailFrom
-SmtpServer 'smtp.office365.com' -Priority 'Normal'
-Subject $EmailSubject -To $EmailTo
-Credential $Credential -UseSsl:$True
-Port 587 -Verbose:$True
-Debug:$False -ErrorAction Continue
-WarningAction Continue

warning about Intra-org Safe Links points to deprecated page

ORCA 1.9.11

warning about Intra-organization Safe Links (Enable Safe Links between internal users) points to deprecated page:

from top of https://protection.office.com/safelinksconverged:

updated page is:
https://protection.office.com/safelinksv2

Powershell 7 support

Probably more of a question, ive noted the ExchangeOnlineManagement module has a preview for Powershell 7. Im hoping that at some point (and im wondering when) the ORCA module can be updated to work with this? Atm im being forced to use an automation account in azure to execute the report, i would much prefer to be using an Azure Powershell Function as soon as possible. Ofc ive no idea when ExchangeOnlineManagement will come out of preview..

orca script does not recognize EOP .us endpoints as EOP

Update orca script to recognize domain.mail.protection.office365.us as EOP.

It appears the orca script only recognizes domain.mail.protection.outlook.com as EOP at this time.

Problem when doing the collection.

Running the report gives me the error below.
This error is repeated for ORCA100 to ORCA230.
Can I change something in my powershell setup to fix this?

PS VersionTable:
Name                           Value
----                           -----
PSVersion                      5.1.19041.1
PSEdition                      Desktop
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
BuildVersion                   10.0.19041.1
CLRVersion                     4.0.30319.42000
WSManStackVersion              3.0
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1

Error

Get-ORCAReport
05/29/2020 09:10:31 Performing ORCA Version check...
New-Object : Cannot find type [ORCA100]: verify that the assembly containing this type is loaded.
At line:32 char:16
return & $origNewObject @psBoundParameters
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
CategoryInfo          : InvalidType: (:) [New-Object], PSArgumentException
FullyQualifiedErrorId : TypeNotFound,Microsoft.PowerShell.Commands.NewObjectCommand

The property 'ORCAParams' cannot be found on this object. Verify that the property exists and can be set.
At C:\Program Files\WindowsPowerShell\Modules\ORCA\1.7.5\ORCA.psm1:288 char:13
 $Check.ORCAParams = $ORCAParams
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
CategoryInfo          : InvalidOperation: (:) [], RuntimeException
FullyQualifiedErrorId : PropertyNotFound

[Request] Publish code in repo to allow pull requests

Hi @cammurray,

thanks for the great tool. Have you considered to also publish the actual code in the repo to allow the community to make pull requests.
I understand that there may be licensing issues that would prevent this, but it would make it so much easier to propose changes / features

All the best, Fabian

Output to Desktop

Hi,
I was just looking at the ORCA tool which is excellent, however I'm trying to write a script that will always output to the desktop, using userprofile\Desktop won't work because if people are using OneDrive KFM then the Desktop doesn't exist in that location(I'm guessing it'll be the same for FolderRedirection also), so I've been trying to use:

$DesktopPath = [System.Environment]::GetFolderPath([System.Environment+SpecialFolder]::Desktop)
Invoke-ORCA -Output HTML -OutputOptions @{HTML=@{OutputDirectory=$DesktopPath}}

The desktop path gives the correct result but it doesn't seem like the Output Directory will accept it as a location

Can you please see what you can do to get this working, or advise if I'm doing something wrong.

Thanks!

Application cannot be started..contact the application vendor

Update Help

In the Get-Help for Get-ORCAReport the '-Report' parameter is listed in Examples 2 and 3. This option does not exist in the cmdlet. Since the help is not updatable nor is there an online version of help ('-online' switch), can the help be updated to show that that '-Report' is not a valid option (removed from Help)? Can we then add Output to examples as well? Or take information from the Readme in this GitHub and place it in the Get-Help for the cmdlet? Thanks.

Invoke-ORCA -CSV Does not produce CSV File

Running Invoke-ORCA -CSV runs successfully, and spits out the last line of "Generating Report" But the log does not indicate that the CSV is actually created. Further, It does not show up in AppData\Local\Microsoft\ORCA or any other location if -OutputDirectory is specified.

Rights in documentation are not sufficient

Hi Community,

the documentation states
`For example you can use the following:

Global Reader - Azure Active Directory role
View-Only Organization Management - built in role group in Exchange Online`

However, upon testing with these permissions we get the error
ORCA was unable to connect to Exchange Online, or you do not have sufficient permissions to check ORCA related configuration.

It seems as if these permissions are not sufficient for all gathering all relevant data. Or am I missing something here?

Errors after typing "GetORCAReport"

I installed the module fine but every time I type "GetORCAReport" I get the following error:
get-orcareport : The 'get-orcareport' command was found in the module 'ORCA', but the module could not be loaded. For
more information, run 'Import-Module ORCA'.
At line:1 char:1

get-orcareport

  + CategoryInfo          : ObjectNotFound: (get-orcareport:String) [], CommandNotFoundException
  + FullyQualifiedErrorId : CouldNotAutoloadMatchingModule

Basic authentication required

Hi,
Is it possible to use the ORCA report tool without enabling basic auth?
Happy to assist in making this happen if you can let me know where I should start.

Thanks,
Daniel

05/19/2020 12:19:49 Performing ORCA Version check...
05/19/2020 12:21:49 Connecting to Exchange Online..
New-ExoPSSession : Connecting to remote server outlook.office365.com failed with the following error message : The
WinRM client cannot process the request. Basic authentication is currently disabled in the client configuration.
Change the client configuration and try the request again. For more information, see the about_Remote_Troubleshooting
Help topic.
At C:\Users\Username\AppData\Local\Apps\2.0\LR9KCPZ3.8JK\PLATQYJN.38Y\micr..tion_1f16bd4ec4c2bb19_0010.0000_673f37
c317fb5976\CreateExoPSSession.ps1:302 char:30
+ ... PSSession = New-ExoPSSession -UserPrincipalName $UserPrincipalName.Va ...
+                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ResourceUnavailable: (:) [New-ExoPSSession], PSRemotingTransportException
    + FullyQualifiedErrorId : System.Management.Automation.Remoting.PSRemotingDataStructureException,Microsoft.Exchang
   e.Management.ExoPowershellSnapin.NewExoPSSession```

DNS Server setting

In order to resolve a potential problem in case of a local split DNS configuration. Could you please add a new parameter in the command line (like -DNSName) to be able to select and force a DNS Server instead of using the default server settings.

Control values are inconsistent

The control values are inconsistently formatted. Some are like "ORAC-101", others like "101", and others like 101. We have a downstream process that uses these values, so it would be nice to have them in a consistent format. I attempted to submit a merge request with consistent values, but I got a 403.

Had to use Install-Module -Name ExchangeOnlineManagement

I couldn't use the http://aka.ms/exopsmodule I had to use
Install-Module -Name ExchangeOnlineManagement . It kept erroring out that the psmodule.client.exe.manifest did not succeed. Once I used the other Powershell command it worked like a champ. Just sharing in case others hit this.

Preset Security Policy shows up even when not set or disabled

Problem: The Preset Security Policies (both Standard and Strict) are shown in the report as "OK" marked with "Standard" or "Strict" tags.

This is the case even when the policies are not configured (i.e. there are no assignments / applies-to yet). Checking with PowerShell "Get-xxxRule" confirms that no rules are in place. PowerShell "Get-ATPProtectionPolicyRule" also returns nothing confirming the same. Yet the ORCA report gladly shows the policies/rules in each section in green and adds to the total "OK" count.

This is also the case when the policies are disabled.

Expected Behaviour: The preset security policies if disabled or not configured should either not be shown in the report, or should show up as informational as in the case of transport rules.

Version: 1.9.11

Steps to reproduce:

Use a tenant where the tile called "Preset Security Policies" under the "Templated Policies" section is never clicked. This means, the preset policies are not created yet.
Run ORCA to generate a report.
The report will not show the preset policies anywhere.
Click on the "Preset Security Policies" tile under the "Templated Policies" section. But do not configure anything and leave the templates/policies in disabled state.
Run ORCA to generate a report.
The report will show all the preset policies (both standard and strict) with respective green tags and adds to the "OK" score.

Note: Toggling enabled/disabled state does not make any difference.

Note 2: Once the "Preset Security Policies" tile is clicked, the present policies are created but without rules unless configured.

Possible enhancement - Orca Report Per Policy

Hello,

after the evaluation, orca report provides recommendation. In will happen that company has more policies (default policy where probably there will be default recommendation and additional policy where recommendation will be applied). As a result of the orca report, recommendation will still be visible although no user is actually using default policy but second policy is applied to all users. Might be good to have a filter while generating Orca report to include default policy or not to include default policy. That was company could have a clean recommendation that applies only to a certain policy.

Thank you very much for the report tool it made my life easier.

Return objects don't always have the required properties set

In check-ORCA117.ps1, the return object doesn't return the "Object" property when no ATP policies are present. E.g.:

According to this, the Object property needs to be present:
$this.CheckType=[CheckType]::ObjectPropertyValue

Line 76-79:
$ConfigObject = [ORCACheckConfig]::new()
$ConfigObject.ConfigItem="All"
$ConfigObject.ConfigData="Enabled False"
$ConfigObject.SetResult([ORCAConfigLevel]::Standard,"Fail")
This should be:
$ConfigObject = [ORCACheckConfig]::new()
$ConfigObject.Object ="All"
$ConfigObject.ConfigItem="IsEnabled"
$ConfigObject.ConfigData="False"
$ConfigObject.SetResult([ORCAConfigLevel]::Standard,"Fail")

Add Hours and Minutes to the Report file name

When running more than once a day, it will be overwritten
For example: Run the report, fix issues, run the report again. Now you don't have original to check back on what you changed.

Add Hours and Minutes:
ORCA--MMDDYYHHMM.html

Actually prefer for all Internationals :
ORCA--YYYYMMDDHHMM.html

Need for other output types i.e. json

Hi (caveat i know virtually nothing about exchange), im working for an organisation where we want to schedule a run of the ORCA report inspect the results and potentially notify individuals of configuration drift that may make their platform insecure. At present im thinking of leveraging this module for this task. It would be very useful to have a json representation of the report to accomplish this. Is this something you could consider adding? Otherwise i will have to fork and maintain a version of this code that would allow for this and would mean that i somehow have to keep up to date with enhancements that you make to this library which is not ideal.

Install EXO Remote Session dependency and detect if WinRMRunning

Add ability to detect if dependency (Exchange Online Remote Session) is installed and install it if not. Be sure to prompt user to install prior to doing it.

you can reference https://raw.githubusercontent.com/jingsta/powershell-scripts/master/Microsoft/O365/O365-SecurityAssessment.ps1 and re-use the function I used.

Need to also determine if WinRM service is running and if not, enable it.

$WinRMRunning = (Get-Service 'WinRM').Status
Enable-PSRemoting -Force

ORCA fails in Azure Cloud Shell

It appears something to do with $PSScriptRoot based on initial investigations.

'Invoke-ORCA -PerformVersionCheck $False' Produces Error

Trying to automate this for another reporting script and when I use the '-PerformVersionCheck $False' parameter, I am getting an error:

05/21/2020 22:11:09 Output - HTML
You cannot call a method on a null-valued expression.
At C:\Users\DScoles\Documents\WindowsPowerShell\Modules\ORCAPreview\1.8.1\Outputs\output-html.ps1:149 char:47

... Version $($this.VersionCheck.Version.ToString())</stron ...

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

CategoryInfo : InvalidOperation: (:) [], RuntimeException

FullyQualifiedErrorId : InvokeMethodOnNull

My guess is that the handling of the no version check is tripping something (I have not delved into it to check). I am using the 1.8.1 version of the Preview module so hopefully this is the right place to report that.

Orca Check 180 "Enable anti-spoofing protection" returning false positive

check-ORCA180.ps1 is checking if EnableAntiSpoofEnforcement returns as true, however EnableAntiSpoofEnforcement appears to have been renamed to EnableSpoofIntelligence in the policy.

This is causing ORCA to always fail this check with $Enabled = $False.

Updating the text EnableAntiSpoofEnforcement in the script to EnableSpoofIntelligence resolves this issue.

Many errors with latest version

In running version 1.4.5 I'm seeing a bunch of "Cannot find type" errors. At first I thought it was because I was using the new EXOv2 module so I tried it from the old one and I see the same things.

The only successful checks are
Anti-Spam Settings
Tenant Settings
Anti Phish Settings
Anti-Malware Settings
Transport Rules
ATP Policies
Accepted Domains
DKIM Configuration

PSVersion 5.1.18362.628
PSEdition Desktop
PSCompatibleVersions {1.0, 2.0, 3.0, 4.0...}
BuildVersion 10.0.18362.628
CLRVersion 4.0.30319.42000
WSManStackVersion 3.0
PSRemotingProtocolVersion 2.3
SerializationVersion 1.1.0.1

New-Object : Cannot find type [ORCA100]: verify that the assembly containing this type is loaded.
At line:32 char:16
+         return & $origNewObject @psBoundParameters
+                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidType: (:) [New-Object], PSArgumentException
    + FullyQualifiedErrorId : TypeNotFound,Microsoft.PowerShell.Commands.NewObjectCommand

New-Object : Cannot find type [ORCA101]: verify that the assembly containing this type is loaded.
At line:32 char:16
+         return & $origNewObject @psBoundParameters
+                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidType: (:) [New-Object], PSArgumentException
    + FullyQualifiedErrorId : TypeNotFound,Microsoft.PowerShell.Commands.NewObjectCommand

New-Object : Cannot find type [ORCA102]: verify that the assembly containing this type is loaded.
At line:32 char:16
+         return & $origNewObject @psBoundParameters
+                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidType: (:) [New-Object], PSArgumentException
    + FullyQualifiedErrorId : TypeNotFound,Microsoft.PowerShell.Commands.NewObjectCommand

New-Object : Cannot find type [ORCA103]: verify that the assembly containing this type is loaded.
At line:32 char:16
+         return & $origNewObject @psBoundParameters
+                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidType: (:) [New-Object], PSArgumentException
    + FullyQualifiedErrorId : TypeNotFound,Microsoft.PowerShell.Commands.NewObjectCommand

New-Object : Cannot find type [ORCA104]: verify that the assembly containing this type is loaded.
At line:32 char:16
+         return & $origNewObject @psBoundParameters
+                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidType: (:) [New-Object], PSArgumentException
    + FullyQualifiedErrorId : TypeNotFound,Microsoft.PowerShell.Commands.NewObjectCommand

New-Object : Cannot find type [ORCA105]: verify that the assembly containing this type is loaded.
At line:32 char:16
+         return & $origNewObject @psBoundParameters
+                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidType: (:) [New-Object], PSArgumentException
    + FullyQualifiedErrorId : TypeNotFound,Microsoft.PowerShell.Commands.NewObjectCommand

New-Object : Cannot find type [ORCA106]: verify that the assembly containing this type is loaded.
At line:32 char:16
+         return & $origNewObject @psBoundParameters
+                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidType: (:) [New-Object], PSArgumentException
    + FullyQualifiedErrorId : TypeNotFound,Microsoft.PowerShell.Commands.NewObjectCommand

New-Object : Cannot find type [ORCA107]: verify that the assembly containing this type is loaded.
At line:32 char:16
+         return & $origNewObject @psBoundParameters
+                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidType: (:) [New-Object], PSArgumentException
    + FullyQualifiedErrorId : TypeNotFound,Microsoft.PowerShell.Commands.NewObjectCommand

New-Object : Cannot find type [ORCA108]: verify that the assembly containing this type is loaded.
At line:32 char:16
+         return & $origNewObject @psBoundParameters
+                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidType: (:) [New-Object], PSArgumentException
    + FullyQualifiedErrorId : TypeNotFound,Microsoft.PowerShell.Commands.NewObjectCommand

New-Object : Cannot find type [ORCA108_1]: verify that the assembly containing this type is loaded.
At line:32 char:16
+         return & $origNewObject @psBoundParameters
+                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidType: (:) [New-Object], PSArgumentException
    + FullyQualifiedErrorId : TypeNotFound,Microsoft.PowerShell.Commands.NewObjectCommand

New-Object : Cannot find type [ORCA109]: verify that the assembly containing this type is loaded.
At line:32 char:16
+         return & $origNewObject @psBoundParameters
+                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidType: (:) [New-Object], PSArgumentException
    + FullyQualifiedErrorId : TypeNotFound,Microsoft.PowerShell.Commands.NewObjectCommand

New-Object : Cannot find type [ORCA110]: verify that the assembly containing this type is loaded.
At line:32 char:16
+         return & $origNewObject @psBoundParameters
+                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidType: (:) [New-Object], PSArgumentException
    + FullyQualifiedErrorId : TypeNotFound,Microsoft.PowerShell.Commands.NewObjectCommand

New-Object : Cannot find type [ORCA111]: verify that the assembly containing this type is loaded.
At line:32 char:16
+         return & $origNewObject @psBoundParameters
+                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidType: (:) [New-Object], PSArgumentException
    + FullyQualifiedErrorId : TypeNotFound,Microsoft.PowerShell.Commands.NewObjectCommand

New-Object : Cannot find type [ORCA112]: verify that the assembly containing this type is loaded.
At line:32 char:16
+         return & $origNewObject @psBoundParameters
+                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidType: (:) [New-Object], PSArgumentException
    + FullyQualifiedErrorId : TypeNotFound,Microsoft.PowerShell.Commands.NewObjectCommand

New-Object : Cannot find type [ORCA113]: verify that the assembly containing this type is loaded.
At line:32 char:16
+         return & $origNewObject @psBoundParameters
+                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidType: (:) [New-Object], PSArgumentException
    + FullyQualifiedErrorId : TypeNotFound,Microsoft.PowerShell.Commands.NewObjectCommand

New-Object : Cannot find type [ORCA114]: verify that the assembly containing this type is loaded.
At line:32 char:16
+         return & $origNewObject @psBoundParameters
+                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidType: (:) [New-Object], PSArgumentException
    + FullyQualifiedErrorId : TypeNotFound,Microsoft.PowerShell.Commands.NewObjectCommand

New-Object : Cannot find type [ORCA118_1]: verify that the assembly containing this type is loaded.
At line:32 char:16
+         return & $origNewObject @psBoundParameters
+                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidType: (:) [New-Object], PSArgumentException
    + FullyQualifiedErrorId : TypeNotFound,Microsoft.PowerShell.Commands.NewObjectCommand

New-Object : Cannot find type [ORCA118_2]: verify that the assembly containing this type is loaded.
At line:32 char:16
+         return & $origNewObject @psBoundParameters
+                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidType: (:) [New-Object], PSArgumentException
    + FullyQualifiedErrorId : TypeNotFound,Microsoft.PowerShell.Commands.NewObjectCommand

New-Object : Cannot find type [ORCA120_malware]: verify that the assembly containing this type is loaded.
At line:32 char:16
+         return & $origNewObject @psBoundParameters
+                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidType: (:) [New-Object], PSArgumentException
    + FullyQualifiedErrorId : TypeNotFound,Microsoft.PowerShell.Commands.NewObjectCommand

New-Object : Cannot find type [ORCA120_phish]: verify that the assembly containing this type is loaded.
At line:32 char:16
+         return & $origNewObject @psBoundParameters
+                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidType: (:) [New-Object], PSArgumentException
    + FullyQualifiedErrorId : TypeNotFound,Microsoft.PowerShell.Commands.NewObjectCommand

New-Object : Cannot find type [ORCA120_spam]: verify that the assembly containing this type is loaded.
At line:32 char:16
+         return & $origNewObject @psBoundParameters
+                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidType: (:) [New-Object], PSArgumentException
    + FullyQualifiedErrorId : TypeNotFound,Microsoft.PowerShell.Commands.NewObjectCommand

New-Object : Cannot find type [ORCA121]: verify that the assembly containing this type is loaded.
At line:32 char:16
+         return & $origNewObject @psBoundParameters
+                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidType: (:) [New-Object], PSArgumentException
    + FullyQualifiedErrorId : TypeNotFound,Microsoft.PowerShell.Commands.NewObjectCommand

New-Object : Cannot find type [ORCA122]: verify that the assembly containing this type is loaded.
At line:32 char:16
+         return & $origNewObject @psBoundParameters
+                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidType: (:) [New-Object], PSArgumentException
    + FullyQualifiedErrorId : TypeNotFound,Microsoft.PowerShell.Commands.NewObjectCommand

New-Object : Cannot find type [ORCA125]: verify that the assembly containing this type is loaded.
At line:32 char:16
+         return & $origNewObject @psBoundParameters
+                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidType: (:) [New-Object], PSArgumentException
    + FullyQualifiedErrorId : TypeNotFound,Microsoft.PowerShell.Commands.NewObjectCommand

New-Object : Cannot find type [ORCA139]: verify that the assembly containing this type is loaded.
At line:32 char:16
+         return & $origNewObject @psBoundParameters
+                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidType: (:) [New-Object], PSArgumentException
    + FullyQualifiedErrorId : TypeNotFound,Microsoft.PowerShell.Commands.NewObjectCommand

New-Object : Cannot find type [ORCA140]: verify that the assembly containing this type is loaded.
At line:32 char:16
+         return & $origNewObject @psBoundParameters
+                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidType: (:) [New-Object], PSArgumentException
    + FullyQualifiedErrorId : TypeNotFound,Microsoft.PowerShell.Commands.NewObjectCommand

New-Object : Cannot find type [ORCA141]: verify that the assembly containing this type is loaded.
At line:32 char:16
+         return & $origNewObject @psBoundParameters
+                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidType: (:) [New-Object], PSArgumentException
    + FullyQualifiedErrorId : TypeNotFound,Microsoft.PowerShell.Commands.NewObjectCommand

New-Object : Cannot find type [ORCA142]: verify that the assembly containing this type is loaded.
At line:32 char:16
+         return & $origNewObject @psBoundParameters
+                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidType: (:) [New-Object], PSArgumentException
    + FullyQualifiedErrorId : TypeNotFound,Microsoft.PowerShell.Commands.NewObjectCommand

New-Object : Cannot find type [ORCA143]: verify that the assembly containing this type is loaded.
At line:32 char:16
+         return & $origNewObject @psBoundParameters
+                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidType: (:) [New-Object], PSArgumentException
    + FullyQualifiedErrorId : TypeNotFound,Microsoft.PowerShell.Commands.NewObjectCommand

New-Object : Cannot find type [ORCA156]: verify that the assembly containing this type is loaded.
At line:32 char:16
+         return & $origNewObject @psBoundParameters
+                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidType: (:) [New-Object], PSArgumentException
    + FullyQualifiedErrorId : TypeNotFound,Microsoft.PowerShell.Commands.NewObjectCommand

New-Object : Cannot find type [ORCA158]: verify that the assembly containing this type is loaded.
At line:32 char:16
+         return & $origNewObject @psBoundParameters
+                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidType: (:) [New-Object], PSArgumentException
    + FullyQualifiedErrorId : TypeNotFound,Microsoft.PowerShell.Commands.NewObjectCommand

New-Object : Cannot find type [ORCA169]: verify that the assembly containing this type is loaded.
At line:32 char:16
+         return & $origNewObject @psBoundParameters
+                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidType: (:) [New-Object], PSArgumentException
    + FullyQualifiedErrorId : TypeNotFound,Microsoft.PowerShell.Commands.NewObjectCommand

New-Object : Cannot find type [ORCA179]: verify that the assembly containing this type is loaded.
At line:32 char:16
+         return & $origNewObject @psBoundParameters
+                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidType: (:) [New-Object], PSArgumentException
    + FullyQualifiedErrorId : TypeNotFound,Microsoft.PowerShell.Commands.NewObjectCommand

New-Object : Cannot find type [ORCA180]: verify that the assembly containing this type is loaded.
At line:32 char:16
+         return & $origNewObject @psBoundParameters
+                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidType: (:) [New-Object], PSArgumentException
    + FullyQualifiedErrorId : TypeNotFound,Microsoft.PowerShell.Commands.NewObjectCommand

New-Object : Cannot find type [ORCA189]: verify that the assembly containing this type is loaded.
At line:32 char:16
+         return & $origNewObject @psBoundParameters
+                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidType: (:) [New-Object], PSArgumentException
    + FullyQualifiedErrorId : TypeNotFound,Microsoft.PowerShell.Commands.NewObjectCommand

New-Object : Cannot find type [ORCA189_2]: verify that the assembly containing this type is loaded.
At line:32 char:16
+         return & $origNewObject @psBoundParameters
+                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidType: (:) [New-Object], PSArgumentException
    + FullyQualifiedErrorId : TypeNotFound,Microsoft.PowerShell.Commands.NewObjectCommand

New-Object : Cannot find type [ORCA205]: verify that the assembly containing this type is loaded.
At line:32 char:16
+         return & $origNewObject @psBoundParameters
+                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidType: (:) [New-Object], PSArgumentException
    + FullyQualifiedErrorId : TypeNotFound,Microsoft.PowerShell.Commands.NewObjectCommand

New-Object : Cannot find type [ORCA220]: verify that the assembly containing this type is loaded.
At line:32 char:16
+         return & $origNewObject @psBoundParameters
+                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidType: (:) [New-Object], PSArgumentException
    + FullyQualifiedErrorId : TypeNotFound,Microsoft.PowerShell.Commands.NewObjectCommand

New-Object : Cannot find type [ORCA221]: verify that the assembly containing this type is loaded.
At line:32 char:16
+         return & $origNewObject @psBoundParameters
+                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidType: (:) [New-Object], PSArgumentException
    + FullyQualifiedErrorId : TypeNotFound,Microsoft.PowerShell.Commands.NewObjectCommand

New-Object : Cannot find type [ORCA222]: verify that the assembly containing this type is loaded.
At line:32 char:16
+         return & $origNewObject @psBoundParameters
+                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidType: (:) [New-Object], PSArgumentException
    + FullyQualifiedErrorId : TypeNotFound,Microsoft.PowerShell.Commands.NewObjectCommand

New-Object : Cannot find type [ORCA223]: verify that the assembly containing this type is loaded.
At line:32 char:16
+         return & $origNewObject @psBoundParameters
+                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidType: (:) [New-Object], PSArgumentException
    + FullyQualifiedErrorId : TypeNotFound,Microsoft.PowerShell.Commands.NewObjectCommand

New-Object : Cannot find type [ORCA224]: verify that the assembly containing this type is loaded.
At line:32 char:16
+         return & $origNewObject @psBoundParameters
+                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidType: (:) [New-Object], PSArgumentException
    + FullyQualifiedErrorId : TypeNotFound,Microsoft.PowerShell.Commands.NewObjectCommand

No output file...

I have updated orca, verified permissions, attempted changing directory before running Get-ORCAReport but not matter what I do I don't get an output file..

final output is always..

08/13/2020 11:17:06 Complete! Output is in

Any suggestions?
(I am on Mac in a Windows VM, could it be a vm/host permission issue maybe?)

Unified Audit Log False Positive

Our tenant has unified audit log enabled, but orcareport shows "improvement."

Unsolveable issue in ORCA Report... false positive?

ORCA 1.9.11 - run date 12/12/2020

I got these 2 warnings for ATP:

Domain Impersonation Action → Configure domain impersonation action to Quarantine → Strict Preset Security Policy → TargetedDomainProtectionAction: MoveToJmf (should be: Quarantine)
Anti-spoofing protection → Enable anti-spoofing protection in Anti-phishing policy → Antispoof Enforced: False (should be: True)

Both of the remediation links above take you to the same page: https://protection.office.com/antiphishing ...

Here's the Default policy: (everything already set to enabled/Quarantine)

...and my 1 custom policy: (also all set to enabled/Quarantine)

Trying to remediate via PowerShell: (change is not allowed)

PS C:\> Set-AntiPhishPolicy -Id "Strict Preset Security Policy" -TargetedDomainProtectionAction Quarantine
WARNING: All recommended properties will be controlled by Microsoft.

...and, nothing in here about `Antispoof Enforced` being off or anything else to be enabled, etc... can't find a refrence to this setting anywhere...?

PS C:\> Get-AntiPhishPolicy | fl Id,*able*,*force*

Id                                  : Strict Preset Security Policy
Enabled                             : True
EnableTargetedUserProtection        : True
EnableMailboxIntelligenceProtection : True
EnableTargetedDomainsProtection     : True
EnableOrganizationDomainsProtection : True
EnableMailboxIntelligence           : True
EnableSimilarUsersSafetyTips        : True
EnableSimilarDomainsSafetyTips      : True
EnableUnusualCharactersSafetyTips   : True
EnableSpoofIntelligence             : True
EnableViaTag                        : True
EnableUnauthenticatedSender         : True
EnableSuspiciousSafetyTip           : True

Id                                  : Custom-AntiPhishPolicy
Enabled                             : True
EnableTargetedUserProtection        : True
EnableMailboxIntelligenceProtection : True
EnableTargetedDomainsProtection     : True
EnableOrganizationDomainsProtection : True
EnableMailboxIntelligence           : True
EnableSimilarUsersSafetyTips        : True
EnableSimilarDomainsSafetyTips      : True
EnableUnusualCharactersSafetyTips   : True
EnableSpoofIntelligence             : True
EnableViaTag                        : True
EnableUnauthenticatedSender         : True
EnableSuspiciousSafetyTip           : False

Id                                  : Office365 AntiPhish Default
Enabled                             : True
EnableTargetedUserProtection        : True
EnableMailboxIntelligenceProtection : True
EnableTargetedDomainsProtection     : True
EnableOrganizationDomainsProtection : True
EnableMailboxIntelligence           : True
EnableSimilarUsersSafetyTips        : True
EnableSimilarDomainsSafetyTips      : True
EnableUnusualCharactersSafetyTips   : True
EnableSpoofIntelligence             : True
EnableViaTag                        : True
EnableUnauthenticatedSender         : True
EnableSuspiciousSafetyTip           : False

are these false positives?

possibly related: #114

HTML output has several formatting errors

The closing tag is not matching the opening tag on several headers in the HTML output.
Some examples:

<h2 class='card-title'>Office 365 ATP Recommended Configuration Analyzer Report</h5>
-->
<h2 class='card-title'>Office 365 ATP Recommended Configuration Analyzer Report</h2>

<div class='card-header'><h5>Informational</h4></div>
<div class='card-body'>
<h2>1</h5>
-->
<div class='card-header'><<h5>Informational</h5></div>
<div class='card-body'>
<h2>1</h2>

<div class='card-header'><h5>Recommendations</h4></div>
<div class='card-body'>
<h2>13</h3>
-->
<div class='card-header'><h5>Recommendations</h5></div>
<div class='card-body'>
<h2>13</h2>

Possible enhancement - Filter by recommendation state

Possible enhancement would be that the report would filter by Informational, Recommendations or
OK when the state is clicked on top of the html report

Get-ORCAReport stalls at - Getting MX Reports for all domains

Hi Cam,

Awesome tool BTW, but currently checking on large Edu tenant and getting stalled at:
Getting MX Reports for all domains - consistently stopping here for 30 - 50 mins plus

12/14/2020 15:23:50 Getting Anti-Spam Settings
12/14/2020 15:23:53 Getting Tenant Settings
12/14/2020 15:23:54 Getting Anti Phish Settings
12/14/2020 15:23:55 Getting Anti-Malware Settings
12/14/2020 15:23:56 Getting Transport Rules
12/14/2020 15:23:58 Getting ATP Policies
12/14/2020 15:23:59 Getting Accepted Domains
12/14/2020 15:24:00 Getting DKIM Configuration
12/14/2020 15:24:01 Getting Connectors
12/14/2020 15:24:01 Getting MX Reports for all domains

Tried running Get-ORCAReport -Collections Get-AcceptedDomain more or less to try "something" and it seemed to get past it - but now reporting that ATP is not installed in Tenant?

12/14/2020 15:50:17 Performing ORCA Version check...
12/14/2020 15:50:26 Analysis - Anti-Spam Policies - Anti-Spam Policy Rules
12/14/2020 15:50:26 Analysis - Anti-Spam Policies - Safety Tips
12/14/2020 15:50:26 Analysis - Anti-Spam Policies - Phish Action
12/14/2020 15:50:26 Analysis - Anti-Spam Policies - Bulk Complaint Level
12/14/2020 15:50:26 Analysis - Anti-Spam Policies - High Confidence Spam Action
12/14/2020 15:50:26 Analysis - Anti-Spam Policies - Allowed Senders
12/14/2020 15:50:26 Analysis - Anti-Spam Policies - Bulk Action
12/14/2020 15:50:27 Analysis - Anti-Spam Policies - IP Allow Lists
12/14/2020 15:50:27 Analysis - Anti-Spam Policies - Domain Whitelisting
12/14/2020 15:50:27 Analysis - Anti-Spam Policies - Quarantine retention period
12/14/2020 15:50:27 Analysis - Anti-Spam Policies - Outbound spam filter policy settings
12/14/2020 15:50:27 Analysis - Anti-Spam Policies - High Confidence Phish Action
12/14/2020 15:50:27 Analysis - Anti-Spam Policies - Mark Bulk as Spam
12/14/2020 15:50:27 Analysis - Anti-Spam Policies - Spam Action
12/14/2020 15:50:27 Analysis - Anti-Spam Policies - Advanced Spam Filter (ASF)
12/14/2020 15:50:27 Analysis - Anti-Spam Policies - End-user Spam notifications
12/14/2020 15:50:27 Analysis - Connectors - Domains
12/14/2020 15:50:27 Analysis - Connectors - Enhanced Filtering Configuration
12/14/2020 15:50:27 Analysis - DKIM - Signing Configuration
12/14/2020 15:50:27 Analysis - DKIM - DNS Records
12/14/2020 15:50:27 Analysis - Malware Filter Policy - Malware Filter Policy Policy Rules
12/14/2020 15:50:27 Analysis - Malware Filter Policy - Internal Sender Notifications
12/14/2020 15:50:27 Analysis - Malware Filter Policy - Common Attachment Type Filter
12/14/2020 15:50:27 Analysis - Malware Filter Policy - External Sender Notifications
12/14/2020 15:50:27 Analysis - Tenant Settings - Unified Audit Log
12/14/2020 15:50:27 Analysis - Transport Rules - Domain Whitelisting
12/14/2020 15:50:27 Analysis - Zero Hour Autopurge - Zero Hour Autopurge Enabled for Malware
12/14/2020 15:50:27 Analysis - Zero Hour Autopurge - Zero Hour Autopurge Enabled for Phish
12/14/2020 15:50:27 Analysis - Zero Hour Autopurge - Zero Hour Autopurge Enabled for Spam
12/14/2020 15:50:27 Analysis - Zero Hour Autopurge - Supported filter policy action
12/14/2020 15:50:27 Generating Output
12/14/2020 15:50:27 Output - HTML
12/14/2020 15:50:28 Complete! Output is in C:\Users......
ORCA--202012141550.zip

ORCA will not connect when used to check an Exchange Online Hybrid instance

ORCA fails to connect in Exchange Online Powershell when used in an Exchange Online Hybrid setup,

PS C:\Users\sean.brackeen> Get-OrcaReport
11/18/2019 14:59:08 Performing ORCA Version check...
11/18/2019 14:59:10 Connecting to Exchange Online..
ORCA was unable to connect to Exchange Online.
At C:\Program Files\WindowsPowerShell\Modules\ORCA\1.2.1\ORCA.psm1:1343 char:9
+         Throw "ORCA was unable to connect to Exchange Online."
+         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : OperationStopped: (ORCA was unable...xchange Online.:String) [], RuntimeException
    + FullyQualifiedErrorId : ORCA was unable to connect to Exchange Online.

Looking at the other issue in this tracker it appears that it depends on Get-Mailbox to see if a connection is already available, which is not an available command once connected to Exchange Online in Hybrid mode, since Hybrid accounts do not always contain mailboxes.

I'm not sure if this is a WontFix since you may not be interested in fixing this for Hybrid deployments, but using EOP and ATP is valid for Exchange Hybrid, and I would be interested in using this script.

ORCA doesn't work with the new preview Exchange module

Currently, we require the released Exchange Online PowerShell module (http://aka.ms/exopsmodule) and will not detect the new preview module.

Code will be improved so that we support both the preview module and the exchange online powershell module.

EndUserSpamNotificationFrequency < 4 Is OK?

BP: Enable End-user Spam notification and set the frequency to 3 days . If customers have EndUserSpamNotificationFrequency = 1 or 2 this should be OK??

Get-ORCAReport does not work

Lots of failures, but most importantly does not show output.

Get-ORCAReport
02/22/2021 16:41:42 Performing ORCA Version check...
New-Object : Cannot find type [html]: verify that the assembly containing this type is loaded.
At line:32 char:18

    return @(& $origNewObject @psBoundParameters)

```
             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
```
- CategoryInfo : InvalidType: (:) [New-Object], PSArgumentException
- FullyQualifiedErrorId : TypeNotFound,Microsoft.PowerShell.Commands.NewObjectCommand

The property 'DefaultOutputDirectory' cannot be found on this object. Verify that the property exists and can be set.
At C:\Program Files\WindowsPowerShell\Modules\ORCA\1.9.11\ORCA.psm1:410 char:17

            $Output.DefaultOutputDirectory = Get-ORCADirectory

```
            ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
```
- CategoryInfo : InvalidOperation: (:) [], RuntimeException
- FullyQualifiedErrorId : PropertyNotFound

The property 'VersionCheck' cannot be found on this object. Verify that the property exists and can be set.
At C:\Program Files\WindowsPowerShell\Modules\ORCA\1.9.11\ORCA.psm1:413 char:17

            $Output.VersionCheck = $VersionCheck

```
            ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
```
- CategoryInfo : InvalidOperation: (:) [], RuntimeException
- FullyQualifiedErrorId : PropertyNotFound

New-Object : Cannot find type [ORCA100]: verify that the assembly containing this type is loaded.
At line:32 char:18

    return @(& $origNewObject @psBoundParameters)

```
             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
```
- CategoryInfo : InvalidType: (:) [New-Object], PSArgumentException
- FullyQualifiedErrorId : TypeNotFound,Microsoft.PowerShell.Commands.NewObjectCommand