Pester throws an exception when run inside runspace about pester HOT 13 CLOSED

No problem, I will provide a PR.

from pester.

BTW, just for fun, repro code refactored to use ForEach-Object -Parallel works:

$cpuCount = [Environment]::ProcessorCount
$results = 1..($cpuCount * 2) | ForEach-Object -Parallel {
    Import-Module Pester -Force -ErrorAction Stop
    $pc = New-PesterConfiguration
    $pc.Run.Container = New-PesterContainer -ScriptBlock {Describe 'd' { It 'i' { 1 | Should -Be 1 }}}
    $pc.Output.Verbosity = 'None'
    $pc.Run.PassThru = $true
    $pc.Run.SkipRemainingOnFailure ='Block'
    [PSCustomObject]@{
        Test    = $_
        Result  = Invoke-Pester -Configuration $pc
    }
} -ThrottleLimit $cpuCount
$results

Output:

The issue is - this is for PowerShell 7+ only. I need parallelization on PowerShell 5.1. Also, @PaulHigin it looks like the $Host.UI.psobject.Properties behavior is ThreadJob bug or limitation?

from pester.

Did you check if UI is null? psobject.properties might work silently because of member enumeration. Not in front of a computer today so can't test myself

Does it work without -StreamingHost?

from pester.

It does work without StreamingHost 👀:

$Host.UI is not null but $Host.UI.psobject.Properties are:

This has to be ThreadJob bug or limitation that I didn't know about 🤔

from pester.

This is very strange:

1. I've changed the Pester.psm1 module to wait for debugger when $Host.UI.posobject.Properties are $null (and store the value in the $p variable):

2. Then I've attached to the process and entered the runspace for debugging:

And as you can see in the right window the $p variable is really $null but by the time the debugger was attached to the runspace the $Host.UI.psobject.Properties has all the values and is not null anymore 👀👀👀

from pester.

OK, got it! Finally! The $Host.UI is kind of lazy loaded! (possible race condition)

I have modified the Pester.psm1 once more with simple loop that waits until the $Host.UI.psobject.Properties are available:

And it works now - this is the output of the repro script now:

Really strange behavior. I would love if @SeeminglyScience could have a look at it for some thorough explanation.

from pester.

@fflaten I was able to chat with @SeeminglyScience yesterday on Discord, and for him it looks like race condition bug (I hope he doesn't mind me posting the conversation screenshot here):

I will investigate some more if that's PowerShell or ThreadJob module bug or something else. In the meantime, I'll not use StreamingHost in my parallel testing module since, in that case, I do not need it anyways.

from pester.

This is yet simplest repro case (the number of jobs has to be quite high though to get the repro):

1..500 | % {Start-ThreadJob {1 | Should -BeExactly 1; $supportsVT = $Host.UI.psobject.Properties['SupportsVirtualTerminal'].Value} -StreamingHost $Host -InitializationScript {Import-Module Pester -Force}} | Wait-Job | Receive-Job | Remove-Job -Force

Output:

Error:

from pester.

Thank you for the thorough troubleshooting! This is a weird one and a race-condition sounds right.

I was able to reproduce a couple of times then suddenly not, so wasn't able to see if we could make the repro simpler. I doubt module import and calling Should is relevant, so was hoping to reproduce with sleep in scriptblock and/or initializationscript ++.

Either way I support the suggested fix with a few conditions:

• Bug report is created in repo for ThreadJob or PowerShell
• Issue is referenced in comment above the fix

Reason being that checking psobject.properties and not just $host.UI is unexpected and would likely be removed in a future cleanup/refactor PR + hopefully they're able to fix the root cause which might help others as well 🙂

Do you want to provide the PR once the bug report is created by you or Patrick?

from pester.

I was able to reproduce the issue without InitializationScript but I still needed to call Should. I was unable to reproduce the issue otherwise.

I also think that since this is race condition then checking only psobject.Properties is not the right solution, because the exception could happen on $Host.UI for someone else, therefore I would refrain from implementing my current workaround.

I think that for the moment there are 2 options:

1. I could suggest in the workaround section to not use StreamingHost when calling Pester from Start-ThreadJob
2. PR with a full $Host.UI check:

 elseif ($null -ne $host.UI -and ($hostProperties = $host.UI.psobject.Properties) -and ($supportsVT = $hostProperties['SupportsVirtualTerminal']) -and $supportsVT.Value) {

What do you think @fflaten?

Ps. I will create an issue in the ThreadJob module repo though.

from pester.

@kborowinski - In your original testing when using Start-ThreadJob I see that you are using the older of the two ThreadJob modules. In my testing in 7.3.6 & 5.1 on Microsoft Windows 10.0.25905 - aka Win 11 23H2 via the Insiders Dev ring with the newer Microsoft.PowerShell.ThreadJob module v2.1.0 I did not get it throwing that error, using this to test 1..500 | % {Start-ThreadJob {1 | Should -BeExactly 1; $supportsVT = $Host.UI.psobject.Properties['SupportsVirtualTerminal'].Value} } | Wait-Job | Receive-Job

So this may be fixed for you using that module (which due to command ordering preferences is being autoloaded over the ThreadJob Module for me) so can you please test this out for me?

Also this wasn't tested using Windows Terminal & I am just about to retest there too.

Additional background info
ThreadJob (v2.0.3) as is what is currently included in 7.3.6 however in future (somepoint soon as I have a PR for this) Microsoft.PowerShell.ThreadJob (v2.1.0 - and is the successor to ThreadJob going forward - joys of module renames eh!!) will also be included. I also noted that you mention that you need this to work on PowerShell 5.1 as well as in 7.x.x - does testing with Microsoft.PowerShell.ThreadJob (v2.1.0) on both repro for you too?
FYI Paul Higin has recently retired so may not answer the question you posted to him above & also thanks for rasing this issue in the PowerShell Org ThreadJob Repo too though as above I think this possibly may have been fixed.

from pester.

@kilasuit Unfortunately the same issue occurs in Microsoft.PowerShell.ThreadJob v2.1.0

from pester.

2. PR with a full $Host.UI check:

 elseif ($null -ne $host.UI -and ($hostProperties = $host.UI.psobject.Properties) -and ($supportsVT = $hostProperties['SupportsVirtualTerminal']) -and $supportsVT.Value) {

I'm fine with this workaround incl. a comment with link to this issue since I doubt we'll be able to cover this with a test consistently.

Thanks for really troubleshooting this and sorry for the delayed response 🙂 . Would you still like to provide a PR?

from pester.