Running on ADO issue about dotnet-test-rerun HOT 5 OPEN

Hi @dgkanatsios ,

Could you please run the tool with loglevel Verbose to see if we can get some more logs to understand what may be happening?

from dotnet-test-rerun.

thanks @ricardofslp for the response!

I modified the bash script to this and it works:

 - task: Bash@3
      continueOnError: true
      displayName: 'Run functional tests with retries'
      inputs:
        failOnStderr: false
        targetType: 'inline'
        script: |
          # https://github.com/joaoopereira/dotnet-test-rerun
          dotnet tool install --global dotnet-test-rerun --version 1.8.0 --no-cache
          
          project="$(Build.SourcesDirectory)/src/project.sln"
          filter="(TestCategory=SomeCategory)"
          
          # build the test solution
          dotnet build $project
          
          # run the tests
          test-rerun $project --loglevel Verbose --no-build --results-directory $(Agent.TempDirectory) --settings "$(Build.SourcesDirectory)/src/test.runsettings" --logger trx --filter=$filter --inlineRunSettings '"TestRunParameters.Parameter(name=\"name\", value=\"value\")"'

I removed the verbosity from the dotnet test tool, maybe this is the culprit? Re-running it with changes now, will post back once I have a concrete answer. Thanks again for the great library!

from dotnet-test-rerun.

Not sure what's going on, but as soon as I add --verbosity detailed the test doesn't do retries and the output is different.

FWIW, I am running the tool with a .sln file that references .csproj files from various test projects, not sure if/how this makes a difference.

Without --verbosity detailed the output is what comes from the tool (this is --verbosity normal)

Starting test execution, please wait...
A total of 1 test files matched the specified pattern.
Results File: /mnt/vss/_work/_temp/_7550dd7ec000001_2023-12-05_05_08_28.trx

Passed!  - Failed:     0, Passed:     5, Skipped:     0, Total:     5, Duration: 8 s

Test retries are done successfully here!

With --verbosity detailed the output is similar to running dotnet test

Total tests: 287
Test Run Failed.
     Passed: 248
     Failed: 36
    Skipped: 3
 Total time: 22.2292 Minutes
     6>VSTest:
         MSB4181: The "Microsoft.TestPlatform.Build.Tasks.VSTestTask" task returned false but did not log an error.
       Done executing task "Microsoft.TestPlatform.Build.Tasks.VSTestTask" -- FAILED.
     6>Done building target "VSTest" in project "test.csproj" -- FAILED.
     6>Done Building Project "/mnt/vss/_work/2/s/src/test.csproj" (VSTest target(s)) -- FAILED.
     1>Done executing task "MSBuild" -- FAILED.
     1>Done building target "VSTest" in project "test.sln" -- FAILED.
     1>Done Building Project "/mnt/vss/_work/2/s/test.sln" (VSTest target(s)) -- FAILED.

Build FAILED.
    0 Warning(s)
    0 Error(s)

Time Elapsed 00:22:17.63

No test retries with this verbosity!

from dotnet-test-rerun.

Hi @dgkanatsios,

Regarding the way you are running should not make a difference.
It seems the issue could be related to the verbosity level.

If you run with other verbosity levels it also fails?

from dotnet-test-rerun.

hey @ricardofslp just edited my post to make it more visible, apologies for the confusion.

With --verbosity detailed, tool doesn't do retries. With --verbosity normal, retries work just fine. Not sure what's up with other verbosity levels.

from dotnet-test-rerun.