👋 Hi, I’m @joaoopereira
Want to know me better? Check my personal website: joaoopereira.com website built with Hugo
automatic rerun dotnet failed tests
Home Page: https://www.nuget.org/packages/dotnet-test-rerun
License: GNU General Public License v3.0
Want to know me better? Check my personal website: joaoopereira.com website built with Hugo
dotnet test
allows passing inline RunSettings as the last arguments on the command line after "-- "
https://learn.microsoft.com/en-us/dotnet/core/tools/dotnet-test
That is missing in test-rerun
. test-rerun
only supports the --settings
parameter for runsettings.
My tests will only run when a certain TestRunParameter in the runssettings file is overriden with a value generated in a pipeline (a token in this case). This option is missing from the dotnet-test-rerun tool, but is possible with dotnet test
using the following syntax:
# cmd
dotnet test -- TestRunParameters.Parameter(name=\"myParam\", value=\"value\")
This is not well documented unfortunately (and this overlooked easily ;), this page on dotnet test options: https://learn.microsoft.com/en-us/dotnet/core/tools/dotnet-test#options links (last line of the options segment) to another page: https://github.com/Microsoft/vstest-docs/blob/main/docs/RunSettingsArguments.md where the snippet above can be found.
Is it possible to add this to the dotnet-test-rerun tool? Would be much appreciated. I really need rerun functionality, I have a working powershell script for this after some work, but I think your tool would be a much better solution.
Hi, it's me again! First off, thanks for your previous quick fix! Unfortunately, another problem popped up when applying the filters, this time pertaining to our tests decorated with the DataTestMethod
in MsTest:
Incorrect format for TestCaseFilter Missing Operator '|' or '&'. Specify the correct format and try again. Note that the incorrect format can lead to no test getting executed.
No test matches the given testcase filter `FullyQualifiedName~SomeTest (Argument1, Argument2) in path/Application.IntegrationTests.dll
The problem lies with the brackets ((
and )
) in the fully qualified name of the test. These should be escaped in order for the filter attribute to work properly; see https://github.com/Microsoft/vstest-docs/blob/main/docs/filter.md.
Now this could be as easy as replacing testsToRerun
on line 74 of RerunCommand.cs with Regex.Replace(testsToRerun, @"[()]", @"\$&")
, but that implementation is pretty naive and I have some doubts whether it might break some things, e.g. when another logger is used or when a fully qualified name is wrapped in brackets (see the last example in the link above). Any way, I hope this helps in pointing to a more robust fix.
If there's anything I can help with, please let me know!
For reference, I'm running on OSX. If I try and use a relative path for my test assembly I get:
➜ net6.0 git:(feature/DEV-14338) test-rerun ./MyTest.dll -d 1
Microsoft (R) Test Execution Command Line Tool Version 17.4.0 (arm64)
The test source file "/./MyTest.dll" provided was not found.
If I specify a full path I get :
➜ net6.0 git:(feature/DEV-14338) test-rerun /Users/tim/Documents/code/some/full/path/bin/Debug/net6.0/MyTest.dll -d 1
Microsoft (R) Test Execution Command Line Tool Version 17.4.0 (arm64)
Copyright (c) Microsoft Corporation. All rights reserved.
Starting test execution, please wait...
Hi,
Options [-e|--environment <NAME="VALUE">] is not supported. https://learn.microsoft.com/en-us/dotnet/core/tools/dotnet-test.
Can you please add support?
Thanks.
First of all, thank you for creating this tool! I recently started using it to rerun our integration tests in our Azure DevOps pipeline. This suite suffers from some ordering effects. We are trying to resolve that issue, but in the meantime we would like to keep our build green. dotnet-test-rerun seems to be made for that use case!
However, I noticed something does not seem to work as intended -- at least in our pipeline. After each test, test-rerun runs the entire test suite again, not just the failed tests. (That is the intended behaviour, right?)
This is the output I see in the integration test task:
Rerun attempt 3/3
Found Failed tests. Rerun filter: [...failing tests...]
working directory: /
forking test [...path...]/[...application...].IntegrationTests.dll --logger "trx" --results-directory [...directory...]
Now I suspect that I should also see --filter
in that last line of code -- is that correct? I've been investigating the code base to confirm my suspicion, but I haven't found the place where the filter is set just yet.
This is the script we execute in our build task:
dotnet tool install --global dotnet-test-rerun
test-rerun [...path...]/[...application...].IntegrationTests.dll --results-directory ../../_temp
We need to specify the results directory because the task did not have write access at the original location.
So, in short, two questions:
When a theory's subset fails then dotnet-test-rerun
re-runs all test cases of the theory.
when running dotnet test
without a path/project it will search for all projects within the current and childfolders for test projects. This is handy so when new test-projects are added, they are found automatically. Would be awesome if test-rerun [somepathtodll] [OPTIONS]
the somepathtodll
would be optional
Thanks for great piece of software!
Firstly, many thanks for putting this tool together! It's absolutely excellent, and has saved a lot of man hours.
I have a dedicated test automation solution, built around MSTest, that consists of API and UI Tests.
These different categories of tests are organised into their own unit test projects, and are filtered by TestCategory.
For example “TestCategory=API & TestCategory=SmokeTests”, executes the tests from the API unit test project.
I have an issue however when it comes to the re-run mechanism kicking in, when I have similar test names acrosss both unit test projects, if “Test1” exists in both unit test projects, or is contained within the test name in another test project (something like “Test12” is sufficient), and I have a failure, the re-run doesn’t use the fully qualified name of the failed test, and thus tests across both projects, can be executed.
Log output looks like this, hence my thoughts are that it’s not using a fully qualified name:
Found Failed tests. Rerun filter: FullyQualifiedName~Test1
I would have expected it to be something like MyTests.API.Test1
I orchestrate my tests with docker compose, and typically with an API test stack, I do not deploy a web browser, as it is not needed. With the re-try however, if it re-runs a web test, this causes a failure. As a work around, I’ve added the browser to the API test stack, but this isn’t ideal.
Context: assume that we use the tool on an .sln file that contains a lot of test (and, maybe, non-test projects).
Upon execution, tool will run dotnet test
which will search for tests in all the test projects, execute them. If there are failures, the tool will re-run dotnet test
on all the test projects searching for the failed tests, having a performance impact.
Would it be possible, on re-run, to run dotnet test
on the test projects (or even the DLLs) that contain the failed tests?
Thanks again for this great library!
Hi, it's me again 😅
At the moment, when you run dotnet-test-rerun in an Azure pipeline, it will always mark the running of the task as successful, even in case of a testsuite that still contains failing tests after the specified amount of reruns.
Failed! - Failed: 4, Passed: 3, Skipped: 0, Total: 7, Duration: 25 s - Application.IntegrationTests.dll (net6.0)
Finishing: Run integration tests
Is this, in your opinion, intended behaviour?
I would suspect that most users would expect the tool to signal failure in such a case. That would require terminating the process with ExitCode 1 instead of 0, which is what happens now. Would you agree with this assessment?
By the way, I believe the same issue now crops op when an unhandled exception is thrown from the code: it ends with ExitCode 0 and the pipeline simply carries on.
As always, thank you for your great work on maintaining this tool, and also for your patience when handling the host of issues I keep dropping. If there's any more information you need from me, please let me know.
Currently, the DotNet test ADO task allows to run test projects using a glob pattern. For example,
- task: DotNetCoreCLI@2
displayName: Run tests
inputs:
command: 'test'
projects: 'src/**/*.[Tt]est?(s).csproj'
This will recursively search all folders for files matching this pattern. It would be great if the tool supported this.
PS. There is a workaround to add all test .csproj files to a solution (.sln). However, this increases the execution time, since on retry, the tool will try and search for the failed tests in all .csproj projects in the .sln file.
add support for all options:
https://learn.microsoft.com/en-us/dotnet/core/tools/dotnet-test
Executing
test-rerun --logger trx --logger logger2
Returns error
Option '--logger' expects a single argument but 2 were provided.
dotnet test
would allow to provide multiple logger arguments.
example:
--delay <seconds>: Specifies the delay between test runs in seconds. The default is 1000.
For some reason, running this on ADO doesn't do test rerun.
This is the way we are using it (altering some sensitive info from project names and variables):
- task: Bash@3
continueOnError: true
displayName: 'Run functional tests with retries'
inputs:
failOnStderr: false
targetType: 'inline'
script: |
# https://github.com/joaoopereira/dotnet-test-rerun
dotnet tool install --global dotnet-test-rerun --version 1.8.0 --no-cache
project="$(Build.SourcesDirectory)/src/project.sln"
filter="(TestCategory=SomeCategory)"
# build the test solution
dotnet build $project --configuration Debug
# run the tests
test-rerun $project --configuration Debug --loglevel Debug --delay 3 --rerunMaxAttempts 3 --no-build --results-directory $(Agent.TempDirectory) --settings "$(Build.SourcesDirectory)/src/test.runsettings" --logger trx --verbosity detailed --filter=$filter --inlineRunSettings '"TestRunParameters.Parameter(name=\"name\", value=\"value\")"'
Detailed log output:
...
Total tests: 287
Test Run Failed.
Passed: 255
Failed: 29
Skipped: 3
Total time: 21.8892 Minutes
6>VSTest:
MSB4181: The "Microsoft.TestPlatform.Build.Tasks.VSTestTask" task returned false but did not log an error.
Done executing task "Microsoft.TestPlatform.Build.Tasks.VSTestTask" -- FAILED.
6>Done building target "VSTest" in project "project.csproj" -- FAILED.
6>Done Building Project "/mnt/vss/_work/2/s/src/project.csproj" (VSTest target(s)) -- FAILED.
1>Done executing task "MSBuild" -- FAILED.
1>Done building target "VSTest" in project "project.sln" -- FAILED.
1>Done Building Project "/mnt/vss/_work/2/s/src/project.sln" (VSTest target(s)) -- FAILED.
Build FAILED.
0 Warning(s)
0 Error(s)
Time Elapsed 00:21:57.57
Test Run Failed.
Test Run Failed.
Exit code 1.
command:
dotnet test /mnt/vss/_work/2/s/src/project.sln --filter "(TestCategory=SomeCategory)" --settings "/mnt/vss/_work/2/s/src/test.runsettings" --logger "trx" --no-build -c "Debug" -v "Detailed" --results-directory "/mnt/vss/_work/_temp" -- "TestRunParameters.Parameter(name=\"name\", value=\"value\")"
at dotnet.test.rerun.DotNetRunner.DotNetTestRunner.HandleProcessEnd() in /home/runner/work/dotnet-test-rerun/dotnet-test-rerun/src/DotNetRunner/DotNetTestRunner.cs:line 83
at dotnet.test.rerun.DotNetRunner.DotNetTestRunner.Run(String arguments) in /home/runner/work/dotnet-test-rerun/dotnet-test-rerun/src/DotNetRunner/DotNetTestRunner.cs:line 58
at dotnet.test.rerun.DotNetRunner.DotNetTestRunner.Test(RerunCommandConfiguration config, String resultsDirectory) in /home/runner/work/dotnet-test-rerun/dotnet-test-rerun/src/DotNetRunner/DotNetTestRunner.cs:line 38
at dotnet.test.rerun.RerunCommand.RerunCommand.Run() in /home/runner/work/dotnet-test-rerun/dotnet-test-rerun/src/RerunCommand/RerunCommand.cs:line 51
at dotnet.test.rerun.RerunCommand.RerunCommand.<>c__DisplayClass6_0.<<-ctor>b__0>d.MoveNext() in /home/runner/work/dotnet-test-rerun/dotnet-test-rerun/src/RerunCommand/RerunCommand.cs:line 42
--- End of stack trace from previous location ---
at System.CommandLine.Invocation.AnonymousCommandHandler.InvokeAsync(InvocationContext context)
at System.CommandLine.Invocation.InvocationPipeline.<>c__DisplayClass4_0.<<BuildInvocationChain>b__0>d.MoveNext()
--- End of stack trace from previous location ---
at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c__DisplayClass17_0.<<UseParseErrorReporting>b__0>d.MoveNext()
--- End of stack trace from previous location ---
at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c__DisplayClass12_0.<<UseHelp>b__0>d.MoveNext()
--- End of stack trace from previous location ---
at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c__DisplayClass22_0.<<UseVersionOption>b__0>d.MoveNext()
--- End of stack trace from previous location ---
at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c__DisplayClass19_0.<<UseTypoCorrections>b__0>d.MoveNext()
--- End of stack trace from previous location ---
at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c.<<UseSuggestDirective>b__18_0>d.MoveNext()
--- End of stack trace from previous location ---
at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c__DisplayClass16_0.<<UseParseDirective>b__0>d.MoveNext()
--- End of stack trace from previous location ---
at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c.<<RegisterWithDotnetSuggest>b__5_0>d.MoveNext()
--- End of stack trace from previous location ---
at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c__DisplayClass8_0.<<UseExceptionHandler>b__0>d.MoveNext()
##[error]Bash exited with code '1'.
Finishing: Run functional tests with retries
Hi.
I think there is some issue with the rerun logic when tests target multiple frameworks, because it doesn't just rerun the test in the framework it failed in, it reruns them both.
I created a simple test project with this content:
[TestClass]
public class UnitTest1
{
[TestMethod]
public void TestMethod1()
{
#if NET7_0
Assert.IsTrue(true);
#else
Assert.IsTrue(false);
#endif
}
}
and when I run the tests I get this output:
Determining projects to restore...
All projects are up-to-date for restore.
MultiFrameworkTest -> C:\Users\sam.holder\source\repos\MultiFrameworkTest\bin\Debug\net6.0\MultiFrameworkTest.dll
MultiFrameworkTest -> C:\Users\sam.holder\source\repos\MultiFrameworkTest\bin\Debug\net7.0\MultiFrameworkTest.dll
Test run for C:\Users\sam.holder\source\repos\MultiFrameworkTest\bin\Debug\net6.0\MultiFrameworkTest.dll (.NETCoreApp,Version=v6.0)
Microsoft (R) Test Execution Command Line Tool Version 17.6.0 (x64)
Copyright (c) Microsoft Corporation. All rights reserved.Starting test execution, please wait...
A total of 1 test files matched the specified pattern.
Failed TestMethod1 [19 ms]
Error Message:
Assert.IsTrue failed.
Stack Trace:
at MultiFrameworkTest.UnitTest1.TestMethod1() in C:\Users\sam.holder\source\repos\MultiFrameworkTest\UnitTest1.cs:line 12Results File: C:\Users\sam.holder\source\repos\MultiFrameworkTest\sam.holder_CW-LAP-SAM2_2023-10-23_15_28_16.trx
Failed! - Failed: 1, Passed: 0, Skipped: 0, Total: 1, Duration: 30 ms - MultiFrameworkTest.dll (net6.0)
Test run for C:\Users\sam.holder\source\repos\MultiFrameworkTest\bin\Debug\net7.0\MultiFrameworkTest.dll (.NETCoreApp,Version=v7.0)
Microsoft (R) Test Execution Command Line Tool Version 17.6.0 (x64)
Copyright (c) Microsoft Corporation. All rights reserved.Starting test execution, please wait...
A total of 1 test files matched the specified pattern.
Results File: C:\Users\sam.holder\source\repos\MultiFrameworkTest\sam.holder_CW-LAP-SAM2_2023-10-23_15_28_17.trxPassed! - Failed: 0, Passed: 1, Skipped: 0, Total: 1, Duration: 9 ms - MultiFrameworkTest.dll (net7.0)
No tests found with the Outcome Failed in file sam.holder_CW-LAP-SAM2_2023-10-23_15_28_17.trx
Rerun attempt 1/3
Found Failed tests: FullyQualifiedNameMultiFrameworkTest.UnitTest1.TestMethod1MultiFrameworkTest.UnitTest1.TestMethod1
Rerun filter: FullyQualifiedName
Determining projects to restore...
All projects are up-to-date for restore.
MultiFrameworkTest -> C:\Users\sam.holder\source\repos\MultiFrameworkTest\bin\Debug\net6.0\MultiFrameworkTest.dll
MultiFrameworkTest -> C:\Users\sam.holder\source\repos\MultiFrameworkTest\bin\Debug\net7.0\MultiFrameworkTest.dll
Test run for C:\Users\sam.holder\source\repos\MultiFrameworkTest\bin\Debug\net6.0\MultiFrameworkTest.dll (.NETCoreApp,Version=v6.0)
Microsoft (R) Test Execution Command Line Tool Version 17.6.0 (x64)
Copyright (c) Microsoft Corporation. All rights reserved.Starting test execution, please wait...
A total of 1 test files matched the specified pattern.
Failed TestMethod1 [15 ms]
Error Message:
Assert.IsTrue failed.
Stack Trace:
at MultiFrameworkTest.UnitTest1.TestMethod1() in C:\Users\sam.holder\source\repos\MultiFrameworkTest\UnitTest1.cs:line 12Results File: C:\Users\sam.holder\source\repos\MultiFrameworkTest\sam.holder_CW-LAP-SAM2_2023-10-23_15_28_19.trx
Failed! - Failed: 1, Passed: 0, Skipped: 0, Total: 1, Duration: 26 ms - MultiFrameworkTest.dll (net6.0)
Test run for C:\Users\sam.holder\source\repos\MultiFrameworkTest\bin\Debug\net7.0\MultiFrameworkTest.dll (.NETCoreApp,Version=v7.0)
Microsoft (R) Test Execution Command Line Tool Version 17.6.0 (x64)
Copyright (c) Microsoft Corporation. All rights reserved.Starting test execution, please wait...
A total of 1 test files matched the specified pattern.
Results File: C:\Users\sam.holder\source\repos\MultiFrameworkTest\sam.holder_CW-LAP-SAM2_2023-10-23_15_28_20.trxPassed! - Failed: 0, Passed: 1, Skipped: 0, Total: 1, Duration: 9 ms - MultiFrameworkTest.dll (net7.0)
No tests found with the Outcome Failed in file sam.holder_CW-LAP-SAM2_2023-10-23_15_28_20.trx
Rerun attempt 2/3
Found Failed tests: FullyQualifiedNameMultiFrameworkTest.UnitTest1.TestMethod1MultiFrameworkTest.UnitTest1.TestMethod1
Rerun filter: FullyQualifiedName
Determining projects to restore...
All projects are up-to-date for restore.
MultiFrameworkTest -> C:\Users\sam.holder\source\repos\MultiFrameworkTest\bin\Debug\net6.0\MultiFrameworkTest.dll
MultiFrameworkTest -> C:\Users\sam.holder\source\repos\MultiFrameworkTest\bin\Debug\net7.0\MultiFrameworkTest.dll
Test run for C:\Users\sam.holder\source\repos\MultiFrameworkTest\bin\Debug\net6.0\MultiFrameworkTest.dll (.NETCoreApp,Version=v6.0)
Microsoft (R) Test Execution Command Line Tool Version 17.6.0 (x64)
Copyright (c) Microsoft Corporation. All rights reserved.Starting test execution, please wait...
A total of 1 test files matched the specified pattern.
Failed TestMethod1 [15 ms]
Error Message:
Assert.IsTrue failed.
Stack Trace:
at MultiFrameworkTest.UnitTest1.TestMethod1() in C:\Users\sam.holder\source\repos\MultiFrameworkTest\UnitTest1.cs:line 12Results File: C:\Users\sam.holder\source\repos\MultiFrameworkTest\sam.holder_CW-LAP-SAM2_2023-10-23_15_28_23.trx
Failed! - Failed: 1, Passed: 0, Skipped: 0, Total: 1, Duration: 28 ms - MultiFrameworkTest.dll (net6.0)
Test run for C:\Users\sam.holder\source\repos\MultiFrameworkTest\bin\Debug\net7.0\MultiFrameworkTest.dll (.NETCoreApp,Version=v7.0)
Microsoft (R) Test Execution Command Line Tool Version 17.6.0 (x64)
Copyright (c) Microsoft Corporation. All rights reserved.Starting test execution, please wait...
A total of 1 test files matched the specified pattern.
Results File: C:\Users\sam.holder\source\repos\MultiFrameworkTest\sam.holder_CW-LAP-SAM2_2023-10-23_15_28_24.trxPassed! - Failed: 0, Passed: 1, Skipped: 0, Total: 1, Duration: 9 ms - MultiFrameworkTest.dll (net7.0)
No tests found with the Outcome Failed in file sam.holder_CW-LAP-SAM2_2023-10-23_15_28_24.trx
Rerun attempt 3/3
Found Failed tests: FullyQualifiedNameMultiFrameworkTest.UnitTest1.TestMethod1MultiFrameworkTest.UnitTest1.TestMethod1
Rerun filter: FullyQualifiedName
Determining projects to restore...
All projects are up-to-date for restore.
MultiFrameworkTest -> C:\Users\sam.holder\source\repos\MultiFrameworkTest\bin\Debug\net6.0\MultiFrameworkTest.dll
MultiFrameworkTest -> C:\Users\sam.holder\source\repos\MultiFrameworkTest\bin\Debug\net7.0\MultiFrameworkTest.dll
Test run for C:\Users\sam.holder\source\repos\MultiFrameworkTest\bin\Debug\net6.0\MultiFrameworkTest.dll (.NETCoreApp,Version=v6.0)
Microsoft (R) Test Execution Command Line Tool Version 17.6.0 (x64)
Copyright (c) Microsoft Corporation. All rights reserved.Starting test execution, please wait...
A total of 1 test files matched the specified pattern.
Failed TestMethod1 [14 ms]
Error Message:
Assert.IsTrue failed.
Stack Trace:
at MultiFrameworkTest.UnitTest1.TestMethod1() in C:\Users\sam.holder\source\repos\MultiFrameworkTest\UnitTest1.cs:line 12Results File: C:\Users\sam.holder\source\repos\MultiFrameworkTest\sam.holder_CW-LAP-SAM2_2023-10-23_15_28_26.trx
Failed! - Failed: 1, Passed: 0, Skipped: 0, Total: 1, Duration: 25 ms - MultiFrameworkTest.dll (net6.0)
Test run for C:\Users\sam.holder\source\repos\MultiFrameworkTest\bin\Debug\net7.0\MultiFrameworkTest.dll (.NETCoreApp,Version=v7.0)
Microsoft (R) Test Execution Command Line Tool Version 17.6.0 (x64)
Copyright (c) Microsoft Corporation. All rights reserved.Starting test execution, please wait...
A total of 1 test files matched the specified pattern.
Results File: C:\Users\sam.holder\source\repos\MultiFrameworkTest\sam.holder_CW-LAP-SAM2_2023-10-23_15_28_27.trxPassed! - Failed: 0, Passed: 1, Skipped: 0, Total: 1, Duration: 9 ms - MultiFrameworkTest.dll (net7.0)
As you can see even though the test only fails in .net 6.0 its rerun in 7.0 every time as well.
I think you can provide the --framework
option to dotnet test
so maybe we should parse the framework out of the path for the test results in the trx and use that to rerun only the failing tests
I am trying to run this dotnet test command using this wrapper command but it seems some options doesn't work especially code coverage
dotnet test C:\a\1\s\UnitTests\UnitTests.csproj --logger trx --results-directory C:\a_temp --configuration Release --no-build --verbosity minimal --filter (TestCategory!=DataWarehouse&TestCategory!=Integration) /p:CollectCoverage=true /p:CoverletOutputFormat=opencover /p:CoverletOutput=C:\a_temp\TestResults\Coverage\UnitTests-TestResults.opencover.xml
what is the equivalent command for above using test-rerun. Thanks.
at the moment it requires a project, but dotnet test
does not, it will run also at root level of your solution, even if your tests are hidden in test
folder. this would be desireable also for this tool
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.