csoltenborn / googletestadapter Goto Github PK
View Code? Open in Web Editor NEWVisual studio extension that adds support for the C++ testing framework Google Test.
License: Other
Visual studio extension that adds support for the C++ testing framework Google Test.
License: Other
Currently, we do not allow to filter test cases via VsTest.Console.exe since we haven't implemented TestCaseFilters (as described here).
Since VS update 2, executing our test suite of around 5000 tests takes much longer than before. The problem seems to be at two places:
while (!process.StandardOutput.EndOfStream)
of file Core/Helpers/ProcessLauncher.cs
for that time (and afterwards continues normally)TestEnvironment.LogInfo("Test execution completed.")
of file Core/GoogleTestExecutor.cs
has been executed), it takes 15-20 minutes until test results are finally displayed in VS's test explorer, and during this time, VS is rather busy (as Windows' task manager shows).We can reproduce this behavior on VS 2015 Enterprise Edition update 2, and we can see that it does not happen with update 1.
Running the installer for 0.3.0 (or 0.2.3) fails on a Windows 10 system with Visual Studio 2013 with the installation log below. Changing the line
<Dependency Id="Microsoft.Framework.NDP" DisplayName="Microsoft .NET Framework" Version="4.5" />
to
<Dependency Id="Microsoft.Framework.NDP" DisplayName="Microsoft .NET Framework" Version="[4.5,4.6]" />
in the extension.vsixmanifest file in the .vsix package fixes the issue
Presumably this is the same as changing the corresponding line in the GoogleTestAdapter\VsPackage\source.extension.vsixmanifest file of the VsPackage project (or some settings in the project), couldn't test that though because I have VS 2015 not installed.
24/02/2016 11:31:26 - Microsoft VSIX Installer
24/02/2016 11:31:26 - -------------------------------------------
24/02/2016 11:31:27 - Initializing Install...
24/02/2016 11:31:27 - Extension Details...
24/02/2016 11:31:27 - Identifier : GoogleTestAdapterVSIX.e57b9720-dfde-4baa-bcb5-0f75772012b4
24/02/2016 11:31:27 - Name : Google Test Adapter
24/02/2016 11:31:27 - Author : Christian Soltenborn, Jonas Gefele
24/02/2016 11:31:27 - Version : 0.2.3.203
24/02/2016 11:31:27 - Description : Adds support for the C++ unit testing framework Google Tests.
24/02/2016 11:31:27 - Locale : en-US
24/02/2016 11:31:27 - MoreInfoURL : https://github.com/csoltenborn/GoogleTestAdapter
24/02/2016 11:31:27 - InstalledByMSI : False
24/02/2016 11:31:27 - SupportedFrameworkVersionRange : [4.5]
24/02/2016 11:31:27 -
24/02/2016 11:31:27 - Supported Products :
24/02/2016 11:31:27 - Microsoft.VisualStudio.Community
24/02/2016 11:31:27 - Version : [12.0,14.0]
24/02/2016 11:31:27 - Microsoft.VisualStudio.Pro
24/02/2016 11:31:27 - Version : [12.0,14.0]
24/02/2016 11:31:27 - Microsoft.VisualStudio.Enterprise
24/02/2016 11:31:27 - Version : [12.0,14.0]
24/02/2016 11:31:27 - Microsoft.VisualStudio.Ultimate
24/02/2016 11:31:27 - Version : [12.0,14.0]
24/02/2016 11:31:27 -
24/02/2016 11:31:27 - References :
24/02/2016 11:31:27 - -------------------------------------------------------
24/02/2016 11:31:27 - Identifier : Microsoft.VisualStudio.MPF.11.0
24/02/2016 11:31:27 - Name : Visual Studio MPF 11.0
24/02/2016 11:31:27 - Version : [11.0]
24/02/2016 11:31:27 - MoreInfoURL :
24/02/2016 11:31:27 - Nested : No
24/02/2016 11:31:27 -
24/02/2016 11:31:27 -
24/02/2016 11:31:27 - Searching for applicable products...
24/02/2016 11:31:27 - Found installed product - Microsoft Visual Studio Community 2013
24/02/2016 11:31:27 - The extension with ID 'GoogleTestAdapterVSIX.e57b9720-dfde-4baa-bcb5-0f75772012b4' is not installed to Microsoft Visual Studio Community 2013.
24/02/2016 11:31:27 - Found installed product - Microsoft Visual Studio 2013 Shell (Integrated)
24/02/2016 11:31:27 - Found installed product - Global Location
24/02/2016 11:31:29 - The following target products have been selected...
24/02/2016 11:31:29 - Microsoft Visual Studio Community 2013
24/02/2016 11:31:29 -
24/02/2016 11:31:29 - Beginning to install extension to Microsoft Visual Studio Community 2013...
24/02/2016 11:31:29 - Install Error : Microsoft.VisualStudio.ExtensionManager.MissingTargetFrameworkException: The extension 'Google Test Adapter' requires a version of the .NET Framework that is not installed.
at Microsoft.VisualStudio.ExtensionManager.ExtensionManagerService.CheckForValidDotNetFramework(IExtension extension)
at Microsoft.VisualStudio.ExtensionManager.ExtensionManagerService.CheckForInstallBlockers(InstallableExtensionImpl extension, IInstalledExtensionList modifiedInstalledExtensionsList, Boolean isNestedExtension, Boolean& olderVersionInstalled)
at Microsoft.VisualStudio.ExtensionManager.ExtensionManagerService.InstallInternal(InstallableExtensionImpl extension, Boolean perMachine, Boolean isNestedExtension, IDictionary`2 extensionsInstalledSoFar, List`1 extensionsUninstalledSoFar, IInstalledExtensionList modifiedInstalledExtensionsList, AsyncOperation asyncOp, UInt64 totalBytesToWrite, UInt64& totalBytesWritten)
at Microsoft.VisualStudio.ExtensionManager.ExtensionManagerService.BeginInstall(IInstallableExtension installableExtension, Boolean perMachine, AsyncOperation asyncOp)
at Microsoft.VisualStudio.ExtensionManager.ExtensionManagerService.InstallWorker(IInstallableExtension extension, Boolean perMachine, AsyncOperation asyncOp)
After updating through VS and then downloading the .visx from here directly, I get the following error on startup/settings page. It would appear the VS2013 installation is built with references to the VS2015 assemblies? 2013 is v12, and I believe 2015 is v14, right?
<entry> <record>939</record> <time>2016/03/26 20:47:28.608</time> <type>Error</type> <source>VisualStudio</source> <description>SetSite failed for package [GoogleTestExtensionOptionsPage]</description> <guid>{E7C90FCB-0943-4908-9AE8-3B6A9D22EC9E}</guid> <hr>80070002</hr> <errorinfo>Could not load file or assembly 'Microsoft.VisualStudio.ComponentModelHost, Version=14.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified.</errorinfo> </entry> <entry> <record>940</record> <time>2016/03/26 20:47:28.608</time> <type>Error</type> <source>VisualStudio</source> <description>End package load [GoogleTestExtensionOptionsPage]</description> <guid>{E7C90FCB-0943-4908-9AE8-3B6A9D22EC9E}</guid> <hr>80070002</hr> <errorinfo>Could not load file or assembly 'Microsoft.VisualStudio.ComponentModelHost, Version=14.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified.</errorinfo> </entry>
Providing variadic arguments will allow users to use an arbitrary number of arguments, and will also greatly reduce code (i.e., macro) duplication
Currently, we are providing the following placeholders for use in the Additional test execution params, test setup batch, and test teardown batch options: Test directory, thread id, and solution dir. Provide more placeholders, e.g.
We just started using google test a few weeks back. We'd been working with the command line output only. We found this extension and it seems to work correctly, but it only finds tests in 3 of the 5 exes I have implementing tests.
I started going through GoogleTestDiscoverer.cs to see if I can figure out what's wrong, but it takes the entire binary exe file as an argument? Can someone help me figure out how to start debugging this?
Thanks.
Symptoms: When running tests, GTA reports that n tests have been executed, but VS shows less test results in the test explorer. When we insert a small delay between between reporting individual test case results all tests are reported.
Reason: We suspect this is due to inter process communication, but we do not have a lot of expertise in this area, and there is no official documentation of the VS test framework API
Workaround: Configure Report waiting time at Tools/Options/Google Test Adapter/Advanced
Not having a well defined software license discourages contributions.
There is a license on the VS Gallery page, and it can be found within the repository as well, but it would be beneficial if the license text would appear in the standard location (a file named LICENSE) and with the placeholder snippets replaced with actual values.
With certain optimizations, the compiler optimizes away the static methods containing trait information. Provide an alternative implementation which does not get lost during optimization.
I'm new to GTA and GoogleTest and have just set up the add-in with my VS2015. Tests are discovered and visible in the Test Explorer.
But as I build the project, test execution is started but never stops.
I've debug enabled in GTA > General and see following in the test execution console.
------ Discover test started ------
C:\Dev\GTTest1\Debug\GTTest.exe matches regex '[Tt]est[s]?.exe'
And I've to kill the process exe from the Task Manager.
I've only one CPP file for test that looks like following:
`
using namespace std;
int Factorial(int n) {
if (n > 0) {
return n*Factorial(n - 1);
}
return 1;
}
int main(int argc, char **argv) {
::testing::InitGoogleTest(&argc, argv);
int testResult = RUN_ALL_TESTS();
getchar();
return testResult;
}
// Tests factorial of 0.
TEST(FactorialTest, HandlesZeroInput) {
EXPECT_EQ(1, Factorial(0));
}
// Tests factorial of positive numbers.
TEST(FactorialTest, HandlesPositiveInput) {
EXPECT_EQ(1, Factorial(1));
EXPECT_EQ(2, Factorial(2));
EXPECT_EQ(24, Factorial(4));
EXPECT_EQ(40320, Factorial(8));
}
`
There are some rare issues with VS2015 update 1 and DIA.
Christian, can you provide the error message please?
VS2012 update 4 is already tested.
VS2012 update 1-3 work run, too.
VS2012 w/o update doesn't work due to missing traits support.
There is an Azure free trial giving $200 credit for 1 month and the Visual Studio Dev Essentials program giving $25/month credits for 12 month.
$25 suffice for 166 hours of Basic A2 instance (2 cores, 3.5GB RAM, 60GB HDD) or 83 hours of Basic A3 instance (4 cores, 7GB RAM, 120GB HDD). See the pricing calculator.
As discussed in #22, in the Test Explorer window if VS can't fit the text after a "/" (that's then terminated by another "/" or the EOS) then it shortens everything after that "/" with "...". Perhaps a replacement string could be provided in GTA's settings? Eg, instead of "/" a user could specify "->" or something else.
Here are truncated instantiated test cases:
And here they are when the window is expanded to fit the next segment:
This makes it appear as though filters are working even though they really aren't.
Here is the fix:
There is a check (AllTestCasesOfExecutableAreRun()) in CommandLineGenerator.cs if the testCases match all test cases. If they do you don't add any filters. I found this because my tests were taking a long time even when I was executing a few. It turned out all test cases were being run even though a filter was passed in as an argument to vstest.console.exe.
In TestExecutor.cs
public void RunTests(IEnumerable<string> executables, IRunContext runContext, IFrameworkHandle frameworkHandle)
{
try
{
InitTestEnvironment(runContext.RunSettings, frameworkHandle);
IList<Model.TestCase> allTestCasesInExecutables = GetAllTestCasesInExecutables(executables).ToList();
ISet<string> allTraitNames = GetAllTraitNames(allTestCasesInExecutables);
TestCaseFilter filter = new TestCaseFilter(runContext, allTraitNames, TestEnvironment);
List<TestCase> vsTestCases =
filter.Filter(allTestCasesInExecutables.Select(DataConversionExtensions.ToVsTestCase)).ToList();
>>>>>>>>>>>> allTestCasesInExecutables =
>>>>>>>>>>>>>>>>>> allTestCasesInExecutables.Where(tc => vsTestCases.Any(vtc => >>>>>>>>>>>>>>>>>> tc.FullyQualifiedName == vtc.FullyQualifiedName)).ToList();
>>>>>>>>>>>>>>>>>>> DoRunTests(allTestCasesInExecutables, allTestCasesInExecutables, runContext, frameworkHandle);
Change to:
>>>>>>>>>>>> var testCasesToRun =
>>>>>>>>>>>>>>>>>> allTestCasesInExecutables.Where(tc => vsTestCases.Any(vtc => >>>>>>>>>>>>>>>>>> tc.FullyQualifiedName == vtc.FullyQualifiedName)).ToList();
>>>>>>>>>>>>>>>>>>> DoRunTests(allTestCasesInExecutables, testCasesToRun, runContext, frameworkHandle);
}
catch (Exception e)
{
TestEnvironment.LogError("Exception while running tests: " + e);
}
}
Symptoms: Selecting the Run tests after build option does not or not always result in the tests being run after build completion. Instead, the following error message is printed to the test console: "The specified type member 'Stale' is not supported in LINQ to Entities. Only initializers, entity members, and entity navigation properties are supported."
Reason: We have no idea
Workaround: None so far
On my machine, the UI tests always show the "Clean instance?" dialog, even if the last test run was fine (and cleanup should have been performed).
Hi there
I have disabled break on failure and enabled catch exceptions in the options but when I invoke "Run selected test" the options are not used.
Executing command 'base_test.exe --gtest_output="xml:C:\Users\thomas.roller\AppData\Local\Temp\tmp7548.tmp" --gtest_catch_exceptions=0 --gtest_break_on_failure=1 --gtest_filter=TestCOPManagerIterator.BackwardBasics'
Am I doing something wrong?
We should offer a checkbox for not showing update info again.
Are we only displaying that document if there were indeed breaking changes? This would of course imply that we need to know the formerly installed version...
Currently, if test execution is canceled, one still has to wait for the running test processes to finish or to kill these processes using the Windows TaskManager. Implement support for actively killing these processes if test execution is canceled.
Looked at the following, hoping it would solve this issue:
http://stackoverflow.com/questions/596142/would-you-like-to-continue-and-run-the-last-successful-build
However, it seems that "Run test" doesn't honor this setting. Is this something inherent to VS or is this something that can be changed in the extension? Thanks!
Currently, traits can only be assigned to typed tests via options. We need to provide enhanced GTest macros as for simple and parameterized tests.
We don't find source location for typed tests and type-parameterized tests.
class MyStrangeArray : public std::array<int, 3>
{
public: MyStrangeArray(std::initializer_list<int> init) : array({ 3,2,1 }) {}
};
template< typename int_container_type >
class TypedTests : public ::testing::Test {
public:
typename int_container_type container{ 1,2,3 };
};
typedef ::testing::Types<std::vector<int>, std::array<int,3>, MyStrangeArray> IntContainerTypes;
TYPED_TEST_CASE(TypedTests, IntContainerTypes);
TYPED_TEST(TypedTests, CanIterate) {
int sum = 0;
for (int value : this->container)
sum += value;
EXPECT_EQ(1 + 2 + 3, sum);
}
TYPED_TEST(TypedTests, CanDefeatMath) {
EXPECT_NE(this->container[0] + this->container[1], this->container[2]);
}
template< typename int_container_type >
class TypeParameterizedTests : public TypedTests<int_container_type> {};
TYPED_TEST_CASE_P(TypeParameterizedTests);
TYPED_TEST_P(TypeParameterizedTests, CanIterate) {
int sum = 0;
for (int value : this->container)
sum += value;
EXPECT_EQ(1 + 2 + 3, sum);
}
TYPED_TEST_P(TypeParameterizedTests, CanDefeatMath) {
EXPECT_NE(this->container[0] + this->container[1], this->container[2]);
}
REGISTER_TYPED_TEST_CASE_P(TypeParameterizedTests, CanIterate, CanDefeatMath);
typedef ::testing::Types<std::array<int, 3>, MyStrangeArray> IntArrayTypes;
INSTANTIATE_TYPED_TEST_CASE_P(Vec, TypeParameterizedTests, std::vector<int>);
INSTANTIATE_TYPED_TEST_CASE_P(Arr, TypeParameterizedTests, IntArrayTypes);
Using the Run All button in Test Explorer fails to execute tests with the following stack in Test Output. Highlighting all tests in Test Explorer, right-clicking and using "Run Selected Tests" successfully executes all tests.
Visual Studio 2015 Community Edition, Update 2.
ERROR: Exception while running tests: System.NotImplementedException: The method or operation is not implemented.
at Dia.IDiaSymbol.findChildren(SymTagEnum symTag, String name, UInt32 compareFlags, IDiaEnumSymbols& ppResult)
at GoogleTestAdapter.DiaResolver.DiaResolver.FindFunctionsByRegex(String pattern)
at GoogleTestAdapter.DiaResolver.DiaResolver.GetFunctions(String symbolFilterString)
at GoogleTestAdapter.TestCases.TestCaseResolver.FindTestCaseLocationsInBinary(String binary, List`1 testMethodSignatures, String symbolFilterString, String pathExtension, List`1 errorMessages)
at GoogleTestAdapter.TestCases.TestCaseResolver.ResolveAllTestCases(String executable, List`1 testMethodSignatures, String symbolFilterString, String pathExtension, List`1 errorMessages)
at GoogleTestAdapter.TestCases.TestCaseFactory.GetTestCaseLocations(IList`1 testCaseDescriptors, String pathExtension)
at GoogleTestAdapter.TestCases.TestCaseFactory.CreateTestCases()
at GoogleTestAdapter.GoogleTestDiscoverer.GetTestsFromExecutable(String executable)
at GoogleTestAdapter.TestAdapter.TestExecutor.GetAllTestCasesInExecutables(IEnumerable`1 executables)
at GoogleTestAdapter.TestAdapter.TestExecutor.RunTests(IEnumerable`1 executables, IRunContext runContext, IFrameworkHandle frameworkHandle)
First of all, thanks for making GTA! And thanks in advance for improving it :o)
Now for the bugs:
The project I'm testing was the first thing I opened in VS after getting GTA. I immediately did "Run All Tests" and got output like the above. I then tried doing "Run Failed Tests" and still got the incomplete results (screenshot was from immediately after this action). VS Version/extension info follows:
Microsoft Visual Studio Premium 2013
Version 12.0.40629.00 Update 5
Microsoft .NET Framework
Version 4.6.00081
Installed Version: Premium
Team Explorer for Visual Studio 2013 06191-004-0456033-02040
Microsoft Team Explorer for Visual Studio 2013
Visual Basic 2013 06191-004-0456033-02040
Microsoft Visual Basic 2013
Visual C# 2013 06191-004-0456033-02040
Microsoft Visual C# 2013
Visual C++ 2013 06191-004-0456033-02040
Microsoft Visual C++ 2013
Visual F# 2013 06191-004-0456033-02040
Microsoft Visual F# 2013
CodeContractsHelper_Package 1.0
A package for the CodeContractsHelper extension.
Common Azure Tools 1.4
Provides common services for use by Azure Mobile Services and Microsoft Azure Tools.
GoogleTestExtensionOptionsPage Extension 1.0
GoogleTestExtensionOptionsPage Visual Studio Extension Detailed Info
JetBrains ReSharper Ultimate 2015.1.3 Build 102.0.20150724.141603
JetBrains ReSharper Ultimate package for Microsoft Visual Studio. For more information about ReSharper Ultimate, visit http://www.jetbrains.com/resharper. Copyright © 2016 JetBrains, Inc.
MicroProfiler 1.1.591.0
Realtime Instrumenting Profiler
Please, submit your questions, issues and suggestions to http://code.google.com/p/micro-profiler/issues/list
Copyright (c) 2011-2015 Artem Gevorkyan
Microsoft Azure Mobile Services Tools 1.4
Microsoft Azure Mobile Services Tools
NuGet Package Manager 2.8.60723.765
NuGet Package Manager in Visual Studio. For more information about NuGet, visit http://docs.nuget.org/.
P4VS - Perforce Plugin for Visual Studio 2015.2.124.8831
P4VS is the Perforce integration for Microsoft Visual Studio. P4VS gives developers working in Visual Studio access to Perforce version control features. P4VS gives you easy access to all of the Perforce version control tools and information you need to do your daily work. Check out, check in, see history, and more, directly from Visual Studio. P4VS is a fully compliant Visual Studio Integration Package, designed for full compatibility and ease of use. P4VS is built on P4API.NET, Perforce's fully supported API for the .NET environment. Perforce Software provides fast, robust version control to teams of all sizes working on any type of digital asset development.
PreEmptive Analytics Visualizer 1.2
Microsoft Visual Studio extension to visualize aggregated summaries from the PreEmptive Analytics product.
PVS-Studio 5.31.14891.5
PVS-Studio is a static code analyzer for C/C++ (Microsoft Visual Studio) with a simple licensing and pricing policies which is easy to install and use without a need to deploy complex maintenance environment.
Python Tools for Visual Studio 2.2.31124.00
Python Tools for Visual Studio provides IntelliSense, projects, templates, Interactive windows, and other support for Python developers.
Python Tools for Visual Studio - Profiling Support 2.2.31124.00
Profiling support for Python projects.
RegexTester 1.0
This is regular extension tester.
Release Management for Visual Studio Package 1.0
Release Management for Visual Studio
SyncfusionAssemblyReference 1.0
Information about my package
SyncfusionItemTemplates 1.0
Information about my package
Visual Assist
For more information about Visual Assist, see the Whole Tomato Software website at http://www.WholeTomato.com. Copyright (c) 1997-2015 Whole Tomato Software, Inc.
Visual Studio Tools for Unity 2.1.0.0
Visual Studio Tools for Unity
Windows Phone 8.1 SDK Integration 1.0
This package integrates the tools for the Windows Phone 8.1 SDK into the menus and controls of Visual Studio.
Open a Visual Studio Developer Command line and run
vstest.console.exe /ListDiscoverers /UseVsixExtensions:true
You will not see GoogleTestAdapter in the output. Other 3rd-party test runners like Chutzpah show up.
I have two solutions, one where all gtests are following the same rule, with 10 gtests project, it works perfectly.
The second solution is a mix of mess, with gtests initializing web servers, making them crash during test discovery with lots of popups, but I can't disable them easily. So if you could associate the parameters of GTA with a sln, it would be perfect for me ( maybe by loading a specific file for config )
$(SolutionDir)\gta.config ?
Parameter values should be parsed from test labels, and the values should be displayed on the results page of the according tests.
Implement integration with the settings support of the VS test framework. This would allow developers to share their test settings via source control.
See e.g. #42...
When starting an experimental VS instance with F5 (or Ctrl-F5) and activating DevelopmentMode in the options of the experimental instance, the dialog which allows to attach to the discovery process is shown twice for some reason.
After the first run there will be no more test output.
All runs have test output.
Probably this is because IMessageLogger is being reused between multiple calls into GTA.
This problem is more important for VS2013 because here the execution engine is being reused by default for x86.
I just found this adapter and like it's configurability very much so I was wondering if the subject can be done with this adapter...
When I use the cppunittestframework and have a failure there is:
Source: link to the test and
StackTrace: link to the failed check
Could this be done with the adapter so that we would have
Test Name: TestMath.AddFails
Test FullName: TestMath.AddFails
Test Source: c:\users\thomas.roller\documents\work\git\win\misc\googletestadapter\sampletests\tests\basictests.cpp : line 29
Test Outcome: Failed
Test Duration: 0:00:00,002
Result StackTrace: at Test_TestMath::AddFails() in c:\users\thomas.roller\documents\work\git\win\misc\googletestadapter\sampletests\tests\basictests.cpp : line 30
Result Message:
Value of: Add(10, 10)
Actual: 20
Expected: 1000
instead of
Test Name: TestMath.AddFails
Test FullName: TestMath.AddFails
Test Source: c:\users\thomas.roller\documents\work\git\win\misc\googletestadapter\sampletests\tests\basictests.cpp : line 29
Test Outcome: Failed
Test Duration: 0:00:00,002Result Message:
c:\users\thomas.roller\documents\work\git\win\misc\googletestadapter\sampletests\tests\basictests.cpp:30
Value of: Add(10, 10)
Actual: 20
Expected: 1000
Has someone already tried to do this?
Best Regards
Thomas
Hi!
I detecting a problem with typed tests. They are not discovered
Output:
Found 306 tests in executable C:\Source\zenon\Supervisor\main\BIN\PC\UTests\PROGRAMFILES32\DebugNoUTest\zenon32misc_net_UTest.exe
Warning: Could not find source location for test Typed/fixCCommandReactionsForNetModule/0.Process_ForwardsCallToConcreteImplementation_Yes
Warning: Could not find source location for test Typed/fixCCommandReactionsForNetModule/1.Process_ForwardsCallToConcreteImplementation_Yes
Warning: Could not find source location for test CNetworkNetModuleCommand_fix/0.AktionAnstossen_NewCommand_HandlerCalled
Warning: Could not find source location for test CNetworkNetModuleCommand_fix/0.AktionAnstossen_WithOutCommandInstance_NoCrash
Warning: Could not find source location for test CNetworkNetModuleCommand_fix/0.Serialize_HappyPath_SameAsWritten
Warning: Could not find source location for test CNetworkNetModuleCommand_fix/0.Serialize_WithOutCommandInstance_NoCrash
Warning: Could not find source location for test CNetworkNetModuleCommand_fix/0.Info_HappyPath_Filled
Warning: Could not find source location for test CNetworkNetModuleCommand_fix/0.Info_WithOutCommandInstance_NoCrash
And so on
At least 262 test cases are detected and executed only 250!?
Output:
Many other …
Warning: Could not find source location for test CNetworkNetModuleCommand_fix/5.AddLogEntry_HappyPath_Filled
Running 250 tests...
Test execution completed.
Thanks for any suggestion!
Alfred
Nunit Test Adapter can be called by vstest.console.exe either via the /TestAdapterPath
switch or by pointing to their dll's in the .runsetttings file :
<RunConfiguration>
<!-- Path to Test Adapters -->
<TestAdaptersPaths>c:\path\adapter</TestAdaptersPaths>
</RunConfiguration>
This allows running NUnit tests like regular MSTests without customizing the TFS server configuration and uploading the test adapter to the TFS controller.
GoogleTestAdapter is for some reason not correctly recognized as an adapter when supplied like that, and thus can't be called from TFS without writing custom scripts, to call it correctly, and then publish the test results (as well as coverage info).
I'd be willing to help research and patch it, if you're willing to spare some time for guidance.
TFS was complaining about missing test names.
I traced it down to what appears to be an omission in XmlTestResultParses.cs:91
private TestResult ParseTestResult(XmlNode testcaseNode)
{
string className = testcaseNode.Attributes["classname"].InnerText;
string testCaseName = testcaseNode.Attributes["name"].InnerText;
string qualifiedName = className + "." + testCaseName;
[...]
TestResult testResult = new TestResult(testCase)
{
ComputerName = Environment.MachineName,
91: DisplayName = " "; // shouldn't this be DisplayName = qualifiedName ?
};
Is this a mistake or is there some reason for it?
Hi!
A small issue I have detecting if you group your tests by project and typed tests are used where the types parameter is a template.
If you modify the types parameter of the macro TYPED_TEST_CASE to be a non-template it will work.
The output of the test detection reports about they a warning:
Warning: Could not find source location for test CNetworkNetModuleCommand_fix/0.AktionAnstossen_NewCommand_HandlerCalled
Attached a file to reproduce the behavior.
TypedTest2nd_test.txt
I don't know if this is the test runner or something worse, but the "Discover test" portion of the test runner is taking a very long time. I'm trying to run one single test. This test discovers (using --gtest_list_tests) and executes instantly on the command line.
This is a copy/paste of a fast discovery "Tests" output:
------ Discover test started ------
Found 1 tests in executable D:\path\to\my\test.exe
========== Discover test finished: 1 found (0:00:20.5250523) ==========
------ Run test started ------
Found 1 tests in executable D:\path\to\my\test.exe
Running 1 tests...
Executing tests on 1 threads
Test execution completed.
========== Run test finished: 1 run (0:00:07.5687568) ==========
Discovery takes between 20 and 37 seconds. Execution takes between 2 and 20 seconds. I don't really have any idea what to do at this point, but this is killing my ability to do TDD. The guy sitting right next to me discovers a single test in <1 second.
I looked at process monitor and it appears NotifyChangeDirectory can take up to 30 seconds sometimes. I've been at this 3 hours and I can't seem to figure it out. The crazy thing is it was working fine up until 3 hours ago.
I've tried everything I can think of to no avail:
Have you ever run into this issue?
Some options should be (also) changeable via a toolbar. We should support at least these options:
The following ones are also candidates:
Sometimes parallel test execution takes longer than expected. On reason might be that tests are distributed to the different threads in a sub optimal way (note that this is expected to happen before all tests have been run at least once). To debug such issues (or to find out that instead, the executed tests themselves have unpredictable durations), print out the duration of each executing thread if in debug mode.
Initial display of release notes showed an (nearly) empty dialog.
The bug is that all of the tests suites are specified in the filter on the command line if the whole project is run or even run all is selected.
This occurs because there is a bug in the CommandLineGenerator.AllTestCasesOfExecutableAreRun function. It assumes that the AllTestCases variable only contains tests for that executable, but it actually contains all of the tests for all executables to be run.
To reproduce:
If running all the tests in an executable there should be no command line parameters.
If we use SCOPED_TRACE we have an additional line for the scope in the error message.
It would be great if this could be integrated into the StackTrace formatting.
I added some senseless examples just to have some examples:
void checkValues( std::vector const& expectedValues, std::vector const& resultValues )
{
ASSERT_EQ( expectedValues.size(), resultValues.size() );
for ( size_t i=0; i<expectedValues.size(); ++i )
{
SCOPED_TRACE( "Checking index " + std::to_string( i ) );
EXPECT_EQ( expectedValues.at(i), resultValues.at(i) );
}// for
}
TEST( TestMath, ScopedFailures )
{
std::vector expected;
expected.push_back( 1 );
expected.push_back( 2 );
expected.push_back( 3 );
expected.push_back( 4 );
expected.push_back( 5 );
std::vector results;
results.push_back( 1 );
results.push_back( 1 );
results.push_back( 3 );
results.push_back( 3 );
results.push_back( 3 );
checkValues( expected, results );
}
TEST( TestMath, ScopedFailures2 )
{
std::vector expected;
expected.push_back( 1 );
expected.push_back( 2 );
expected.push_back( 3 );
expected.push_back( 4 );
expected.push_back( 5 );
std::vector results;
results.push_back( 1 );
results.push_back( 1 );
results.push_back( 3 );
results.push_back( 3 );
results.push_back( 3 );
std::vector results2;
results2.push_back( 1 );
results2.push_back( 2 );
results2.push_back( 3 );
results2.push_back( 5 );
results2.push_back( 6 );
{
SCOPED_TRACE( "Check first results");
checkValues( expected, results );
}
{
SCOPED_TRACE( "Check second results");
checkValues( expected, results2 );
}
}
::testing::AssertionResult CheckVectors( std::vector const& expectedValues, std::vector const& resultValues )
{
::testing::AssertionResult result = ::testing::AssertionFailure();
bool failure = false;
if ( expectedValues.size() != resultValues.size() ) {
failure = true;
result << "Vector size does not match: " << std::to_string( expectedValues.size() ) << " != " << std::to_string( resultValues.size() );
}
else {
size_t index = 0;
for ( size_t index = 0; index < resultValues.size(); ++index ) {
if( expectedValues.at( index ) != resultValues.at( index ) ){
failure = true;
result << "Index " << std::to_string(index) << ": expected(" << std::to_string( expectedValues.at( index ) ) << ") != given(" << std::to_string( resultValues.at( index ) ) << ")\n";
}
}
}
if ( failure )
return result;
else
return ::testing::AssertionSuccess();
}
TEST( TestMath, ScopedFailure3 )
{
std::vector expected;
expected.push_back( 1 );
expected.push_back( 2 );
expected.push_back( 3 );
expected.push_back( 4 );
expected.push_back( 5 );
std::vector results;
results.push_back( 1 );
results.push_back( 1 );
results.push_back( 3 );
std::vector results2;
results2.push_back( 1 );
results2.push_back( 2 );
results2.push_back( 3 );
results2.push_back( 5 );
results2.push_back( 6 );
{
SCOPED_TRACE( "Check first results" );
EXPECT_TRUE( CheckVectors( expected, results ) );
}
{
SCOPED_TRACE( "Check second results" );
EXPECT_TRUE( CheckVectors( expected, results2 ) );
}
}
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.