Code Monkey home page Code Monkey logo

googletestadapter's People

Contributors

augp avatar csoltenborn avatar frboyer avatar jbzdarkid avatar jgefele avatar jimorc avatar lukaszmendakiewicz avatar martinoschmidt avatar nafest avatar peterbudai avatar rpjohnst avatar sebkraemer avatar shane-sm avatar spebl avatar zahirtezcan-bugs avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

googletestadapter's Issues

Run All fails with NotImplementedException

Using the Run All button in Test Explorer fails to execute tests with the following stack in Test Output. Highlighting all tests in Test Explorer, right-clicking and using "Run Selected Tests" successfully executes all tests.

Visual Studio 2015 Community Edition, Update 2.

ERROR: Exception while running tests: System.NotImplementedException: The method or operation is not implemented.
   at Dia.IDiaSymbol.findChildren(SymTagEnum symTag, String name, UInt32 compareFlags, IDiaEnumSymbols& ppResult)
   at GoogleTestAdapter.DiaResolver.DiaResolver.FindFunctionsByRegex(String pattern)
   at GoogleTestAdapter.DiaResolver.DiaResolver.GetFunctions(String symbolFilterString)
   at GoogleTestAdapter.TestCases.TestCaseResolver.FindTestCaseLocationsInBinary(String binary, List`1 testMethodSignatures, String symbolFilterString, String pathExtension, List`1 errorMessages)
   at GoogleTestAdapter.TestCases.TestCaseResolver.ResolveAllTestCases(String executable, List`1 testMethodSignatures, String symbolFilterString, String pathExtension, List`1 errorMessages)
   at GoogleTestAdapter.TestCases.TestCaseFactory.GetTestCaseLocations(IList`1 testCaseDescriptors, String pathExtension)
   at GoogleTestAdapter.TestCases.TestCaseFactory.CreateTestCases()
   at GoogleTestAdapter.GoogleTestDiscoverer.GetTestsFromExecutable(String executable)
   at GoogleTestAdapter.TestAdapter.TestExecutor.GetAllTestCasesInExecutables(IEnumerable`1 executables)
   at GoogleTestAdapter.TestAdapter.TestExecutor.RunTests(IEnumerable`1 executables, IRunContext runContext, IFrameworkHandle frameworkHandle)

Add stacktrace in output to directly jump to failed check

I just found this adapter and like it's configurability very much so I was wondering if the subject can be done with this adapter...

When I use the cppunittestframework and have a failure there is:
Source: link to the test and
StackTrace: link to the failed check
Could this be done with the adapter so that we would have

Test Name: TestMath.AddFails
Test FullName: TestMath.AddFails
Test Source: c:\users\thomas.roller\documents\work\git\win\misc\googletestadapter\sampletests\tests\basictests.cpp : line 29
Test Outcome: Failed
Test Duration: 0:00:00,002
Result StackTrace: at Test_TestMath::AddFails() in c:\users\thomas.roller\documents\work\git\win\misc\googletestadapter\sampletests\tests\basictests.cpp : line 30
Result Message:
Value of: Add(10, 10)
Actual: 20
Expected: 1000

instead of

Test Name: TestMath.AddFails
Test FullName: TestMath.AddFails
Test Source: c:\users\thomas.roller\documents\work\git\win\misc\googletestadapter\sampletests\tests\basictests.cpp : line 29
Test Outcome: Failed
Test Duration: 0:00:00,002

Result Message:
c:\users\thomas.roller\documents\work\git\win\misc\googletestadapter\sampletests\tests\basictests.cpp:30
Value of: Add(10, 10)
Actual: 20
Expected: 1000

Has someone already tried to do this?

Best Regards
Thomas

Test result names aren't properly published to tfs / trx

TFS was complaining about missing test names.

I traced it down to what appears to be an omission in XmlTestResultParses.cs:91

private TestResult ParseTestResult(XmlNode testcaseNode)
{
    string className = testcaseNode.Attributes["classname"].InnerText;
    string testCaseName = testcaseNode.Attributes["name"].InnerText;
    string qualifiedName = className + "." + testCaseName;

    [...]

    TestResult testResult = new TestResult(testCase)
    {
        ComputerName = Environment.MachineName,
91:     DisplayName = " ";      // shouldn't this be DisplayName = qualifiedName ?
    };

Is this a mistake or is there some reason for it?

Logging stops working if host process is being reused

  • Enable Test>Test Settings>Keep Test Execution Engine Running
  • Enable Test>Test Settings>Default Processor Architecture>x64

    (for x86 the "keep ... running" setting is a no-op.)
  • Run tests multiple times.

After the first run there will be no more test output.

  • Disable Test>Test Settings>Keep Test Execution Engine Running
  • Kill the execution engine process
  • Run tests multiple times.

All runs have test output.

Probably this is because IMessageLogger is being reused between multiple calls into GTA.

This problem is more important for VS2013 because here the execution engine is being reused by default for x86.

Tests can't execute and the process is hanged

I'm new to GTA and GoogleTest and have just set up the add-in with my VS2015. Tests are discovered and visible in the Test Explorer.
But as I build the project, test execution is started but never stops.

I've debug enabled in GTA > General and see following in the test execution console.

------ Discover test started ------
C:\Dev\GTTest1\Debug\GTTest.exe matches regex '[Tt]est[s]?.exe'

image

And I've to kill the process exe from the Task Manager.

I've only one CPP file for test that looks like following:

`

include "stdafx.h"

include <gtest/gtest.h>

include

using namespace std;

int Factorial(int n) {
if (n > 0) {
return n*Factorial(n - 1);
}
return 1;
}

int main(int argc, char **argv) {
::testing::InitGoogleTest(&argc, argv);
int testResult = RUN_ALL_TESTS();
getchar();
return testResult;
}

// Tests factorial of 0.
TEST(FactorialTest, HandlesZeroInput) {
EXPECT_EQ(1, Factorial(0));
}

// Tests factorial of positive numbers.
TEST(FactorialTest, HandlesPositiveInput) {
EXPECT_EQ(1, Factorial(1));
EXPECT_EQ(2, Factorial(2));
EXPECT_EQ(24, Factorial(4));
EXPECT_EQ(40320, Factorial(8));
}
`

If you group tests by project, typed tests with template argument are not linked with the correctly project. Instead they are in a project called external.

Hi!
A small issue I have detecting if you group your tests by project and typed tests are used where the types parameter is a template.
If you modify the types parameter of the macro TYPED_TEST_CASE to be a non-template it will work.

The output of the test detection reports about they a warning:
Warning: Could not find source location for test CNetworkNetModuleCommand_fix/0.AktionAnstossen_NewCommand_HandlerCalled

Attached a file to reproduce the behavior.
TypedTest2nd_test.txt

Sometimes not all test results are reported to VS

Symptoms: When running tests, GTA reports that n tests have been executed, but VS shows less test results in the test explorer. When we insert a small delay between between reporting individual test case results all tests are reported.

Reason: We suspect this is due to inter process communication, but we do not have a lot of expertise in this area, and there is no official documentation of the VS test framework API

Workaround: Configure Report waiting time at Tools/Options/Google Test Adapter/Advanced

Discover test takes a long time

I don't know if this is the test runner or something worse, but the "Discover test" portion of the test runner is taking a very long time. I'm trying to run one single test. This test discovers (using --gtest_list_tests) and executes instantly on the command line.

This is a copy/paste of a fast discovery "Tests" output:

------ Discover test started ------
Found 1 tests in executable D:\path\to\my\test.exe
========== Discover test finished: 1 found (0:00:20.5250523) ==========
------ Run test started ------
Found 1 tests in executable D:\path\to\my\test.exe
Running 1 tests...
Executing tests on 1 threads
Test execution completed.
========== Run test finished: 1 run (0:00:07.5687568) ==========

Discovery takes between 20 and 37 seconds. Execution takes between 2 and 20 seconds. I don't really have any idea what to do at this point, but this is killing my ability to do TDD. The guy sitting right next to me discovers a single test in <1 second.

I looked at process monitor and it appears NotifyChangeDirectory can take up to 30 seconds sometimes. I've been at this 3 hours and I can't seem to figure it out. The crazy thing is it was working fine up until 3 hours ago.

I've tried everything I can think of to no avail:

  • I rechecked out the repo
  • cleared temp directories
  • downgraded the extension to 0.3.0
  • run VS as admin

Have you ever run into this issue?

Properly handle DIA imports

There are some rare issues with VS2015 update 1 and DIA.
Christian, can you provide the error message please?

Specifying a TestCaseFilter still runs all tests but displays only results for filtered tests

This makes it appear as though filters are working even though they really aren't.

Here is the fix:

There is a check (AllTestCasesOfExecutableAreRun()) in CommandLineGenerator.cs if the testCases match all test cases. If they do you don't add any filters. I found this because my tests were taking a long time even when I was executing a few. It turned out all test cases were being run even though a filter was passed in as an argument to vstest.console.exe.

In TestExecutor.cs

public void RunTests(IEnumerable<string> executables, IRunContext runContext, IFrameworkHandle frameworkHandle)
        {
            try
            {
                InitTestEnvironment(runContext.RunSettings, frameworkHandle);

                IList<Model.TestCase> allTestCasesInExecutables = GetAllTestCasesInExecutables(executables).ToList();

                ISet<string> allTraitNames = GetAllTraitNames(allTestCasesInExecutables);
                TestCaseFilter filter = new TestCaseFilter(runContext, allTraitNames, TestEnvironment);
                List<TestCase> vsTestCases =
                    filter.Filter(allTestCasesInExecutables.Select(DataConversionExtensions.ToVsTestCase)).ToList();

>>>>>>>>>>>>               allTestCasesInExecutables =
>>>>>>>>>>>>>>>>>>                    allTestCasesInExecutables.Where(tc => vsTestCases.Any(vtc => >>>>>>>>>>>>>>>>>>  tc.FullyQualifiedName == vtc.FullyQualifiedName)).ToList();

>>>>>>>>>>>>>>>>>>> DoRunTests(allTestCasesInExecutables, allTestCasesInExecutables, runContext, frameworkHandle);

Change to:
>>>>>>>>>>>>                var testCasesToRun =
>>>>>>>>>>>>>>>>>>                    allTestCasesInExecutables.Where(tc => vsTestCases.Any(vtc => >>>>>>>>>>>>>>>>>>  tc.FullyQualifiedName == vtc.FullyQualifiedName)).ToList();

>>>>>>>>>>>>>>>>>>> DoRunTests(allTestCasesInExecutables, testCasesToRun, runContext, frameworkHandle);
            }
            catch (Exception e)
            {
                TestEnvironment.LogError("Exception while running tests: " + e);
            }
        }

Tests: "Debug discovery process?" dialog is shown twice

When starting an experimental VS instance with F5 (or Ctrl-F5) and activating DevelopmentMode in the options of the experimental instance, the dialog which allows to attach to the discovery process is shown twice for some reason.

Handle SCOPED_TRACE output for stack trace message

If we use SCOPED_TRACE we have an additional line for the scope in the error message.
It would be great if this could be integrated into the StackTrace formatting.

I added some senseless examples just to have some examples:

void checkValues( std::vector const& expectedValues, std::vector const& resultValues )
{
ASSERT_EQ( expectedValues.size(), resultValues.size() );
for ( size_t i=0; i<expectedValues.size(); ++i )
{
SCOPED_TRACE( "Checking index " + std::to_string( i ) );
EXPECT_EQ( expectedValues.at(i), resultValues.at(i) );
}// for
}
TEST( TestMath, ScopedFailures )
{
std::vector expected;
expected.push_back( 1 );
expected.push_back( 2 );
expected.push_back( 3 );
expected.push_back( 4 );
expected.push_back( 5 );
std::vector results;
results.push_back( 1 );
results.push_back( 1 );
results.push_back( 3 );
results.push_back( 3 );
results.push_back( 3 );
checkValues( expected, results );
}
TEST( TestMath, ScopedFailures2 )
{
std::vector expected;
expected.push_back( 1 );
expected.push_back( 2 );
expected.push_back( 3 );
expected.push_back( 4 );
expected.push_back( 5 );
std::vector results;
results.push_back( 1 );
results.push_back( 1 );
results.push_back( 3 );
results.push_back( 3 );
results.push_back( 3 );
std::vector results2;
results2.push_back( 1 );
results2.push_back( 2 );
results2.push_back( 3 );
results2.push_back( 5 );
results2.push_back( 6 );
{
SCOPED_TRACE( "Check first results");
checkValues( expected, results );
}
{
SCOPED_TRACE( "Check second results");
checkValues( expected, results2 );
}
}

::testing::AssertionResult CheckVectors( std::vector const& expectedValues, std::vector const& resultValues )
{
::testing::AssertionResult result = ::testing::AssertionFailure();
bool failure = false;
if ( expectedValues.size() != resultValues.size() ) {
failure = true;
result << "Vector size does not match: " << std::to_string( expectedValues.size() ) << " != " << std::to_string( resultValues.size() );
}
else {
size_t index = 0;
for ( size_t index = 0; index < resultValues.size(); ++index ) {
if( expectedValues.at( index ) != resultValues.at( index ) ){
failure = true;
result << "Index " << std::to_string(index) << ": expected(" << std::to_string( expectedValues.at( index ) ) << ") != given(" << std::to_string( resultValues.at( index ) ) << ")\n";
}
}
}
if ( failure )
return result;
else
return ::testing::AssertionSuccess();
}
TEST( TestMath, ScopedFailure3 )
{
std::vector expected;
expected.push_back( 1 );
expected.push_back( 2 );
expected.push_back( 3 );
expected.push_back( 4 );
expected.push_back( 5 );
std::vector results;
results.push_back( 1 );
results.push_back( 1 );
results.push_back( 3 );
std::vector results2;
results2.push_back( 1 );
results2.push_back( 2 );
results2.push_back( 3 );
results2.push_back( 5 );
results2.push_back( 6 );
{
SCOPED_TRACE( "Check first results" );
EXPECT_TRUE( CheckVectors( expected, results ) );
}
{
SCOPED_TRACE( "Check second results" );
EXPECT_TRUE( CheckVectors( expected, results2 ) );
}
}

Support for vstest.console.exe

Open a Visual Studio Developer Command line and run
vstest.console.exe /ListDiscoverers /UseVsixExtensions:true

You will not see GoogleTestAdapter in the output. Other 3rd-party test runners like Chutzpah show up.

v0.5.0.340 is broken in VS2013

After updating through VS and then downloading the .visx from here directly, I get the following error on startup/settings page. It would appear the VS2013 installation is built with references to the VS2015 assemblies? 2013 is v12, and I believe 2015 is v14, right?

<entry> <record>939</record> <time>2016/03/26 20:47:28.608</time> <type>Error</type> <source>VisualStudio</source> <description>SetSite failed for package [GoogleTestExtensionOptionsPage]</description> <guid>{E7C90FCB-0943-4908-9AE8-3B6A9D22EC9E}</guid> <hr>80070002</hr> <errorinfo>Could not load file or assembly 'Microsoft.VisualStudio.ComponentModelHost, Version=14.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified.</errorinfo> </entry> <entry> <record>940</record> <time>2016/03/26 20:47:28.608</time> <type>Error</type> <source>VisualStudio</source> <description>End package load [GoogleTestExtensionOptionsPage]</description> <guid>{E7C90FCB-0943-4908-9AE8-3B6A9D22EC9E}</guid> <hr>80070002</hr> <errorinfo>Could not load file or assembly 'Microsoft.VisualStudio.ComponentModelHost, Version=14.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified.</errorinfo> </entry>

Does not install on Windows 10 with Visual Studio 2013 (with fix)

Running the installer for 0.3.0 (or 0.2.3) fails on a Windows 10 system with Visual Studio 2013 with the installation log below. Changing the line

<Dependency Id="Microsoft.Framework.NDP" DisplayName="Microsoft .NET Framework" Version="4.5" />

to

<Dependency Id="Microsoft.Framework.NDP" DisplayName="Microsoft .NET Framework" Version="[4.5,4.6]" />

in the extension.vsixmanifest file in the .vsix package fixes the issue

Presumably this is the same as changing the corresponding line in the GoogleTestAdapter\VsPackage\source.extension.vsixmanifest file of the VsPackage project (or some settings in the project), couldn't test that though because I have VS 2015 not installed.

24/02/2016 11:31:26 - Microsoft VSIX Installer
24/02/2016 11:31:26 - -------------------------------------------
24/02/2016 11:31:27 - Initializing Install...
24/02/2016 11:31:27 - Extension Details...
24/02/2016 11:31:27 -   Identifier      : GoogleTestAdapterVSIX.e57b9720-dfde-4baa-bcb5-0f75772012b4
24/02/2016 11:31:27 -   Name            : Google Test Adapter
24/02/2016 11:31:27 -   Author          : Christian Soltenborn, Jonas Gefele
24/02/2016 11:31:27 -   Version         : 0.2.3.203
24/02/2016 11:31:27 -   Description     : Adds support for the C++ unit testing framework Google Tests.
24/02/2016 11:31:27 -   Locale          : en-US
24/02/2016 11:31:27 -   MoreInfoURL     : https://github.com/csoltenborn/GoogleTestAdapter
24/02/2016 11:31:27 -   InstalledByMSI  : False
24/02/2016 11:31:27 -   SupportedFrameworkVersionRange : [4.5]
24/02/2016 11:31:27 - 
24/02/2016 11:31:27 -   Supported Products : 
24/02/2016 11:31:27 -       Microsoft.VisualStudio.Community
24/02/2016 11:31:27 -           Version : [12.0,14.0]
24/02/2016 11:31:27 -       Microsoft.VisualStudio.Pro
24/02/2016 11:31:27 -           Version : [12.0,14.0]
24/02/2016 11:31:27 -       Microsoft.VisualStudio.Enterprise
24/02/2016 11:31:27 -           Version : [12.0,14.0]
24/02/2016 11:31:27 -       Microsoft.VisualStudio.Ultimate
24/02/2016 11:31:27 -           Version : [12.0,14.0]
24/02/2016 11:31:27 - 
24/02/2016 11:31:27 -   References      : 
24/02/2016 11:31:27 -       -------------------------------------------------------
24/02/2016 11:31:27 -       Identifier   : Microsoft.VisualStudio.MPF.11.0
24/02/2016 11:31:27 -       Name         : Visual Studio MPF 11.0
24/02/2016 11:31:27 -       Version      : [11.0]
24/02/2016 11:31:27 -       MoreInfoURL  : 
24/02/2016 11:31:27 -       Nested       : No
24/02/2016 11:31:27 - 
24/02/2016 11:31:27 - 
24/02/2016 11:31:27 - Searching for applicable products...
24/02/2016 11:31:27 - Found installed product - Microsoft Visual Studio Community 2013
24/02/2016 11:31:27 - The extension with ID 'GoogleTestAdapterVSIX.e57b9720-dfde-4baa-bcb5-0f75772012b4' is not installed to Microsoft Visual Studio Community 2013.
24/02/2016 11:31:27 - Found installed product - Microsoft Visual Studio 2013 Shell (Integrated)
24/02/2016 11:31:27 - Found installed product - Global Location
24/02/2016 11:31:29 - The following target products have been selected...
24/02/2016 11:31:29 -   Microsoft Visual Studio Community 2013
24/02/2016 11:31:29 - 
24/02/2016 11:31:29 - Beginning to install extension to Microsoft Visual Studio Community 2013...
24/02/2016 11:31:29 - Install Error : Microsoft.VisualStudio.ExtensionManager.MissingTargetFrameworkException: The extension 'Google Test Adapter' requires a version of the .NET Framework that is not installed.
   at Microsoft.VisualStudio.ExtensionManager.ExtensionManagerService.CheckForValidDotNetFramework(IExtension extension)
   at Microsoft.VisualStudio.ExtensionManager.ExtensionManagerService.CheckForInstallBlockers(InstallableExtensionImpl extension, IInstalledExtensionList modifiedInstalledExtensionsList, Boolean isNestedExtension, Boolean& olderVersionInstalled)
   at Microsoft.VisualStudio.ExtensionManager.ExtensionManagerService.InstallInternal(InstallableExtensionImpl extension, Boolean perMachine, Boolean isNestedExtension, IDictionary`2 extensionsInstalledSoFar, List`1 extensionsUninstalledSoFar, IInstalledExtensionList modifiedInstalledExtensionsList, AsyncOperation asyncOp, UInt64 totalBytesToWrite, UInt64& totalBytesWritten)
   at Microsoft.VisualStudio.ExtensionManager.ExtensionManagerService.BeginInstall(IInstallableExtension installableExtension, Boolean perMachine, AsyncOperation asyncOp)
   at Microsoft.VisualStudio.ExtensionManager.ExtensionManagerService.InstallWorker(IInstallableExtension extension, Boolean perMachine, AsyncOperation asyncOp)

Support alternative test name formatting

As discussed in #22, in the Test Explorer window if VS can't fit the text after a "/" (that's then terminated by another "/" or the EOS) then it shortens everything after that "/" with "...". Perhaps a replacement string could be provided in GTA's settings? Eg, instead of "/" a user could specify "->" or something else.

Here are truncated instantiated test cases:
image

And here they are when the window is expanded to fit the next segment:
image

Print overall tests duration per thread if in debug mode

Sometimes parallel test execution takes longer than expected. On reason might be that tests are distributed to the different threads in a sub optimal way (note that this is expected to happen before all tests have been run at least once). To debug such issues (or to find out that instead, the executed tests themselves have unpredictable durations), print out the duration of each executing thread if in debug mode.

Support for VS2012

VS2012 update 4 is already tested.
VS2012 update 1-3 work run, too.
VS2012 w/o update doesn't work due to missing traits support.

Run tests after build does not work

Symptoms: Selecting the Run tests after build option does not or not always result in the tests being run after build completion. Instead, the following error message is printed to the test console: "The specified type member 'Stale' is not supported in LINQ to Entities. Only initializers, entity members, and entity navigation properties are supported."

Reason: We have no idea

Workaround: None so far

Executing lots of tests takes MUCH longer since VS update 2

Since VS update 2, executing our test suite of around 5000 tests takes much longer than before. The problem seems to be at two places:

  • After Google test finished test execution, the process executing the tests seems to be very busy for another 2-3 minutes; the test adapter code "hangs" at while (!process.StandardOutput.EndOfStream) of file Core/Helpers/ProcessLauncher.cs for that time (and afterwards continues normally)
  • Even worse, after the test adapter's code has finished its tasks (i.e., after TestEnvironment.LogInfo("Test execution completed.") of file Core/GoogleTestExecutor.cs has been executed), it takes 15-20 minutes until test results are finally displayed in VS's test explorer, and during this time, VS is rather busy (as Windows' task manager shows).

We can reproduce this behavior on VS 2015 Enterprise Edition update 2, and we can see that it does not happen with update 1.

Provide toolbar for easy access to runtime options

Some options should be (also) changeable via a toolbar. We should support at least these options:

  • Break on failure
  • Catch exceptions
  • Enable parallel test execution

The following ones are also candidates:

  • Also run disabled tests
  • Shuffle tests per execution
  • Print test output

Tests to run always specified on command line, even if running all tests in executable

The bug is that all of the tests suites are specified in the filter on the command line if the whole project is run or even run all is selected.

This occurs because there is a bug in the CommandLineGenerator.AllTestCasesOfExecutableAreRun function. It assumes that the AllTestCases variable only contains tests for that executable, but it actually contains all of the tests for all executables to be run.

To reproduce:

  • Make a solution with two test executable projects and add tests to both.
  • Put a breakpoint in one of the tests
  • In Test Explorer select the project view and right click on one project and "Debug Selected Tests" or even "Debug All".
  • When the breakpoint is hit, check Process Explorer for the test executable and examine the command line parameters.

If running all the tests in an executable there should be no command line parameters.

VS2013 Test Explorer shows incomplete results

First of all, thanks for making GTA! And thanks in advance for improving it :o)

Now for the bugs:

  1. When I run all, failed test cases don't properly appear (yet their space in the tree list seems to be consumed) while passed tests do. To be crystal clear, there's no 'Failed Tests' root node or children, as is the case with Passed Tests.
  2. The formatting of instantiated test cases is cut off after the first forward slash in the test explorer tree list, but prints fine in the detail pane.

image

The project I'm testing was the first thing I opened in VS after getting GTA. I immediately did "Run All Tests" and got output like the above. I then tried doing "Run Failed Tests" and still got the incomplete results (screenshot was from immediately after this action). VS Version/extension info follows:

Microsoft Visual Studio Premium 2013
Version 12.0.40629.00 Update 5
Microsoft .NET Framework
Version 4.6.00081

Installed Version: Premium

Team Explorer for Visual Studio 2013 06191-004-0456033-02040
Microsoft Team Explorer for Visual Studio 2013

Visual Basic 2013 06191-004-0456033-02040
Microsoft Visual Basic 2013

Visual C# 2013 06191-004-0456033-02040
Microsoft Visual C# 2013

Visual C++ 2013 06191-004-0456033-02040
Microsoft Visual C++ 2013

Visual F# 2013 06191-004-0456033-02040
Microsoft Visual F# 2013

CodeContractsHelper_Package 1.0
A package for the CodeContractsHelper extension.

Common Azure Tools 1.4
Provides common services for use by Azure Mobile Services and Microsoft Azure Tools.

GoogleTestExtensionOptionsPage Extension 1.0
GoogleTestExtensionOptionsPage Visual Studio Extension Detailed Info

JetBrains ReSharper Ultimate 2015.1.3 Build 102.0.20150724.141603
JetBrains ReSharper Ultimate package for Microsoft Visual Studio. For more information about ReSharper Ultimate, visit http://www.jetbrains.com/resharper. Copyright © 2016 JetBrains, Inc.

MicroProfiler 1.1.591.0
Realtime Instrumenting Profiler

Please, submit your questions, issues and suggestions to http://code.google.com/p/micro-profiler/issues/list
Copyright (c) 2011-2015 Artem Gevorkyan

Microsoft Azure Mobile Services Tools 1.4
Microsoft Azure Mobile Services Tools

NuGet Package Manager 2.8.60723.765
NuGet Package Manager in Visual Studio. For more information about NuGet, visit http://docs.nuget.org/.

P4VS - Perforce Plugin for Visual Studio 2015.2.124.8831
P4VS is the Perforce integration for Microsoft Visual Studio. P4VS gives developers working in Visual Studio access to Perforce version control features. P4VS gives you easy access to all of the Perforce version control tools and information you need to do your daily work. Check out, check in, see history, and more, directly from Visual Studio. P4VS is a fully compliant Visual Studio Integration Package, designed for full compatibility and ease of use. P4VS is built on P4API.NET, Perforce's fully supported API for the .NET environment. Perforce Software provides fast, robust version control to teams of all sizes working on any type of digital asset development.

PreEmptive Analytics Visualizer 1.2
Microsoft Visual Studio extension to visualize aggregated summaries from the PreEmptive Analytics product.

PVS-Studio 5.31.14891.5
PVS-Studio is a static code analyzer for C/C++ (Microsoft Visual Studio) with a simple licensing and pricing policies which is easy to install and use without a need to deploy complex maintenance environment.

Python Tools for Visual Studio 2.2.31124.00
Python Tools for Visual Studio provides IntelliSense, projects, templates, Interactive windows, and other support for Python developers.

Python Tools for Visual Studio - Profiling Support 2.2.31124.00
Profiling support for Python projects.

RegexTester 1.0
This is regular extension tester.

Release Management for Visual Studio Package 1.0
Release Management for Visual Studio

SyncfusionAssemblyReference 1.0
Information about my package

SyncfusionItemTemplates 1.0
Information about my package

Visual Assist
For more information about Visual Assist, see the Whole Tomato Software website at http://www.WholeTomato.com. Copyright (c) 1997-2015 Whole Tomato Software, Inc.

Visual Studio Tools for Unity 2.1.0.0
Visual Studio Tools for Unity

Windows Phone 8.1 SDK Integration 1.0
This package integrates the tools for the Windows Phone 8.1 SDK into the menus and controls of Visual Studio.

Parameters ared set for visual studio, not per projects

I have two solutions, one where all gtests are following the same rule, with 10 gtests project, it works perfectly.

The second solution is a mix of mess, with gtests initializing web servers, making them crash during test discovery with lots of popups, but I can't disable them easily. So if you could associate the parameters of GTA with a sln, it would be perfect for me ( maybe by loading a specific file for config )
$(SolutionDir)\gta.config ?

Adapter only identifies tests in 3 of 5 of my exe files

We just started using google test a few weeks back. We'd been working with the command line output only. We found this extension and it seems to work correctly, but it only finds tests in 3 of the 5 exes I have implementing tests.

I started going through GoogleTestDiscoverer.cs to see if I can figure out what's wrong, but it takes the entire binary exe file as an argument? Can someone help me figure out how to start debugging this?

Thanks.

Support for running without VSIX

Nunit Test Adapter can be called by vstest.console.exe either via the /TestAdapterPath switch or by pointing to their dll's in the .runsetttings file :

<RunConfiguration>
    <!-- Path to Test Adapters -->
    <TestAdaptersPaths>c:\path\adapter</TestAdaptersPaths>
</RunConfiguration>

This allows running NUnit tests like regular MSTests without customizing the TFS server configuration and uploading the test adapter to the TFS controller.

GoogleTestAdapter is for some reason not correctly recognized as an adapter when supplied like that, and thus can't be called from TFS without writing custom scripts, to call it correctly, and then publish the test results (as well as coverage info).

I'd be willing to help research and patch it, if you're willing to spare some time for guidance.

Options have no effect on run selected?

Hi there
I have disabled break on failure and enabled catch exceptions in the options but when I invoke "Run selected test" the options are not used.
Executing command 'base_test.exe --gtest_output="xml:C:\Users\thomas.roller\AppData\Local\Temp\tmp7548.tmp" --gtest_catch_exceptions=0 --gtest_break_on_failure=1 --gtest_filter=TestCOPManagerIterator.BackwardBasics'

Am I doing something wrong?

Not all tests discovered

Hi!
I detecting a problem with typed tests. They are not discovered

Output:
Found 306 tests in executable C:\Source\zenon\Supervisor\main\BIN\PC\UTests\PROGRAMFILES32\DebugNoUTest\zenon32misc_net_UTest.exe
Warning: Could not find source location for test Typed/fixCCommandReactionsForNetModule/0.Process_ForwardsCallToConcreteImplementation_Yes
Warning: Could not find source location for test Typed/fixCCommandReactionsForNetModule/1.Process_ForwardsCallToConcreteImplementation_Yes
Warning: Could not find source location for test CNetworkNetModuleCommand_fix/0.AktionAnstossen_NewCommand_HandlerCalled
Warning: Could not find source location for test CNetworkNetModuleCommand_fix/0.AktionAnstossen_WithOutCommandInstance_NoCrash
Warning: Could not find source location for test CNetworkNetModuleCommand_fix/0.Serialize_HappyPath_SameAsWritten
Warning: Could not find source location for test CNetworkNetModuleCommand_fix/0.Serialize_WithOutCommandInstance_NoCrash
Warning: Could not find source location for test CNetworkNetModuleCommand_fix/0.Info_HappyPath_Filled
Warning: Could not find source location for test CNetworkNetModuleCommand_fix/0.Info_WithOutCommandInstance_NoCrash
And so on

At least 262 test cases are detected and executed only 250!?
Output:
Many other …
Warning: Could not find source location for test CNetworkNetModuleCommand_fix/5.AddLogEntry_HappyPath_Filled
Running 250 tests...
Test execution completed.

Thanks for any suggestion!
Alfred

Actively kill test processes if test execution is canceled

Currently, if test execution is canceled, one still has to wait for the running test processes to finish or to kill these processes using the Windows TaskManager. Implement support for actively killing these processes if test execution is canceled.

No source locations for typed tests and type-parameterized tests.

We don't find source location for typed tests and type-parameterized tests.

    class MyStrangeArray : public std::array<int, 3>
    {
    public:  MyStrangeArray(std::initializer_list<int> init) : array({ 3,2,1 }) {}
    };


    template< typename int_container_type >
    class TypedTests : public ::testing::Test {
    public:
        typename int_container_type container{ 1,2,3 };
    };

    typedef ::testing::Types<std::vector<int>, std::array<int,3>, MyStrangeArray> IntContainerTypes;
    TYPED_TEST_CASE(TypedTests, IntContainerTypes);

    TYPED_TEST(TypedTests, CanIterate) {
        int sum = 0;
        for (int value : this->container)
            sum += value;
        EXPECT_EQ(1 + 2 + 3, sum);
    }

    TYPED_TEST(TypedTests, CanDefeatMath) {
        EXPECT_NE(this->container[0] + this->container[1], this->container[2]);
    }


    template< typename int_container_type >
    class TypeParameterizedTests : public TypedTests<int_container_type> {};

    TYPED_TEST_CASE_P(TypeParameterizedTests);

    TYPED_TEST_P(TypeParameterizedTests, CanIterate) {
        int sum = 0;
        for (int value : this->container)
            sum += value;
        EXPECT_EQ(1 + 2 + 3, sum);
    }

    TYPED_TEST_P(TypeParameterizedTests, CanDefeatMath) {
        EXPECT_NE(this->container[0] + this->container[1], this->container[2]);
    }

    REGISTER_TYPED_TEST_CASE_P(TypeParameterizedTests, CanIterate, CanDefeatMath);

    typedef ::testing::Types<std::array<int, 3>, MyStrangeArray> IntArrayTypes;
    INSTANTIATE_TYPED_TEST_CASE_P(Vec, TypeParameterizedTests, std::vector<int>);
    INSTANTIATE_TYPED_TEST_CASE_P(Arr, TypeParameterizedTests, IntArrayTypes);

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.