Tests won´t show up in VS 2019 Test Explorer about googletestadapter HOT 16 CLOSED

The VS test framework works in 2 phases: Discovery (=> tests are displayed in the test explorer, but not run) and execution. For the former, GTA uses the gtest_list_tests flag, so this is perfectly normal.

To figure out why your tests are not discovered any more, first follow the trouble shooting guide. My initial guess would be that your change prevents gtest from properly listing the tests - are they displayed on the console if you run your executable with the option?

If this doesn't help, please activate all debugging options in the settings, rebuild your project (which triggers test discovery), and attach the resulting output from the Test Output window as a text file - maybe we (i.e., I ;-) can see the reason this way...

from googletestadapter.

Closed due to inactivity - feel free to reopen...

from googletestadapter.

Sorry, was gone for a break!
You say:

• are they displayed on the console if you run your executable with the option?

What option do you mean, gtest_list_tests? Yes, they all (TEST_P - each param test) will be dispayed in console but won´t be executed which seems normal according to your statement.

I link some libraries like this: $(SolutionDir)$(Configuration);$(SolutionDir)lib; in the project properties "Additional Library Directories", could that be a problem? You mention something in the troubleshoot concerning the "PATH extension" which I don´t quite understand...

from googletestadapter.

GTA does not make use of any VS settings for technical reasons. If these libraries are required for test execution, then I wonder why your executable can be run from the console... Anyways: the PATH extension option is for this very purpose - use it to add the required libraries to the path.

Once more: if in the console, you cd into the directory the executable is located at, and if you run the very same command that GTA prints out in the Test output window, your tests should be discovered (because that's what GTA does). If this is not the case, fix it :-) - if they do run, I need the complete output.

from googletestadapter.

Ok, that is good to know! I assumed GTA uses VS settings as well. The problem is most likely the working directory, which I redirected to a directory which contains several dlls needed for execution. Running it from console in its original directory fails!
So, would I add the particular working directory to the path extension in the GTA settings?

from googletestadapter.

You would first RTFM :-)

It kind of sounds like the Working directory option might be useful here, probably in combination with placeholders. The other possibility would be to add each dll with the *Path extension" option.

from googletestadapter.

Alright, thanks for those hints ;-)
It is still not working but I will play around with both options and see what happends.
What I just don´t understand is how the GTA settings would have influence on the test execution via console? Is a rebuild always necessary after changing settings?

from googletestadapter.

They don't - what made you think that? It's just that you can check on the console whether it should work with GTA, and under what circumstances (e.g., using a particular working directory). And then you can configure GTA such that it invokes the executable under the same circumstances.

from googletestadapter.

Here's how I would tackle the problem (assuming that you can successfully list your tests by cd-ing into directory C:\\foo (which does not contain your executable bar.exe) and just running <some path>\bar.exe --gtest_list_tests):

• configure Working dir in VS options:
  
• check if discovery works by rebuilding project (tests should appear in Test Explorer)
• switch the definition of Working dir to something relative to e.g. your test executable, making use of the available placeholders (e.g., $(ExecutableDir)\..\foo)
• check if discovery still works
• add a solution settings file (or a project settings file) to your solution with that setting (see example) to make sure that tests are discovered if somebody freshly checks out the solution
• remove above setting from VS options
• check if discovery still works
• [optional] donate to the GTA project ;-)

from googletestadapter.

When
"...cd-ing into directory" is supposed to be my working directory, then yes, gtest_list_tests works as well as executing it without this option!
However, setting the working dir and rebuilding wouldn´t reveal the tests.
Btw. I am using version 0.18.0.1432, where my settings mask looks different to yours - is that a problem?

Could there be a problem with the "Output Directory" in VS? I am using a solution wide setting: $(SolutionDir)$(Configuration)

from googletestadapter.

First of all, I see a different problem: You have installed Test Adapter for Google Test (the MS clone of my adapter which comes with VS) as well as Google Test Adapter (my adapter), which does not work - please uninstall one of them (and if that happens to be my adapter, please get support from the MS guys).

It's hard to say what exactly happens without the log output from the adapter. Please enable all debugging options, rebuild, and attach the output of the Test Output window as a file.

Any, by the way, I asked for this in my very first reply. I do not want to come across unfriendly, but this is really starting to get annoying. Please do not expect any more answers from me before you have read the docs, and provided all information I need.

from googletestadapter.

Sorry for the odyssey, but as long as I configure in the wrong Test Adapter your docs won´t be effective there - or did I overlooked the passage with those two adapters and one to be uninstalled?!
Anyways, it was my fault and I thank you for your support (I will forward a donation query to my boss!).

from googletestadapter.

Did this indeed fix the issue? Sweet - glad that we got it working finally... (and thanks for the donation request!).

Btw (and I might loose my donation now ;-) ): The option Prefix output with [GTA] is rather useful for similar cases - we could have solved this within two steps :-)

I agree that this info should be contained in the docs - the reason that it isn't is that having both adapters installed resulted in errors. They have apparently been fixed by MS in the meantime, making this problem much more subtle... I will add according information soon.

from googletestadapter.

Ya, that fixed it, however now I have the problem passing the working directory via the settings file. Could you have a quick look, what I do wrong? The file is called "aqs.gta.runsettings" and is located in the solution folder of aqs.sln.
<RunSettings <GoogleTestAdapterSettings> <SolutionSettings> <Settings> <WorkingDir>c:\mg\aqs_test\medigraf</WorkingDir> </Settings> </SolutionSettings> <ProjectSettings> <Settings ProjectRegex="aqs_modulesTests_gta\.exe"> <TestDiscoveryRegex>.*Tests.*_gta.exe</TestDiscoveryRegex> <WorkingDir>c:\mg\aqs_test\medigraf</WorkingDir> </Settings> <Settings ProjectRegex="aqs_softDapTests_gta\.exe"> <TestDiscoveryRegex>.*Tests.*_gta.exe</TestDiscoveryRegex> <WorkingDir>c:\mg\aqs_test\medigraf</WorkingDir> </Settings> </ProjectSettings> </GoogleTestAdapterSettings> </RunSettings>

The test output is:

Executing all tests in project: aqs_SoftDapTests
========== Starting test run ==========
[GTA] Visual Studio Version: VS2019
[GTA] Google Test Adapter: Test execution starting...
[GTA] Solution settings: AdditionalPdbs: '', AdditionalTestExecutionParam: '', BatchForTestSetup: '', BatchForTestTeardown: '', BreakOnFailure: False, CatchExceptions: True, DebuggerKind: Native, EnvironmentVariables: '', EnvironmentVariablesParser: GoogleTestAdapter.Helpers.EnvironmentVariablesParser, ExitCodeTestCase: '', HelperFilesCache: GoogleTestAdapter.Settings.HelperFilesCache, KillProcessesOnCancel: False, MaxNrOfThreads: 16, MissingTestsReportMode: ReportAsNotFound, NrOfTestRepetitions: 1, OutputMode: Debug, ParallelTestExecution: False, ParseSymbolInformation: True, PathExtension: '', PrefixOutputWithGta: True, PrintTestOutput: True, RunDisabledTests: False, SeverityMode: Automatic, ShuffleTests: False, ShuffleTestsSeed: 0, SkipOriginCheck: False, SummaryMode: WarningOrError, TestDiscoveryRegex: '', TestDiscoveryTimeoutInSeconds: 30, TestNameSeparator: '', TimestampMode: Automatic, TraitsRegexesAfter: {}, TraitsRegexesBefore: {}, WorkingDir: '$(ExecutableDir)'
[GTA] No test case filter provided
[GTA] Running 162 tests...
[GTA] No settings configured for test executable 'C:\mg\aqs\Debug\aqs_SoftDapTests.exe'; running with solution settings
[GTA] >>>>>>>>>>>>>>> Output of command 'C:\mg\aqs\Debug\aqs_SoftDapTests.exe --gtest_output="xml:C:\Users\examion\AppData\Local\Temp\tmpE805.tmp" --gtest_catch_exceptions=1 --gtest_break_on_failure=0'
[GTA] <<<<<<<<<<<<<<< End of Output
[GTA] Executable C:\mg\aqs\Debug\aqs_SoftDapTests.exe returned with exit code -1073741515
[GTA] Reported 0 test results to VS during test execution, executable: 'C:\mg\aqs\Debug\aqs_SoftDapTests.exe'
[GTA] Test result file C:\Users\examion\AppData\Local\Temp\tmpE805.tmp could not be parsed (completely) - your test executable has probably crashed. Exception message: Das Stammelement ist nicht vorhanden.
[GTA] 162 test cases seem to not have been run - possible causes:

• A test run is repeated, but tests have changed in the meantime
• A test dependency has been removed or changed without Visual Studio noticing
  Failed to add result for test 'ValidateCanonDap/aqsSoftDapCanonTestSuite.ValidateCanonDap/0' with ID '03241a1d-5ed6-cf0b-4c75-050e9a7bc853'.
  [GTA] Reported 162 test results to VS, executable: 'C:\mg\aqs\Debug\aqs_SoftDapTests.exe', duration: 00:00:00.0601451
  [GTA] Execution took 00:00:00.5608551
  [GTA] No settings configured for test executable 'C:\mg\aqs\Debug\aqs_SoftDapTests.exe'; running with solution settings
  [GTA] Google Test execution completed, overall duration: 00:00:00.6025033.
  [GTA]
  ================
  The following warnings and errors occured during test execution:
  [GTA] Test result file C:\Users\examion\AppData\Local\Temp\tmpE805.tmp could not be parsed (completely) - your test executable has probably crashed. Exception message: Das Stammelement ist nicht vorhanden.
  [GTA] 162 test cases seem to not have been run - possible causes:
• A test run is repeated, but tests have changed in the meantime
• A test dependency has been removed or changed without Visual Studio noticing
  ========== Test run finished: 162 Tests (0 Passed, 0 Failed, 0 Skipped) run in 633,8 ms ==========>

from googletestadapter.

I see a broken XML tag at the very beginning (<RunSettings) - could that be the reason? Otherwise, I have no idea - I do not recall any problems in finding the solution settings...

from googletestadapter.

Ah, sorry I did only copy a snippet of the original XML Runsettings where I forgot to close the tag.
I reference your xml schema at the beginning - this is the actual content which does not work:
<?xml version="1.0" encoding="utf-8"?> <RunSettings xmlns="GoogleTestAdapterSettings.xsd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.examion.com/ ../UnitTestUtils/GoogleTestSettings/GoogleTestAdapterSettings.xsd"> <GoogleTestAdapterSettings> <SolutionSettings> <Settings> <WorkingDir>c:\mg\aqs_test\medigraf</WorkingDir> </Settings> </SolutionSettings> <ProjectSettings> <Settings ProjectRegex="aqs_modulesTests_gta\.exe"> <TestDiscoveryRegex>.*Tests.*_gta.exe</TestDiscoveryRegex> <WorkingDir>c:\mg\aqs_test\medigraf</WorkingDir> </Settings> <Settings ProjectRegex="aqs_softDapTests_gta\.exe"> <TestDiscoveryRegex>.*Tests.*_gta.exe</TestDiscoveryRegex> <WorkingDir>c:\mg\aqs_test\medigraf</WorkingDir> </Settings> </ProjectSettings> </GoogleTestAdapterSettings> </RunSettings>

from googletestadapter.