xunit / devices.xunit Goto Github PK
View Code? Open in Web Editor NEWxUnit.net Runners for Devices
License: Other
xUnit.net Runners for Devices
License: Other
Prior to XF 2.3.4, the UWP test runner supporting calling Xamarin.Forms.Forms.Init(e)
to enable XF for tests which require it.
However, starting with XF 2.3.4 the following exception occurs when you attempt to initialize Xamarin.Forms in the UWP test runner:
HRESULT E_FAIL has been returned from a call to a COM component
Changing the initialization entry point has no effect.
Hi.
I have a solution with an UWP application where I need to add some unit so I have decided to tryout xunit and run it on devices like described here: https://xunit.github.io/docs/getting-started-devices-uwp.html.
In respect to running the xunit test on a device everything works fine, but at the moment I add a reference to the xunit.runner.devices nuget package Visual Studio are not able to find/detect test methods anymore.
Output windows shows:
[13-11-2017 10:31:23 Informational] ------ Discover test started ------
[13-11-2017 10:31:23 Warning] [xUnit.net 00:00:00.0838783] Skipping: UwpTestRunner (Could not find/load any of the following assemblies: xunit.execution.desktop.dll)
[13-11-2017 10:31:23 Informational] ========== Discover test finished: 0 found (0:00:00,1760319) ==========
[13-11-2017 10:33:21 Informational] ------ Discover test started ------
[13-11-2017 10:33:22 Warning] [xUnit.net 00:00:00.0835425] Skipping: UwpTestRunner (Could not find/load any of the following assemblies: xunit.execution.desktop.dll)
[13-11-2017 10:33:22 Informational] ========== Discover test finished: 0 found (0:00:00,1620295) ==========
[13-11-2017 10:34:41 Informational] ------ Discover test started ------
[13-11-2017 10:34:41 Warning] [xUnit.net 00:00:00.0745213] Skipping: UwpTestRunner (Could not find/load any of the following assemblies: xunit.execution.desktop.dll)
[13-11-2017 10:34:41 Informational] ========== Discover test finished: 0 found (0:00:00,1550351) ==========
I can see that other people have seen similar problems also but they seems to be related to older earlier versions of xunit/xunit devices which should have been fixed by now.
Any suggestions to what is wrong in my case:
[Details]
UWP target (min = max = Anniversary Update)
Xunit (2.3.1)
Xunit devices (2.3.0)
NuGet 3.3 added support for content files. We need to add support for this here.
Tags: Android, native, C++, NDK, Visual Studio, xunit, GDB, power fail/restart, crash
Microsoft Visual Studio Community 2017
Version 15.0.26228.4 D15RTWSVC
Microsoft .NET Framework
Version 4.7.02046
Installed Version: Community
Visual Basic 2017 00369-60000-00001-AA033
Microsoft Visual Basic 2017
Visual C# 2017 00369-60000-00001-AA033
Microsoft Visual C# 2017
Visual C++ 2017 00369-60000-00001-AA033
Microsoft Visual C++ 2017
Common Azure Tools 1.9
Provides common services for use by Azure Mobile Services and Microsoft Azure Tools.
EclipseAndroidImporter 1.0
Import Android projects from Eclipse.
GitHub.VisualStudio 2.2.0.8
A Visual Studio Extension that brings the GitHub Flow into Visual Studio.
JavaScript Language Service 2.0
JavaScript Language Service
KofePackagePackage Extension 1.0
KofePackagePackage Visual Studio Extension Detailed Info
Markdown Editor 1.11.201
A full featured Markdown editor with live preview and syntax highlighting. Supports GitHub flavored Markdown.
Microsoft MI-Based Debugger 1.0
Provides support for connecting Visual Studio to MI compatible debuggers
Microsoft Visual Studio VC Package 1.0
Microsoft Visual Studio VC Package
Mono Debugging for Visual Studio Mono.Debugging.VisualStudio
Support for debugging Mono processes with Visual Studio.
NuGet Package Manager 4.0.0
NuGet Package Manager in Visual Studio. For more information about NuGet, visit http://docs.nuget.org/.
Visual C++ for Cross Platform Mobile Development (Android) 15.0.26228.00
Visual C++ for Cross Platform Mobile Development (Android)
Visual Studio Spell Check Everywhere VSSpellCheckEverywhere
An extension that enables spell checking within any Visual Studio file editor or tool window that uses WPF text boxes.
https://GitHub.com/EWSoftware/VSSpellChecker
Visual Studio Spell Checker VSSpellChecker
An editor extension that checks the spelling of comments, strings, and plain text as you type or interactively with tool windows.
https://GitHub.com/EWSoftware/VSSpellChecker
Xamarin 4.3.0.784 (73f58d6)
Visual Studio extension to enable development for Xamarin.iOS and Xamarin.Android.
Xamarin.Android SDK 7.1.0.41 (9578cdc)
Xamarin.Android Reference Assemblies and MSBuild support.
Xamarin.iOS and Xamarin.Mac SDK 10.4.0.123 (35d1ccd)
Xamarin.iOS and Xamarin.Mac Reference Assemblies and MSBuild support.`
Here are the steps:
With this code
[Activity(Label = "xUnit Android Runner", MainLauncher = true, Theme= "@android:style/Theme.Material.Light")]
public class MainActivity : RunnerActivity
{
protected override void OnCreate(Bundle bundle)
{
AutoStart = false;
var gea = Assembly.GetExecutingAssembly();
AddTestAssembly(gea);
// you cannot add more assemblies once calling base
base.OnCreate(bundle);
}
}
I get the following error at startup right after launching the test project:
The target device is x86 Google's emulator.
With AddTestAssembly(..)
commented out:
var gea = Assembly.GetExecutingAssembly();
//AddTestAssembly(gea);
the project launches. Therefore there is a bug somewhere in AddTestAssembly(...)
, associated XamarinForms code, and/or the GDB used by VS for Android debugging.
Need to switch to the unified API in order to support 64-bit.
Hello,
I'm trying to add a Xamarin.Forms test project.
I created a Xamarin.Forms PCL and when trying to install the xunit.runner.devices
NuGet package, I get the following error:
Could not install package 'System.Runtime.InteropServices.RuntimeInformation 4.0.0'. You are trying to install this package into a project that targets '.NETPortable,Version=v4.6,Profile=Profile44', but the package does not contain any assembly references or content files that are compatible with that framework. For more information, contact the package author.
What am I doing wrong?
Filtering is more useful if it allows the execution of a subset of tests. Currently to execute a subset of tests one must:
The Run All button does what it says, regardless of any filtering.
To address this, I would suggest a Run Filtered button above the Run All button which executes only those tests that are visible.
I've created very simple skeleton of solution with PCL unit test library & iOS unit test library as suggested in your How To. I uncommented line for PCL in AppDelegate and added few tests in PCL library - as result nothing was discovered by runner.
I think that once runner get test assembly loaded and test assembly has no tests - all further test assemblies are ignored
Having upgraded an older project with lots of tests to the latest device runner, I'm getting an exception on this line of DeviceRunner.cs. The debugging experience in VS+Xamarin is still terrible, but I managed to get this poorly-formatted stack trace:
" at System.Threading.Tasks.TaskCompletionSource`1[System.Object].SetResult (System.Object result) [0x0000c] in /Users/builder/data/lanes/2320/1f068b49/source/maccore/_build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/src/mono/external/referencesource/mscorlib/system/threading/Tasks/TaskCompletionSource.cs:322 \n at Xunit.Runners.DeviceRunner+<>c__DisplayClass17_0.<RunTests>b__0 () [0x000f9] in C:\\BuildAgent\\work\\a3e04f08310ca743\\src\\xunit.runner.viewmodels\\DeviceRunner.cs:254 \n at Xunit.Runners.ThreadPoolHelper+<>c__DisplayClass0_0.<RunAsync>b__0 (System.Object _) [0x00000] in C:\\BuildAgent\\work\\a3e04f08310ca743\\src\\xunit.runner.ios-unified\\ThreadPoolHelper.cs:14 "
I tried at least narrowing down the test that was causing the problem by writing a BeforeAfterTestAttribute
that spat out the test name before and after execution. However, this then causes all tests to pass! So it must be some kind of race condition.
Before I go digging too deep, any ideas what might cause this?
I think this is pretty straightforward and clear, but basically: I would like to get xUnit-style XML Results, because I can run a transform on them to change them to NUnit-style and have our build servers read the XML as part of our CI processes.
As it stands, without just importing this projects code and modifying it myself, I'm forced to write a parser for the text-file output of the current ResultListener. It doesn't look like I can use my own ResultListener, either, which further forces this requirement. I tried importing some of the source code to make a modified DeviceRunner, but ITestRunner is a private interface.
Overall, it seems like this would be a reasonable desire from other people trying to do Continuous Integration as well. I doubt I'm the only one who'd prefer this to be facilitated a little better, but I could easily be wrong.
Consider changing the UX so that when I tap a test name it does not re-execute the test, but rather switches to a backstage view that shows the full test name (no matter how long), shows any relevant test output and failure text, and offers a re-execute button.
For tests that have not yet executed when I tap on them, executing them (before or instead of showing backstage) feels right.
When I updated xam forms and the android support libs to the latest versions (23.2), nuget will go through and update, but then the dependency on this package will then initiate a downgrade of devices.xunit and xamarin forms.
If I update devices.xunit back to 2.0, then it will downgrade other libs.
It gets stuck in a weird cycle. Not sure if it's a nuget bug, but one way to fix it would be to release a nuget update bumping the version numbers in the nuspec, which you have done recently.
VS2017 v15.4.5
xunit.runner.devices v2.3.2
xunit v2.3.1
Followed https://xunit.github.io/docs/getting-started-devices-uwp.html and works successfully on x86 and ARM.
Upon adding a reference to my UWP application, I get an error:
3>error PRI175 : 0x80073b0f - Processing Resources failed with error : Duplicate Entry.
3>GENERATEPROJECTPRIFILE : error PRI277: 0xdef00532 - Conflicting values for resource ''
Do I have to move all logic I want tested to a class library?
When running a test under Android when a filter has been entered, the status for the test is never updated. It just kind of hangs indefinitely. Running the same test without a filter (or on iOS with or without a filter) works as expected.
A repro solution can be downloaded from here.
Here's an example showing the repro with a filter entered and the test consequently stuck:
Get rid of the packages.config's
The font is so large, with no word wrap in the UI, that if I have two or more tests that all start with the same word or two, I can't discern between them. No UI I can find will display the full test name.
For example, here is a screenshot that demonstrates no test name is shown. When the test happens to fail I can usually figure out the test name from the stack trace. But when a test passed, or before I run it, I can't tell.
The layout is such that test names are not discernible on small devices. The text just gets cut off somewhere within the namespace name, or class name if you're lucky. Also, rotating the device in an attempt to find out test names seems to reset everything.
Perhaps trim the start of the text?
Need to read an app.config file and load it into the new xUnitDiscover/Exec settings.
Often one just wants to run tests in a certain fixture. The current filter mechanism only allows filtering based on the test name. I think it would be super useful to allow filtering on the fixture name too.
The Duration
property is assigned to the TestResultViewModel
instance after the TestCaseViewModel
creates its success message, so it always reads as zero.
I just created a test [Theory] with [ClassData] as input. When one of the test case fails, xunit.devices still shows the whole test as successful. I tried it on WP8 test runner only. If it is the last Theory that fails, the whole test is failed. Can you reproduce this error?
[Theory]
[ClassData(typeof(PropertyInputData))]
public void ShouldReadWriteNativeTypes(ITestValue testValue)
{
// Arrange
var inputValue = testValue.Value;
// Act
var outputValue = ReadWriteValueOfType(inputValue);
// Assert // HERE you can manipulate the Assertion to reproduce the error
Assert.Equal(inputValue, outputValue);
}
public class PropertyInputData : IEnumerable<object[]>
{
private readonly List<object[]> data = new List<object[]>
{
new object[] { new TestValue<bool>(true) },
new object[] { new TestValue<short>(short.MaxValue) },
new object[] { new TestValue<ushort>(ushort.MaxValue) },
new object[] { new TestValue<int>(int.MaxValue) },
new object[] { new TestValue<uint>(uint.MaxValue) },
new object[] { new TestValue<float>(float.MaxValue) },
};
public IEnumerator<object[]> GetEnumerator()
{
return this.data.GetEnumerator();
}
IEnumerator IEnumerable.GetEnumerator()
{
return this.GetEnumerator();
}
}
Hi,
More of a question than an Issue, is there anyway to ask the xUnit Runner to run its
tests on the Main UI Thread?
Thanks for any help.
Best Regards
Matt
My tests log to ITestOutputHelper
but the text from these logs doesn't appear to be accessible in the mobile test runner either for failed or successful tests.
I understand that it is easier to maintain runner based on Xamarin.Forms, however it would be useful to have a runner that is not dependent on that library - only Xamarin.Android or Xamarin.iOS.
I have 2 Xamarin.Android app (without Xamarin.Forms) with android support library in newest version (24.2.1). Xamarin.Forms depends on older version of that library (23.3.0) and because of that test project cannot be compiled.
Sometimes you can work around that issue by not using support library in project referenced by test project, but I guess this won't be always possible. So it would be nice to have a runner without Xamarin.Forms, so we can use any version of android support library.
Hi,
It seems that xunit.runner.devices 2.x cannot be installed on netstandard projects? I have tried different profiles (I have added the imports) and based on the supported packages listed on nuget it should work.
Thanks
After upgrading to 1.2.3-build0010 from the CI, I eventually discovered my Android test project stopped building with errors such as those mentioned in this xamarin forum post:
11>\Trunk\MobileUI\Android\Resources\Resource.designer.cs(329,137,329,165): error CS0117: 'Resource.Color' does not contain a definition for 'abc_color_highlight_material'
11>\Trunk\MobileUI\Android\Resources\Resource.designer.cs(529,146,529,177): error CS0117: 'Resource.Drawable' does not contain a definition for 'abc_control_background_material'
11>\Trunk\MobileUI\Android\Resources\Resource.designer.cs(846,133,846,157): error CS0117: 'Resource.Style' does not contain a definition for 'Base_V23_Theme_AppCompat'
11>\Trunk\MobileUI\Android\Resources\Resource.designer.cs(847,139,847,169): error CS0117: 'Resource.Style' does not contain a definition for 'Base_V23_Theme_AppCompat_Light'
The advice in the post is to upgrade to SDK 23. While I can certainly do that, and am working to do so now, it requires I install the SDK 23 and takes a while. Did this devices runner package need to depend on a newer Xamarin.Forms, or would it be OK to refer to the one previously used?
It would be cool if there was a "running" status so that one could quickly filter to those tests that are stuck in their execution (quite a common occurrence when writing async tests).
This problem seems specific to Android. iOS projects are using 2.3.3 without issue for the same set of tests.
Not sure what is going on or how to get more information but using the latest stable version of xunit (2.3.1) and xunit.runner.devices 2.2.0 when the unit test app starts I see an activity indicator and then a list of test assemblies and tests inside.
I couldn't attempt the xunit 2.3.1 / runner 2.3.0 combination because it seems that they import different versions of xunit.execution.dotnet which multiply defines types.
Changing nothing except upgrading to xunit.runner.devices 2.3.2 or 2.3.3 causes the UI to show no test assemblies. Are there any changes needed to the way that AddExecutionAssembly
or AddTestAssembly
are used?
I'm running this on a Raspberry Pi 3, with Windows 10 IoT Core 10.0.14393.576 "Anniversary update".
My unit test app builds, deploys and runs as expected. I can click "Run Everything" and all the tests run (some fail, this is expected).
However, if I click on the summary bar where it tells me how many tests there are, the runner app simply quits instead of showing the test details. It doesn't matter whether I do this before or after running the tests, the app always just quits.
xunit.runner.devices 2.1.0
I think I'm encountering what is a bug, but I'm not really sure where it's happening. I'm trying to setup a file writer for test results on Android 4.4 (Nexus 4)
I have the following as my main activity:
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Reflection;
using Android.App;
using Android.OS;
using Xunit.Runners.UI;
using Xunit.Sdk;
using AndroidEnvironment = Android.OS.Environment;
using Environment = System.Environment;
namespace DeviceGatewaySink.Tests.Android
{
[Activity(Label = "xUnit Android Runner", MainLauncher = true, Theme = "@android:style/Theme.Material.Light")]
public class MainActivity : RunnerActivity
{
protected override void OnCreate(Bundle bundle)
{
this.AddTestAssembly(Assembly.GetExecutingAssembly());
this.AddExecutionAssembly(typeof(ExtensibilityPointFactory).Assembly);
base.OnCreate(bundle);
var path =
AndroidEnvironment.MediaMounted == AndroidEnvironment.ExternalStorageState ?
AndroidEnvironment.GetExternalStoragePublicDirectory(AndroidEnvironment.DirectoryDocuments).AbsolutePath
: Environment.GetFolderPath(Environment.SpecialFolder.MyDocuments);
path = Path.Combine(path, string.Format("tests-{0}", DateTime.UtcNow.ToString("O").Replace(":", "-") + ".log"));
Console.WriteLine("Writing log to: {0}", path);
this.Writer = File.CreateText(path);
}
}
}
However, when I do a test run, it creates a file, but the file is 0 bytes -- no content at all inside the file. I tried overriding onpause and closing the file stream but that didn't do anything different.
Any idea what might be going on here? Any other way to get a file with results to get written out?
The UI thread plays an important role in device apps. Certain tests might need to execute on it.
The STAExample can likely be adapted:
https://github.com/xunit/samples.xunit/tree/master/STAExamples
Hi,
Thanks for this project! I found that I couldn't install the latest RC with WP8.1 support into my solution when running VS2013. I get Install-Package : 'xunit.core' already has a dependency defined for 'xunit.extensibility.core'.
In VS2015 I had no trouble, I wonder if it is related the to change in NuGet dependency version resolution?
Hey there, @onovotny,
I was wondering if using this nuget package to test Xamarin Native projects (not forms). I was curious if anyone has done this yet. Or if everyone is pretty much using Xamarin Forms with xUnit. I've had some struggles with getting the build to behave consistently. I think a big part of my issue is that I'm referencing the original native projects to my test projects. Is this the correct way of going about it?
I have several follow up questions if I'm on the right track. But if I'm going about his wrong, I'd like to know before going further down the wrong path.
Thank you for your time,
Kyle
When running tests under the Android runner, the test counts do not reset back to zero. Thus, running a single test multiple times results in output like this:
Tests run: 1 Passed: 1 Failed: 0 Skipped: 0
Tests run: 2 Passed: 2 Failed: 0 Skipped: 0
Tests run: 3 Passed: 3 Failed: 0 Skipped: 0
I'm using xunit pretty extensively, this project is particularly great for running tests on the devices. However, after yesterday's major update, my breakpoints within unit tests are no longer causing the debugger to stop in visual studio. However, breakpoints do hit, but application execution is suspended -- it's just visual studio doesn't pick up on the breakpoint and I can't resume or step over, or anything like that.
Looking for ideas on how to troubleshoot this. Let me know if this belongs more in the xamarin territory. I figured I'd try here, because breakpoints still work within my application -- just not the test runner.
Visual Studio 2013
I followed the instructions here
http://blog.novotny.org/2014/07/10/getting-started-with-xunit-for-xamarin/
I copied the the content of AppDelegate.cs.txt to AppDelegate.cs, but when i run it, the Simulator window stays black. When i look to Application Output window, there is a logged error:
Could not register the assembly 'xunit.runner.ios': System.Reflection.ReflectionTypeLoadException: The classes in the module cannot be loaded.
at (wrapper managed-to-native) System.Reflection.Assembly:GetTypes (System.Reflection.Assembly,bool)
at System.Reflection.Assembly.GetTypes () [0x00000] in /Developer/MonoTouch/Source/mono/mcs/class/corlib/System.Reflection/Assembly.cs:351
at MonoTouch.Registrar.DynamicRegistrar.CollectTypes (System.Reflection.Assembly assembly) [0x00000] in /Developer/MonoTouch/Source/maccore/src/ObjCRuntime/DynamicRegistrar.cs:228
at MonoTouch.Registrar.Registrar.RegisterAssembly (System.Reflection.Assembly assembly) [0x00049] in /Developer/MonoTouch/Source/maccore/src/ObjCRuntime/Registrar.cs:1307
Here is all my version information
xunit version is 2.0.0-beta4-build-2738
xunit.runner.xamarin version is 0.99.3-beta4-build-1017
=== Xamarin Studio ===
Version 5.5 (build 227)
Installation UUID: 671bdca8-d671-4579-9ae2-006dce0e536d
Runtime:
Mono 3.10.0 ((detached/47db868)
GTK+ 2.24.23 (Raleigh theme)
Package version: 310000019
=== Apple Developer Tools ===
Xcode 6.0.1 (6528)
Build 6A317
=== Xamarin.iOS ===
Version: 8.2.0.193 (Indie Edition)
Hash: 959c1e4
Branch:
Build date: 2014-10-03 00:25:37-0400
=== Build Information ===
Release ID: 505000227
Git revision: 7b721eeec7a2fa4c4f4de0ecd2aed4dc25edac95
Build date: 2014-10-02 15:53:38-04
Xamarin addins: 99ed56b428b31eba1efaace4d82188d6f334e6ca
=== Operating System ===
Mac OS X 10.9.5
Darwin lfwdev08.local 13.4.0 Darwin Kernel Version 13.4.0
Sun Aug 17 19:50:11 PDT 2014
root:xnu-2422.115.4~1/RELEASE_X86_64 x86_64
We have some test cases that do something like the following:
public class ExampleTestClass
{
public static IEnumerable<object[]> MemberDataObject
{
get
{
return new[] {
new object[] { new TestObjectAlpha() },
new object[] { new TestObjectBeta() }
};
}
}
[Theory, MemberData("MemberDataObject")]
public void ExampleTestMethodUsingObjectParameters(TestObject pTestObject)
{
//Test code here
}
Where TestObjectAlpha and TestObjectBeta inherit from TestObject.
What ends up happening, is that the test runner GUI only has ONE case listed to run -- ExampleTestMethodUsingObjectParameters(). Additionally, the test result output from the writer provides no parameters and simply lists this test twice, since there were two cases. That means that if one of the tests failed, I'm not provided any information as to which one failed.
If you use literals or strings, the GUI shows each case individually, and the parameters are provided both in the GUI and from the results writer:
public class ExampleTestClass
{
public static IEnumerable<object[]> MemberDataObject
{
get
{
return new[] {
new object[] { "TestCaseAlpha" },
new object[] { "TestCaseBeta" }
};
}
}
[Theory, MemberData("MemberDataObject")]
public void ExampleTestMethodUsingStringParameters(string pTestCase)
{
//Test code here
}
This results in individual cases in the GUI:
Assembly.Name.ExampleTestClass.ExampleTestMethodUsingStringParameters(pTestCase: "TestCaseAlpha")
Assembly.Name.ExampleTestClass.ExampleTestMethodUsingStringParameters(pTestCase: "TestCaseBeta")
And the results writer also has this information. Since the runner already has to do some kind of reflection, it seems to me that it would make more sense just to output the object name as a string and make it look like so:
Assembly.Name.ExampleTestClass.ExampleTestMethodUsingObjectParameters(pTestObject: TestObjectAlpha())
Assembly.Name.ExampleTestClass.ExampleTestMethodUsingObjectParameters(pTestObject: TestObjectBeta())
Are there some other considerations for why this is not done, like speed? Currently, to get this desired operation, I am forced to do the following instead:
public class ExampleTestClass
{
private static Dictionary<string, TestObject> _testCases = new Dictionary<string, TestObject>
{
{ "TestObjectAlpha", new TestObjectAlpha() },
{ "TestObjectBeta", new TestObjectBeta() }
};
public static IEnumerable<object[]> MemberDataObject
{
get
{
return new[] {
new object[] { "TestObjectAlpha" },
new object[] { "TestObjectBeta" }
};
}
}
[Theory, MemberData("MemberDataObject")]
public void ExampleTestMethod(string pTestCase)
{
//Test code here references _testCases[pTestCase] instead
}
which produces the following:
Assembly.Name.ExampleTestClass.ExampleTestMethod(pTestCase: "TestObjectAlpha")
Assembly.Name.ExampleTestClass.ExampleTestMethod(pTestCase: "TestObjectBeta")
The Android test runner is failing with this exception when I try to upgrade XForms to something greater than 2.1.x. I have scoured everything to make sure I don't have XForms assemblies of another version. Is no one else getting this?
Silverlight based Windows Phone unit tests cannot be easily automated with the vstest.console.exe
runner because the target project in which they are hosted is a standard Windows Phone application contrary to the built-in MSTest Windows Phone (Silverlight) Unit Test project. Is this scenario supported?
@onovotny I am looking for a solution to enable xunit unit test automation in CI platforms such as the Xamarin Mobile Center. I seems that currently I can only run unit tests via MSBuild or post-build schell scripts (and the tests are not carried out in the devices)... After checking the project a bit, I think currently it is not possible to automate this type of testing? As Mobile Center supports Xamarin.UITest, I wonder if it is possible to create UITest for the xunit Android/iOS test applications and integrate the results directly to the output window.
For instance, OOTB, we can create a single test case, simulate the Tap event on "Run Everything" and then retrieve the result of each test and create a summary in the output. A bit more advance way is to create dynamically multiple test cases using something like:
[Test, TestCaseSource("GetTestResults")]
public void ExportTestResults(TestResult testResult)
{
// pass or not depends on the results displayed by the xUnit Device UI
}
static TestResult [] GetTestResults()
{
return GetAllTestResultsFound();
}
I haven't really tested this but it seems that this can be implemented as an external plugin to support CI testing for xunit.device.
Warning Path/xxx.csproj: Warning: Target '_CopyOutOfDateSourceItemsToOutputDirectory', not found in the project (xxx).
Create single page iOS application inside Xamarin.Forms solution. Add package xUnit.net for Devices (stable 2.2.0). Compile project and observe warning.
No warnings
Single warning is shown.
Xamarin Studio for mac.
Just downloading and opening the xUnit.devices.sln solution file on the latest VS Mac yields the following error on xunit.runner.devices:
Load Failed: Project does not support framework '_,Version=10.0'
Playing around with the target frameworks in the csproj just changes the version number (if monoandroid80 comes first, it's version 8.0, and if xamarinios10 comes first it's version 1.0).
To avoid the XY problem, the reason I'm trying to compile is so that I can step through to see why my Xamarin Android runner discovers no tests (see #64), while the iOS one from the same testing sources does (as does .net core). So if I get an answer to that, I won't need to compile anymore but this is still an interesting piece of info I think.
Hi,
Using the filtering box with my project (>700 tests) fails often (but not always) with:
System.ArgumentOutOfRangeException: Argument is out of range.
Parameter name: index
at Xamarin.Forms.ListProxy.get_Item (Int32 index) [0x00000] in <filename unknown>:0
at Xamarin.Forms.TemplatedItemsList`2[Xamarin.Forms.ItemsView`1[Xamarin.Forms.Cell],Xamarin.Forms.Cell].get_Item (Int32 index) [0x00000] in <filename unknown>:0
at Xamarin.Forms.Platform.iOS.ListViewRenderer+ListViewDataSource.GetCellForPath (Foundation.NSIndexPath indexPath) [0x00000] in <filename unknown>:0
at Xamarin.Forms.Platform.iOS.ListViewRenderer+ListViewDataSource.GetCell (UIKit.UITableView tableView, Foundation.NSIndexPath indexPath) [0x00000] in <filename unknown>:0
at at (wrapper managed-to-native) ObjCRuntime.Messaging:void_objc_msgSend (intptr,intptr)
at UIKit.UITableView.EndUpdates () [0x00010] in /Developer/MonoTouch/Source/monotouch/src/build/native/UIKit/UITableView.g.cs:279
at Xamarin.Forms.Platform.iOS.ListViewRenderer.UpdateItems (System.Collections.Specialized.NotifyCollectionChangedEventArgs e, Int32 section, Boolean resetWhenGrouped) [0x00000] in <filename unknown>:0
at Xamarin.Forms.Platform.iOS.ListViewRenderer+<>c__DisplayClass5.<OnCollectionChanged>b__4 () [0x00000] in <filename unknown>:0
at Foundation.NSAsyncActionDispatcher.Apply () [0x00000] in /Developer/MonoTouch/Source/maccore/src/Foundation/NSAction.cs:164
at at (wrapper managed-to-native) UIKit.UIApplication:UIApplicationMain (int,string[],intptr,intptr)
at UIKit.UIApplication.Main (System.String[] args, IntPtr principal, IntPtr delegate) [0x00005] in /Developer/MonoTouch/Source/monotouch/src/UIKit/UIApplication.cs:62
at UIKit.UIApplication.Main (System.String[] args, System.String principalClassName, System.String delegateClassName) [0x0001c] in /Developer/MonoTouch/Source/monotouch/src/UIKit/UIApplication.cs:45
Quite possibly a Xamarin Forms issue, but nonetheless it's blocking me from using the feature. It is also painfully slow, especially when typing the first few characters to filter on.
When running the app an error occurs stating that it cannot find xunit.runner.utility.dll to load, and inspecting the app/apk output you can see it is indeed not bundled (implicit linked assemblies are optimised out)
To get around this you can manually include the xunit.runner.utility package from nuget in the project.
Better solution to ensure that mono bundles this assembly in the app would be to add the xunit.runner.utility nugget package as a dependency to xunit.runner.xamarin and not a file reference:
NOTE: this happens even if you include the following line in the appdelegate or mainactivity classes:
AddExecutionAssembly(typeof(ExtensibilityPointFactory).Assembly);
Basic functionality works -- tests are discoverable and runnable. There is a lot of room for improvements in each of the runner apps
http://developer.xamarin.com/guides/cross-platform/xamarin-forms/
Now I can talk about it :). We should do it with that, so that we can have a single mobile runner across all three mobile platforms!
Installation of the Nuget packages gives an error and rolls back. What am I doing wrong?
I tried also the '2.0.0-build0007' version, same error.
Could not install package 'xunit.runner.utility 2.1.0'. You are trying to install this package into a project that targets 'MonoAndroid,Version=v5.0', but the package does not contain any assembly references or content files that are compatible with that framework. For more information, contact the package author.
Full log:
PM> Install-Package xunit.runner.devices
Attempting to resolve dependency 'xunit.core (≥ 2.1.0)'.
Attempting to resolve dependency 'xunit.extensibility.core (= 2.1.0)'.
Attempting to resolve dependency 'xunit.abstractions (= 2.0.0)'.
Attempting to resolve dependency 'xunit.extensibility.execution (= 2.1.0)'.
Attempting to resolve dependency 'xunit.runner.utility (≥ 2.1.0)'.
Attempting to resolve dependency 'Xamarin.Forms (≥ 1.5.1.6471)'.
Attempting to resolve dependency 'Xamarin.Android.Support.v4 (≥ 23.0.1.1)'.
Attempting to resolve dependency 'Xamarin.Android.Support.Design (≥ 23.0.1.1)'.
Attempting to resolve dependency 'Xamarin.Android.Support.v7.AppCompat (≥ 23.0.1.1)'.
Attempting to resolve dependency 'Xamarin.Android.Support.v7.CardView (≥ 23.0.1.1)'.
Attempting to resolve dependency 'Xamarin.Android.Support.v7.MediaRouter (≥ 23.0.1.1)'.
Installing 'xunit.runner.utility 2.1.0'.
Successfully installed 'xunit.runner.utility 2.1.0'.
Installing 'Xamarin.Android.Support.v4 23.0.1.1'.
Successfully installed 'Xamarin.Android.Support.v4 23.0.1.1'.
Installing 'Xamarin.Android.Support.v7.AppCompat 23.0.1.1'.
Successfully installed 'Xamarin.Android.Support.v7.AppCompat 23.0.1.1'.
Installing 'Xamarin.Android.Support.Design 23.0.1.1'.
Successfully installed 'Xamarin.Android.Support.Design 23.0.1.1'.
Installing 'Xamarin.Android.Support.v7.CardView 23.0.1.1'.
Successfully installed 'Xamarin.Android.Support.v7.CardView 23.0.1.1'.
Installing 'Xamarin.Android.Support.v7.MediaRouter 23.0.1.1'.
Successfully installed 'Xamarin.Android.Support.v7.MediaRouter 23.0.1.1'.
Installing 'Xamarin.Forms 1.5.1.6471'.
Successfully installed 'Xamarin.Forms 1.5.1.6471'.
Installing 'xunit.runner.devices 1.3.2'.
Successfully installed 'xunit.runner.devices 1.3.2'.
Adding 'xunit.runner.utility 2.1.0' to NLog.UnitTests.Xamarin.Android.
Uninstalling 'xunit.runner.utility 2.1.0'.
Successfully uninstalled 'xunit.runner.utility 2.1.0'.
Install failed. Rolling back...
Install-Package : Could not install package 'xunit.runner.utility 2.1.0'. You are trying to install this package into a project that targets 'MonoAndroid,Version=v5.0', but the package does not contain any assembly references or content files that are compatible with that framew
ork. For more information, contact the package author.
At line:1 char:1
+ Install-Package xunit.runner.devices
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (:) [Install-Package], InvalidOperationException
+ FullyQualifiedErrorId : NuGetCmdletUnhandledException,NuGet.PowerShell.Commands.InstallPackageCommand
PM>
Since upgrading to xunit 2.0.0-beta5-build2785 and unit.runner.xamarin 0.99.5-beta5 the flags to AutoStart and TerminateAfterExecution no longer work for iOS or Android, in that the test execution doesn't automatically start, and the app doesn't close automatically.
Manual execution still works.
When running tests under iOS, sometimes the test count does not reflect the actual number of tests executed. For example, when running a single test multiple times, one can see output like this:
Tests run: 1 Passed: 1 Failed: 0 Skipped: 0
Tests run: 1 Passed: 1 Failed: 0 Skipped: 0
Tests run: 0 Passed: 0 Failed: 0 Skipped: 0
Tests run: 1 Passed: 1 Failed: 0 Skipped: 0
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.