Code Monkey home page Code Monkey logo

winium.mobile's Introduction

English description | Описание на русском

Winium for Store Apps

Build Status Winium.StoreApps.InnerServer Inner Server NuGet version Winium.Silverlight.InnerServer NuGet version GitHub release GitHub license

Winium.Mobile is Selenium Remote WebDriver implementation for automated testing of Windows StoreApps or Silverlight apps on Windows Phone 8.1 or Windows 10 Mobile

Winium.Mobile is an open source test automation tool for both Windows StoreApps and Windows Silverlight apps tested on Windows Phone or Windows Mobile emulators.

Supported Platforms

  • StoreApps and Silverlight
  • Windows Phone 8.1
  • Windows 10 Mobile

For Windows Desktop (WPF, WinForms) test automation tool see Winium Desktop.

Why Winium?

You have Selenium WebDriver for testing of web apps, Appium for testing of iOS and Android apps. And now you have Selenium-based tools for testing of Windows apps too. What are some of the benefits? As said by Appium:

  • You can write tests with your favorite dev tools using any WebDriver-compatible language such as Java, Objective-C, JavaScript with Node.js (in promise, callback or generator flavors), PHP, Python, Ruby, C#, Clojure, or Perl with the Selenium WebDriver API and language-specific client libraries.
  • You can use any testing framework.

Requirements

  • Windows 8 or higher
  • Visual Studio 2013 with Update 2 or higher
  • Windows phone 8.1 SDK and/or Windows 10 SDK

You can get Visual Studio and SDK from Microsoft here.

Quick Start

App under test (AUT) is application that you would like to test.

  1. Add reference to either Winium.StoreApps.InnerServer or Winium.Silverlight.InnerServer in AUT project (install Winium.StoreApps.InnerServer NuGet package / install Winium.Silverlight.InnerServer NuGet package or build project yourself)

  2. In your AUT's source code add following lines to be called on UI thread after visual root is initialized (usually in MainPageOnLoaded for vanilla app or PrepareApplication if you use Caliburn.Micro)

    // For StoreApps
    AutomationServer.Instance.InitializeAndStart();
    
    // For Silverlight apps
    AutomationServer.Instance.InitializeAndStart(RootFrame);

    or (will include driver only for debug build)

    #if DEBUG
    	// For StoreApps
    	AutomationServer.Instance.InitializeAndStart();
    
    	// For Silverlight apps
    	AutomationServer.Instance.InitializeAndStart(RootFrame);
    #endif // DEBUG
  3. Assure that Internet (Client & Server) capability is enabled in package manifest of your AUT. It should be enabled by default for Windows 8.1 apps. In UWP (Windows Mobile 10) it is disabled by default (only Internet (Client) is enabled).

  4. Write your tests using your favorite language. In your tests use app desired capability to set path to tested app's appx/xap file. Here is python example:

    # put it in setUp
    app_path = 'C:\\path\\to\\testApp.appx' # For StoreApps
    app_path = 'C:\\path\\to\\testApp.xap' # For Silverlight apps
    self.driver = webdriver.Remote(
    	command_executor='http://localhost:9999',
    	desired_capabilities={'app': app_path}
    )
    # put it in test method body
    element = self.driver.find_element_by_id('SetButton')
    element.click()
    assert 'CARAMBA' == self.driver.find_element_by_id('MyTextBox').text

    Make sure to set deviceName capability to Emulator to run on Windows Phone 8.1 if you are using the driver on a system where Visula Studio 2015 or Winodws 10 SDK is installed.

  5. Start Winium.Mobile.Driver.exe (download release from github or build it yourself)

  6. Run your tests and watch the magic happening

Writing tests

Essentially, Winium.Mobile supports limited subset of WebDriver JSON Wire Protocol, which means that you can write tests just like you would write for Selenium or Appium, here are some docs. For test samples look at our functional tests or test samples page.

Winium.Mobile vs Winium.StoreApps.CodedUi

Winium.Mobile vs Winium.StoreApps.CodedUi

How it works

Winium.Mobile consists of two essential parts:

  1. Winium.Mobile.Driver implements Selenium Remote WebDriver and listens for JsonWireProtocol commands. It is responsible for launching emulator, deploying AUT, simulating input, forwarding commands to Winium.StoreApps.InnerServer, etc.

  2. Winium.StoreApps.InnerServer / Winium.Silverlight.InnerServer (the one that should be embedded into AUT) communicates with Winium.Mobile.Driver.exe and executes different commands, like finding elements, getting or setting text values, properties, etc., inside your application.

Winium.StoreApps structure

Contributing

Contributions are welcome!

  1. Check for open issues or open a fresh issue to start a discussion around a feature idea or a bug.
  2. Fork the repository to start making your changes to the master branch (or branch off of it).
  3. We recommend to write a test which shows that the bug was fixed or that the feature works as expected.
  4. Send a pull request and bug the maintainer until it gets merged and published. 😃

Contact

Have some questions? Found a bug? Create new issue or contact us at [email protected]

License

Winium is released under the MPL 2.0 license. See LICENSE for details.

winium.mobile's People

Contributors

bayandin avatar nickab avatar ole-vegard avatar sargissargsyan avatar skyline-gleb avatar tkurnosova avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

winium.mobile's Issues

Add `pre-launch` driver option

Similar to Appium's --pre-launch

Pre-launch the application before allowing the first session (Requires –app and, for Android, –app-pkg and –app-activity)

This will be useful for unsupervised CI test runs on virtual machines (which are slower than bare hosts, resulting in remote procedure call failing in xde or smartdevice connectivity due to timeouts)

Popup child root element is not included in search

Currently we cannot find and return popup child root element to tests as we start searching from it and only consider his descendants. Meaning that we cannot locate ContentDialog by class name as it is a popup root element, but we can locate all his descendants, this might make searching for such standard popup elements and distinguishing between them unnecessary complicated.

Popup child root elements should be included in search.

Driver response hangs for a long time if app crashed

Add timeouts to all forwarded commands, so that if the app crashes. Driver will get timeout earlier and will be able to respond to client with error, instead of making a client to wait for a big default timeout.

Add support for XPath locator strategy

XPath or similar (css selector-like) locator strategy might be useful for apps that have complex UIs or not quite tuned for ease of testing.

  1. XPath locator can be already implemented in driver based on PageSourceCommand and standard XmlDocument, but it will require PageSourceCommand to be modified in a way that it will save all elements to registered elements storage. Such solution is easiest for implementation at the moment, but will be very slow as it will require whole UI visual tree to be traversed and represented as XML nodes regardless of actual XPath, which can match first node.
  2. Implementing custom XmlReader, that would return nodes on the fly as needed, might be a better option in terms of performance. See Implementing XmlReader Classes for Non-XML Data Structures and Formats.

Add server `deviceName` capability

Add server deviceName capability which will enable users run tests in parallel as in Appium by launching n instances of driver listening on different ports with different deviceNames specified for each.

Automatically inject InnerServer code in AUT before deployment

@jimevans gave and idea on how we could implement automatic InnerServer injection into the AUT. If implemented, then there will be no need to manually add InnerServer to your app code and have a special configuration that you have to build especially for automated testing.

An example demo project for xap files can be found here https://github.com/jimevans/InjectedCodeDemo.
For appx you can't simple unzip it and zip it back, you will have to use MakeAppx and SignTool for signing and unpacking/packing.

Regarding IsUserVisible and IsHitTestVisible

Currently, the IsUserVisible function checks IsHitTestVisible but I'm not sure it should. IsHitTestVisible is a bit of a misnomer. It doesn't actually say anything about if the element is visible or not. It just says whether you can interact with it. It is true by default. Documentation here : https://msdn.microsoft.com/en-us/library/system.windows.uielement.ishittestvisible(v=vs.110).aspx#NotExistJustToMakeTheAElementVisible

We did a few tests with removing IsHitTestVisible from IsUserVisible. It works in most cases. The only case it didn't work was a case where we generate the components in the .cs file, rather than defining them in the XAML file. In this case, the driver would successfully get the correct coordinates and click at that position. But somehow the element wasn't ready and nothing happened. Since the driver just clicks a position, it reported a success and the test moved on to the next step Which of course failed because our application never reacted to the click. We also tried checking HitTestVisible with the .attribute command through Winium before clicking. But it reported it to true so that wasn't much of a help either.

It was working before we removed IsHitTestVisible, so when we debugged IsUserVisible we found that the IsHitTestVisible was set to false, even though we never set it ourselves. I'm not sure why this happens. But we found a workaround by implementing retry functionality for every step of the test.

So removing IsHitTestVisible would be the logical option, but here is still a chance that this might fail other tests. Otherwise there is no reason to use it as far as I can see. If you wanted to check IsHitTestVisible, you should use the .attribute() command.

Add support for Toggle pattern

AppBarToggleButton does not support invoke pattern and could not be clicked due to AppBar coordinates mixup.

Add support for TogglePattern via ExecuteScript with automation: toggle.
See more at msdn

The remote procedure call failed on app launch

Rarely after deploying app and uploading files to app storage an "Unknown error: The remote procedure call failed." might occur when trying to launch app.

This only happens on build machine that is under hard memory and hdd usage, does not happen on dev machines that usually run tests faster. Might be related to some inner timeouts in SmartDevice API.

Possible hot fix: simple retry launching app a few times until some timeout is reached. Yet it would be better to research why exactly such exception is raised inside SmartDevice API.

New sessions fails to start if user name is not in lower case

New sessions fails to start because AutomationClient is unable to connect to emulator by name if user name is not in lower case.

new XdeWmiFactory().GetVirtualMachine(Name) looks up for VM using case insensitive matching and returns VM with a name set to the one that was passed in (so that the actual VM name might have different case). But AutomationClient.CreateAutomationClient uses case sensitive matching to find the emulator. This means that we either have to:

  1. lower case user name before appending it to VM name,
  2. or we should use new XdeWmiFactory().GetAllVirtualMachines(settings) method to get a full list of emulators with a names in correct case.
  • Option 1 is easy fix, but we might get into some another case problem in the future.
  • Option 2 should be more robust and preferred if it does not hit performance.

EmulatorController.cs#L205-L208, EmulatorController.cs#L40-L41

Incorrect driver response

Драйвер возвращает ответ на любой запрос кроме создания сессии с null значением поля SessionId. Из-за чего биндинги не могут его распарсить.
Ниже приведены списки команд и ответы от ChromeDriver

  • CreateSession
    {(45c49ade8e122a00a6d00e3a3dd8e098 Success: System.Collections.Generic.Dictionary`2[System.String,System.Object])}
  • GoUrl
    {(45c49ade8e122a00a6d00e3a3dd8e098 Success: )}
  • FindElement
    {(45c49ade8e122a00a6d00e3a3dd8e098 Success: System.Collections.Generic.Dictionary`2[System.String,System.Object])
    То есть на каждый запрос ChromeDriver возвращает значение идентификатора сессии. Возможно это верно ещё и потому, что есть возможность работы с несколькими сессиями.

Т.о. нужно добавить значение SessionId в ответ на любой запрос к драйверу

C# TouchActions throws exception

Calling

var openNextScreen = new TouchActions(Driver);

throws exception

exception (An exception of type 'System.ArgumentException' occurred in WebDriver.dll but was not handled in user code  |  Additional information: The IWebDriver object must implement or wrap a driver that implements IHasTouchScreen.

Probably driver should return some kind of special capability to indicate to C# bindings that it can do touch actions.

isEnabled property doesn't work

When I try to access the Enabled property of a component, an exception is thrown. Should I perform some configuration or extra implementation for this to work?

Change SendKeysToElement and SendKeysToActiveElementExecutor to use IXdeAutomation.InputPipe.SendText

Currently SendKeys and SendKeysToActiveElementExecutor is separated into two steps:

  1. Extract regular keys from value and send to InnerServer to be set on element text property using automation (not even close to input simulation).
  2. Send special keys using emulator API (actual input simulation)

Using AutomationClient.CreateAutomationClient(this.emulatorVm.Name).InputPipe.SendText will simplify keys sending, as it will require execution only on Driver side (no code in InnerServer). Also, it will let us fully implement send keys as they should be, including ability to send Backspace and other special keys at any moment of text input.

Add port discovery and make innerPort dynamic

Making innerPort dynamically assignable from predefined range and including innerPort auto-discovery will greatly simplify parallel execution of tests on different emulators.

Currently to run tests in parallel on same machine one would have to:

  1. build n copies of .appx, but with different innerPorts,
  2. start n copies of driver,
  3. start n sessions against each driver with different innerPort capability.

All this hassle with innerPort is caused by the fact that emulators do forward their port to localhost. Maybe there is another way to deal with it, for example modifying Hyper-V machine settings or switch in someway, but it seems more invasive.

Adding port discovery will let us get rid of step 1 completely and simplify step 3.

Research Driver - InnerServer communication refactoring

  • Server --Proxy request--> Inner Server (same as it is now)
  • Duplex Socket Server <----> Web Socket Client (simplifies code on the phone side)
  • File based communications (solves the problem of #67 network throttling killing communication, but introduces polling)

Add additional properties to Winium.StoreApps.InnerServer PageSourceCommand

Currently PageSourceCommand returns only few base attributes of WiniumElement. If extend it, we can display in Inspector additional properties depending on the type of FrameworkElement.

For example, if FrameworkElement is TextBox, then add FontSize, FontStyle, Foreground and other Windows.UI.Xaml.Controls.TextBlock properties.

Use JSON RPC for communication between Driver and InnerServer

Currently we use custom mix of JSON RPC and HTTP to communicate between Driver and InnerServer, it has it roots in older implementations, where Driver forwarded HTTP requests to InnerServer without processing. But now it makes code hard to understand and maintain as we have tow hacked HTTP server implementations based on TcpListener and StreamSocketListener (because TcpListener is not supported in StoreApps).

Replace custom mix of JSON RPC and HTTP with clean and simple JSON RPC implementation.

Research possibility of using WebSocket client in Inner server instead of SocketStreamListener

Once #57 is done, we could research a possibility of transitioning from InnerServer being a StreamSocketListener to it being a WebSocket client which will start establish duplex connection to a driver on start.
This will let get rid of custom HTTP code on InnerServer side and generally simplify InnerServer. It will also allow us to implement some advanced features, that might require a two-way connection. But this might require a more complex implementation on Driver side for NewSession command which will have to wait for incoming InnerServer connection while blocking response to test client (NewSession should return response to test client only once the InnerServer started and established connection).

https://msdn.microsoft.com/library/windows/apps/br226842

Add support for Hybrid apps

Support for testing of hbrid apps can be done in the same way as it is done in Appium.
We can implement switching between contexts. When in WebView context driver will simple proxy commands and execute InvokeScript on WebView.

Here are docs on InvokeScript https://msdn.microsoft.com/en-us/library/windows.ui.xaml.controls.webview.invokescript.aspx

Also, there is a project of selenium-based driver for Windows Phone internet Explorer:
https://github.com/forcedotcom/windowsphonedriver
This project includes driver, that proxies commands and test app, that has single WebView. commands are executed using InvokeScript and selenium atoms.

Add ability to wait for animations to complete

Click will fail If one tried to click element that is still animating. Either click command should wait for element to stop animation or there should be a command to explicitly wait for animations to finish. Preferable both.

Getting "There is an error in XML document (2,2)".

After making winium setup ready. which looks like
screen shot 2015-09-03 at 2 12 38 pm

The driver is running on port 9999 and test automation server under test in installed i.e by placing "AutomationServer.Instance.InitializeAndStart();" this in AUT Code and also added reference to Inner Server in AUT project

And on running the TestCase getting an error which looks like
screen shot 2015-09-03 at 2 22 45 pm

As i am trying to run my First Test Case using Winium . Am i missing soemthing ? Please help me out.

InnerServer should target WindowsPhone only

Winium.StoreApps.InnerServer is portable, i.e. it targets both Windows and Windows Phone. InnerServer should target only Windows Phone as it is only used in WP apps, and, also, to reduce any possibility of confusion of the purpose of this package on NuGet.

Support implicit wait for find commands

Driver should support setting the amount of time the driver should wait when searching for elements. Currently it fails on first try if no element was found and end user has to use explicit wait from bindings.

Location returns wrong coordinates when apps screen slides up due to some input getting focus

Coordinates returned by location command and used by click command are incorrect when apps screen slid up due to some input being focused, e.g. AutoSuggestBox, which leads to driver tapping incorrect screen point.

See AutoSuggestSample.cs. Replacing temporary execute script solution with suggest.Click() in test case will result in incorrect tap.

Possible solution: use GetClickablePoint for all elements that support it instead of current screen point calculation algorithm.

Simple replacement of location algorithm with GetClickablePoint resulted in test_automation_scroll failing, needs further investigation.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.