bhecquet / seleniumrobot-grid Goto Github PK

linked to issue seleniumRobot#7

linked to issue #32

due to delivery which is done via a zip file instead of a full jar. This has been changed because aspects were not active

When grid hub is started with default parameters, it waits indefintely for free slots on nodes. (newSessionTimeout = -1). So, when sleeniumRobot tries to create a session whereas there is no free slot, request waits for 2 minutes (Socket timeout defined in SR) and then retries.
But, grid hub records all of the new session requests and creates as many sessions as retries even if SR request went to timeout. This lead to staled sessions.

Add hability to clic left, clic right, upload file, send keys, write text using robot
This is the first step to selenium 3.11 migration

see: bhecquet/seleniumRobot#40
Use the same code to get PID, but be sure not to reference the browser of an other session (maybe through semaphore around session creation if possible to be sure only one driver is created at a time

add

• nodeTags
• browsers
• capacity by browser
• driverVersion

Allow, via option "restrictToNodeTags" the execution of test on a node only to tests which request the nodeTags in their options

Updating each grid node may be a pain when there are several.
We need to update:

• seleniumRobot-grid which also contains drivers
• selenium grid jar which may be updated to support a newer version of selenium

Would allow, when starting a test to pin to a node in grid mode

There should be some impact to migrate
see the same issue for seleniumRobot as both migration should be made in parallel

driver path is now recognized in configuration https://seleniumhq.github.io/docs/grid.html
{ "browserName": "internet explorer", "maxInstances": 1, "platform": "WINDOWS", "webdriver.ie.driver": "C:/Program Files (x86)/Internet Explorer/iexplore.exe" }

Replace DriverRegisterAspect by implementing DriverProvider interface for mobile drivers. May not be possible as DefaultDriverProvider is explicitely called

-enablePassThrough option that could prevent some driver override to work => test desktop capture for example

In order to cleanly start the grid and remove old handles, kill all browser processes before starting grid

For now, if a node disappear and we call StatusServlet, request is long and we get an unkown status for this node

This is useless, so do not build node status if hub cannot connect to it

enablePassThrough mode will be removed in future selenium version so relying on it is a problem
Replace all features linked to CustomEventFiringWebDriver call by servlets (mouse and keyboard use, desktop capture, ...)
Also CustomDriverProvider and AppiumDriverProvider should not be used anymore so their features must be replaced by servlet calls: killing driver processes, starting and stopping appium

If driver crashes for some reason, hub detects it and stop video capture immediately which deletes the file
So when robot try to get it, file does not exist anymore

move file deletion to the "clean" method and keep video files for 8 hours

Add a Servlet / update existing servlet that places hub into a shutdown mode. In this mode, current sessions will still be served, but no new session will be allowed

Look if it's possible to forward this shutdown state to seleniumRobot so that displayed error is not "no slot available"

The new UI should provide, for each node

• list of running sessions
• history of sessions with logging
• node information: RAM, CPU
• screen resolution
• screenshot of the node (1 fps)

• Node connected
• Node do not reply to request
• GET StatusServlet/

Expected:
hub status returns 200 HTTP code

Current
hub status returns 500 HTTP code

Calling InternetExplorerDriver is raising error because it does not recognize se:CONFIG_UUID capability which is added by hub

17:49:17.812 INFO - Executing: [new session: Capabilities {acceptSslCerts: true, browserName: internet explorer, ensureCleanSession: true, ie.ensureCleanSession: true, ignoreProtectedModeSettings: true, ignoreZoomSetting: true, initialBrowserUrl: about:blank, javascriptEnabled: true, proxy: {proxyType: autodetect}, se:CONFIG_UUID: 36e2ff67-6c03-47bc-9264-7f9..., takesScreenshot: true, version: , webdriver.ie.driver: D:/Dev/seleniumRobot-grid/d...}])
17:49:17.812 INFO - Creating a custom new session for Capabilities {acceptSslCerts: true, browserName: internet explorer, ensureCleanSession: true, ie.ensureCleanSession: true, ignoreProtectedModeSettings: true, ignoreZoomSetting: true, initialBrowserUrl: about:blank, javascriptEnabled: true, proxy: {proxyType: autodetect}, se:CONFIG_UUID: 36e2ff67-6c03-47bc-9264-7f9..., takesScreenshot: true, version: , webdriver.ie.driver: D:/Dev/seleniumRobot-grid/d...}
17:50:33.596 WARN - Exception thrown
org.openqa.selenium.WebDriverException: java.lang.reflect.InvocationTargetException
Build info: version: '3.8.1', revision: '6e95a6684b', time: '2017-12-01T18:33:54.468Z'
System info: host: 'SN782980', ip: '172.23.21.176', os.name: 'Windows 7', os.arch: 'amd64', os.version: '6.1', java.version: '1.8.0_131'
Driver info: driver.version: unknown
at com.infotel.seleniumrobot.grid.CustomDriverProvider.callConstructor(CustomDriverProvider.java:102)
at com.infotel.seleniumrobot.grid.CustomDriverProvider.newInstance(CustomDriverProvider.java:56)
at org.openqa.selenium.remote.server.DefaultDriverFactory.newInstance(DefaultDriverFactory.java:87)
at org.openqa.selenium.remote.server.DefaultSession$BrowserCreator.call(DefaultSession.java:171)
at org.openqa.selenium.remote.server.DefaultSession.(DefaultSession.java:88)
at org.openqa.selenium.remote.server.DefaultSession.createSession(DefaultSession.java:76)
at org.openqa.selenium.remote.server.DefaultDriverSessions.newSession(DefaultDriverSessions.java:72)
at org.openqa.selenium.remote.server.handler.NewSession.handle(NewSession.java:63)
at org.openqa.selenium.remote.server.handler.NewSession.handle(NewSession.java:37)
at org.openqa.selenium.remote.server.rest.ResultConfig.handle(ResultConfig.java:112)
at org.openqa.selenium.remote.server.JsonHttpCommandHandler.handleRequest(JsonHttpCommandHandler.java:191)
at org.openqa.selenium.remote.server.DriverServlet.lambda$handleRequest$0(DriverServlet.java:261)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at com.infotel.seleniumrobot.grid.CustomDriverProvider.callConstructor(CustomDriverProvider.java:94)
... 16 more
Caused by: org.openqa.selenium.SessionNotCreatedException: No matching capability sets found.
Unable to match capability set 0: se:CONFIG_UUID is an unknown extension capability for IE
Build info: version: '3.8.1', revision: '6e95a6684b', time: '2017-12-01T18:33:54.468Z'
System info: host: 'SN782980', ip: '172.23.21.176', os.name: 'Windows 7', os.arch: 'amd64', os.version: '6.1', java.version: '1.8.0_131'
Driver info: driver.version: InternetExplorerDriver
remote stacktrace:
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.openqa.selenium.remote.W3CHandshakeResponse.lambda$new$0(W3CHandshakeResponse.java:57)
at org.openqa.selenium.remote.W3CHandshakeResponse.lambda$getResponseFunction$2(W3CHandshakeResponse.java:104)
at org.openqa.selenium.remote.ProtocolHandshake.lambda$createSession$0(ProtocolHandshake.java:123)
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at java.util.Spliterators$ArraySpliterator.tryAdvance(Spliterators.java:958)
at java.util.stream.ReferencePipeline.forEachWithCancel(ReferencePipeline.java:126)
at java.util.stream.AbstractPipeline.copyIntoWithCancel(AbstractPipeline.java:498)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:485)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.FindOps$FindOp.evaluateSequential(FindOps.java:152)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.findFirst(ReferencePipeline.java:464)
at org.openqa.selenium.remote.ProtocolHandshake.createSession(ProtocolHandshake.java:126)
at org.openqa.selenium.remote.ProtocolHandshake.createSession(ProtocolHandshake.java:73)
at org.openqa.selenium.remote.HttpCommandExecutor.execute(HttpCommandExecutor.java:142)
at org.openqa.selenium.remote.service.DriverCommandExecutor.execute(DriverCommandExecutor.java:83)
at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:601)
at org.openqa.selenium.remote.RemoteWebDriver.startSession(RemoteWebDriver.java:219)
at org.openqa.selenium.ie.InternetExplorerDriver.run(InternetExplorerDriver.java:223)
at org.openqa.selenium.ie.InternetExplorerDriver.(InternetExplorerDriver.java:215)
at org.openqa.selenium.ie.InternetExplorerDriver.(InternetExplorerDriver.java:160)
... 21 more
17:50:33.596 WARN - Exception: No matching capability sets found.
Unable to match capability set 0: se:CONFIG_UUID is an unknown extension capability for IE
Build info: version: '3.8.1', revision: '6e95a6684b', time: '2017-12-01T18:33:54.468Z'
System info: host: 'SN782980', ip: '172.23.21.176', os.name: 'Windows 7', os.arch: 'amd64', os.version: '6.1', java.version: '1.8.0_131'
Driver info: driver.version: InternetExplorerDriver
remote stacktrace:

currently, if session fails to be created, it's the standard selenium timeout which is used (540 secs)
no driver takes so much time to create
reduce it to 90 secs

Needs to have a screen unlocked and does not work on virtual machines

For now, there is no way to know how the grid is used: how many slots / nodes are used, how many are available
This would be useful to know so that we can avoid too many session requests at the same time.

A way is to store data in a database, but this needs some improvements and change the way grid is installed
An other way is to create usage files which can be read by grid
API would allow to request these data (new UsageServlet)
GuiServlet would display statistic usages

Allow user, when starting grid node, to set tags (e.g: location, type of network, ...)
these tags will be set via option: -tags <tag1>,<tag2>

CustomCapabilityMatcher will use these tags if the selenium client request one
If client do not specify a tag, it's ignored

Expose a way to start a program on grid node
Use cases are business applications, soap ui, ...

NodeTaskServlet will serve this action

On start, grid reads all environment variables and searchs for SELENIUM_TOOL_ prefixed variables.
Grid record the name (without prefix) which is the tool name that will be exposed to clients and the value which is the path to executable

These tools will be added as capabilities to slots
CustomCapabilityMatcher will have to take tools into account when selecting slot

Servlet exposes "startProgram" action with 2 parameters:

• name: the name of the program to start (the one referenced in environment variables)
• params: parameters to add to program start
  It also records program PID so that it can be killed
  It returns an ID

Servlet exposes "stopProgram" action with 1 parameter:

• id: the id of the program to stop
  It uses the recorded PID to stop program

When stopping session, all remaining running programs are stopped

To do after: bhecquet/seleniumRobot#56

get installed browser version

Currently, sending
http://gridhub:4444/grid/admin/StatusServlet?jsonpath=$['someinvalidkey'] raises a 500 error

configure JsonPath so that it returns null (option DEFAULT_PATH_LEAF_TO_NULL)

move should be positive or negative so that mouse remains almost in the same position
No mode than 1 px so that picture detection and actions are not impacted

Through API, it's possible to disable a node using NodeStatusServlet POST request
allow this with a button on GuiServlet.
This button would have 2 states (Activate & Deactivate) depending on the current node state

Trying to launch a mobile test on seleniumRobot grid leads to error

grid should log to rotating files so that some debug can be done
These logs should be visible inside the GuiServlet

what about the security as this gives access to the server

When a test scenario needs to start more than 1 browser (bhecquet/seleniumRobot#231), these browser should be started on the same node to mimic real world applications

add a processList command to NodeTaskServlet, which takes the name parameter
It returns the comma separated list of pids whose name is the name given as parameter

SeleniumRobot allows to execute host based actions:

• upload file using Robot class
• do desktop screenshot

Add a servlet that could allow these actions remotely

By default, http client on grid is set to 3 hours
So, if a node stops responding (blocking all incoming request, hub is also frozen. Even console do not reply

to improve reliability