subj

it was discovered that Appium can work fine directly but integration with selenium hub is broken.
We have to find a way to detect that started appium is not registered in hub anymore and restart service again

as workaround before the start we have to rest primary http port to 80:

i.e.
- "8082:80" # primary HTTP port
- "80:80" # primary HTTP port

Need to find a way to make it woarkable as part of the Zebrunner CE and independently

Execute:
ansible-playbook -vvv -i hosts devices.yml --tags "registerDevice"

Error message appears:
fatal: [localhost]: FAILED! => {
"ansible_facts": {},
"changed": false,
"failed_modules": {
"setup": {
"ansible_facts": {
"discovered_interpreter_python": "/usr/bin/python"
},
"deprecations": [
{
"msg": "Distribution Ubuntu 16.04 on host localhost should use /usr/bin/python3, but is using /usr/bin/python for backward compatibility with prior Ansible releases. A future Ansible release will default to using the discovered platform python for this host. See https://docs.ansible.com/ansible/2.9/reference_appendices/interpreter_discovery.html for more information",
"version": "2.12"
}
],
"failed": true,
"module_stderr": "sudo: a password is required\n",
"module_stdout": "",
"msg": "MODULE FAILURE\nSee stdout/stderr for the exact error",
"rc": 1
}
},
"msg": "The following modules failed to execute: setup\n"
}

Currently we have some variables in startNodeSTF start up script. Move these variables to set_selenium_properties

we do verification onto the connected devices per each device in device.txt, for example:
device=instruments -s devices | grep $udid

but it make sense to move outside the while cycle and export to any temp text file all devices/simulators. Inside cycle just verify presence of the concrete uuid

subj

Steps:

1. Open and sign in http://stage.solvd.com/#!/devices
2. Go somewhere and come back In 2 mins
   Actual result: Need to retry connection

in caase someone connected and whitelisted devices with newer OS version which is currently non-supported by XCode we have zombie process which always tries to start WDA.
Investigate if we can organize any exclusion list based on conditions.

Example of the implementation:

1. improve WDA startup script to track failures like

The requested device could not be found because no available devices matched the request.

Testing failed:
        WebDriverAgentRunner:
                testRunner encountered an error (Encountered a problem with the test runner after launch. (Underlying error: Lost connection to DTServiceHub))

** TEST FAILED **

or

** TEST FAILED **

2. as only failure detected exit from startNodeWDA.sh script and put message into the log
3. in several cases add current device id into the exclusion list (extra metadata file or files!). No sense to start WDA if XCode doesn't support it!

review the changes andmerge if possible into our stf related part

as we have 5 min timeout on NGiNX level for mcloud we have to configure less time out for selenium-hub: GRID_NEW_SESSION_WAIT_TIMEOUT, for example 4 min, i.e. 240000

Example
selenium-hub:
image: "qaprosoft/selenium-hub:${TAG_SELENIUM_HUB}"
container_name: selenium-hub
env_file:
- variables.env
environment:
# 4min session wait timeout as 5 min is configured on NGiNX
- GRID_NEW_SESSION_WAIT_TIMEOUT=240000
- GRID_TIMEOUT=30000
ports:
- "4446:4444"
depends_on:
- stf-app
restart: always

Test scenario:

1.Clone mcloud
2.Nano mcloud/docker-compose.yml
3.Change port to "80:80"
4.Open browser and enter ip instance:80/stf

Actual result: At startup mcloud redirects to localhost
Please see screenshot:

zebrunner/stf#171

no need to provide public access to stf node (android and ios) anymore if above ticket is solved^^^

there is no sense to have startNodeAppium.sh in shell scripts. Current naming was born when we had single node script for everything

For now we implemented killing of Appium when WDA is crashed/not unavailabe/not started.
But that produce a lot of forwarding to node crashes on test layer.

So we have to make WDA verification as part of STF sync and kill STF when no appropriate wda service is detected

Hello dear friends,) again have some problem on start,

Prepared all prerequisites

• Installed XCode 12.0.1
• Install Appium 1.18.3 - done
• Install node v13.14.0 (npm v6.14.4) - done
• Choose WebDriverAgentRunner and your device( iphone 7) - done
• Choose your dev certificate - done
• Write my phone UUID to devices.txt

Now if i start Test from xcode, i have no errors, everything fine build success and i see icon on phone. (Then i click Product -> Stop)

If i start ./startNodeWDA.sh 00e1c6ef02e0bce60a363f74a0d94a431d3000000
i will get error in log iPhone_7_wda.log

....
.......
.......
/usr/bin/codesign --force --sign 239551D2306160A055160DF32988B35984674756 --timestamp\=none /Users/myfarmacc/Library/Developer/Xcode/DerivedData/WebDriverAgent-ciegwgvxzxdrqthilmrmczmqvrgu/Build/Products/Debug-iphoneos/WebDriverAgentLib.framework
/Users/myfarmacc/Library/Developer/Xcode/DerivedData/WebDriverAgent-ciegwgvxzxdrqthilmrmczmqvrgu/Build/Products/Debug-iphoneos/WebDriverAgentLib.framework: replacing existing signature
/Users/myfarmacc/Library/Developer/Xcode/DerivedData/WebDriverAgent-ciegwgvxzxdrqthilmrmczmqvrgu/Build/Products/Debug-iphoneos/WebDriverAgentLib.framework: errSecInternalComponent
Command CodeSign failed with a nonzero exit code


Test session results, code coverage, and logs:
    /Users/myfarmacc/Library/Developer/Xcode/DerivedData/WebDriverAgent-ciegwgvxzxdrqthilmrmczmqvrgu/Logs/Test/Test-WebDriverAgentRunner-2020.10.13_21-22-11-+0300.xcresult

Testing failed:
    Command CodeSign failed with a nonzero exit code
    Testing cancelled because the build failed.

** TEST FAILED **


The following build commands failed:
    CodeSign /Users/myfarmacc/Library/Developer/Xcode/DerivedData/WebDriverAgent-ciegwgvxzxdrqthilmrmczmqvrgu/Build/Products/Debug-iphoneos/WebDriverAgentLib.framework
(1 failure)
iPhone_7 device wasn't connected yet. waiting 1 sec...
iPhone_7 device wasn't connected yet. waiting 1 sec...

On top of the log see run command, there is nothing extra:
/Applications/Xcode.app/Contents/Developer/usr/bin/xcodebuild -project /usr/local/lib/node_modules/appium/node_modules/appium-webdriveragent/WebDriverAgent.xcodeproj -scheme WebDriverAgentRunner -destination id=00e1c6ef02e0bce60a363f74a0d94a431d3000000 USE_PORT=20001 MJPEG_SERVER_PORT=20002 test

tried all this solutions:
First one
Second one

• tried to clean reboot, nothing :(

Hope for any help,
Thank's
Mark.

Currently we sync Simulators and Devices using the same script. Let's split this script for simulators and devices. Also, Update check if wda exists to grep real WDA status using curl.

in caase someone connected and whitelisted devices with newer OS version which is currently non-supported by XCode we have zombie process which always tries to start WDA.
Investigate if we can organize any exclusion list based on conditions.

Example of the implementation:

1. improve WDA startup script to track failures like

The requested device could not be found because no available devices matched the request.

Testing failed:
        WebDriverAgentRunner:
                testRunner encountered an error (Encountered a problem with the test runner after launch. (Underlying error: Lost connection to DTServiceHub))

** TEST FAILED **

or

** TEST FAILED **

2. as only failure detected exit from startNodeWDA.sh script and put message into the log
3. in several cases add current device id into the exclusion list (extra metadata file or files!). No sense to start WDA if XCode doesn't support it!

subj

add this conditio to selenium-hub docker container
depends_on:
- stf-app

in case hub started first it can't establish STF connection and all tests failed as no access to adb connect host:port

Open http://stage.solvd.com/
Verify device Model and Product
Actual result: Model and Product are from device
Expected: Model and Product of devices should be from main.yml in stf

Currently we are not able to unlock screen from iSTF.

screenshots can't be captured over the android device via STF.
Earlier it was due to the ownership and permission issue for stf-storage-temp folder

we do process verification by sync service script for appium but it will be great to reuse http call for that

Each device service register as appium url:
http://${publicIp}/stf/appium/${host}/${port}/wd/hub

So using /stf general NGiNX will redirect to stf-proxy service.
stf-proxy service should parse host and port and proxy pass to appium using private network connection

Steps:

1. Request driver session on mcloud from external network
2. If driver not started during timeout - we got exception [#285]
3. Carina tries to occupy the same device
   -> Selenium hub doesn't return device on stf
   -> Huge queue is created in the result

during research we found that STF service should be killed for disconnected device.

Currently we have hardcoded path to WDA. Starting from appium 1.14.1 location had been changed. Now it is /usr/local/lib/node_modules/appium/node_modules/appium-webdriveragent/

sed -i is not supported on mac platforms. we have to reuse replace bash functiona instead. Later it should allow to support Mac platforms for MCloud master host (for now only Linux)

Need single name stf-auth

Currently we don't have in sources and don't create this folder automatically. Branch : ios-slave

real devices verification:
device=/usr/local/bin/ios-deploy -c -t 1 | grep $udid

simulators use:
device=instruments -s devices | grep $udid

"instruments" are rather heavy operation which can't be synced too often.

So need a decision about necessity to split these commands

Hub during the connection uses stf token to establish communication.
Let’s reorganize mcloud making stf as independent sub-module which started first in scope of mcloud

issue with unlocking iOS device is fixed: zebrunner/stf#145

in scope of this ticket let's do a research if we can organize device farm for manual and automation usage with locked screens.
benefits: not used devices sleep and lock display, i.e. no battery use

When we occupy device via STF we have to wake it up.
Also investigate how can we unlock via pin automatically by appium

Hello dear friends,

trying to setup IOS farm, but stack on step

Sign WebDriverAgent using your Dev Apple certificate and install WDA on each device manually

• Install Appium 1.18.3 - done
• Install node v13.14.0 (npm v6.14.4) - done
• Choose WebDriverAgentRunner and your device( iphone 7) - done
• Choose your dev certificate - done

When i try to run test i get error
'CocoaAsyncSocket/GCDAsyncSocket.h' file not found

/usr/local/lib/node_modules/appium/node_modules/appium-webdriveragent/WebDriverAgentLib/Vendor/RoutingHTTPServer/RoutingHTTPServer.m:1:9: In file included from /usr/local/lib/node_modules/appium/node_modules/appium-webdriveragent/WebDriverAgentLib/Vendor/RoutingHTTPServer/RoutingHTTPServer.m:1:

Hope for any help,
Thank's
Mark.

Test scenario:

1.Open ec2 instance link/stf
2.Tap to any device

Actual result: Error: Service is currently unavailable.Try to attempt later after open device

reuse runner pipeline from qps-infra

the is no sense to keep it anymore as branch.
this code should be placed into the independent repo with valid readme just for iOS mobile device farm

Execute:
ansible-playbook -vvv -i hosts devices.yml --tags "registerDevice"

Error message appears:
fatal: [localhost]: FAILED! => {
"ansible_facts": {},
"changed": false,
"failed_modules": {
"setup": {
"ansible_facts": {
"discovered_interpreter_python": "/usr/bin/python"
},
"deprecations": [
{
"msg": "Distribution Ubuntu 16.04 on host localhost should use /usr/bin/python3, but is using /usr/bin/python for backward compatibility with prior Ansible releases. A future Ansible release will default to using the discovered platform python for this host. See https://docs.ansible.com/ansible/2.9/reference_appendices/interpreter_discovery.html for more information",
"version": "2.12"
}
],
"failed": true,
"module_stderr": "sudo: a password is required\n",
"module_stdout": "",
"msg": "MODULE FAILURE\nSee stdout/stderr for the exact error",
"rc": 1
}
},
"msg": "The following modules failed to execute: setup\n"
}

Every sync script write into the log that it was started, finished and even "Nothing to do for..."
To minimize disk usage :) let's keep only started and actual actions...

so we have to remove in every sync script
else echo "Nothing to do for ${udid} - device name : ${name}"

and
echo date +"%T" Script finished

during any chane in variables.env and restarting let's say mcloud-grid docker recreates rethinkdb.
STF itself is very sensitive to rethinkdb so it will be better to keep it up&running as much as possible.

Solution:
remove sharing of variables.env to rethinkdb as not needed. That's suppose to minimize number of db restarts

I'm trying to run stf in iOS but I did not get the real values in the set_selenium_properties.sh file. Can someone explain me what should I set those variables?
-STF_PUBLIC_HOST
-STF_PRIVATE_HOST
-WEBSOCKET_PROTOCOL
-WEB_PROTOCOL
-hubHost
-hubPort
-STF_PUBLIC_NODE_HOST
-STF_PRIVATE_NODE_HOST

OS: android 7.0-9

Description:
While working with different devices sometimes we couldn't use them manually in STF due to grey screen issue. Meanwhile, all automation tests could be executed properly on the same devices.

P.S. From personal observations, this issue is actual, when device's docker container wasn't restarted for a long period of time (2+ weeks)

We have to handle WDA status in syncAppiumSimulators.sh . No sense to start appium if WDA is not ready for simulator. Use curl to check WDA status = success.

mcloud still uses their own ftp server to store video recording. need investigate if we can consolidate s3 compatible storage and ftp using below container or similar:
https://github.com/Factual/open-dockerfiles/tree/master/s3-backed-ftp

For example:
<key>StandardErrorPath</key> <string>/Users/build/tools/mcloud/logs/agent_appium.log</string> <key>StandardOutPath</key> <string>/Users/build/tools/mcloud/logs/agent_appium.log</string>

Let's combine all stdout and stderr in the same file. Ideally relative one like agents.log or mcloud-agetns.log etc.
It is ok even if it will be generated inside ~/Library/LaunchAgents folder

all communication between tcp services better to setup via private ip. Potentially that's could allow to avoid public dns name assigning to private ip