Comments (14)
Also tested on a fast PC with default X-server, same problem, so the speed seems not to be related to this issue.
I just found a slight hint to a possible cause for this. Here https://bugs.launchpad.net/sikuli/+bug/1025991/comments/6 it seems from what is reported that removing the timeout given to the click function removed the meta keys from being held. While this seems to reduce the likelyness of meta keys being held (according to the logs), I do observe the issue (that no click event reaches the app) also for click() without a timeout specified. What does help is to follow with a click(Mouse.at())
, e.g.
click(Pattern("clickpos.png").similar(0.95).targetOffset(0, -10), 100) # text field needs click to get focus
click(Mouse.at()) # just to make sure that text field got a click
from sikulix1.
What do you intend by the 100 as second parameter of the click()?
The parameter is taken as specifying the modifier key(s) to be pressed while clicking.
It should be one of the KeyModifier.SHIFT, .ALT, .CTRL, .META (Mac: CMD, Win: WIN)
or a sum of more of them.
So your 100 leads to the press-and-hold of META (ignored on Linux currently. Should hold CTRL I think - so a bug?) and the ALT-GRAPH key (right ALT key I think).
I do not think this is your intention ;-)
So IMHO this should work:
click(Pattern("clickpos.png").similar(0.95).targetOffset(0, -10))
BTW: your above workaround should have been
hover(Pattern("clickpos.png").similar(0.95).targetOffset(0, -10), 100) # only move the mouse
click(Mouse.at())
... which of course would have given you an error about the 2nd parameter ,100
;-)
from sikulix1.
... and I admit, that the logging in this case should be improved ;-)
from sikulix1.
Many thanks @RaiMan for your reply and pointing out that the click function does not take a timeout as second parameter but instead modifies. Not sure where I got it from because it is documented all correctly but it seems the very same mistake was made here: https://bugs.launchpad.net/sikuli/+bug/1025991/comments/6. Probably a remainder from a former wait()
or exists()
and the general python disadvantage that it does not check for types in function calls.
from sikulix1.
Hope it works now.
I will revise the handling of the second click parameter a bit, so it creates an error message if the given value does not make any sense (value can max be 15 if you want to hold all 4 possible keys ).
BTW: was the intention of the 100, that SikuliX should wait max 100 secs for the image to appear?
from sikulix1.
BTW: was the intention of the 100, that SikuliX should wait max 100 secs for the image to appear?
Yes, basically setting the default timeout but just for this call.
Hope it works now.
Sadly not, the actual problem that the running app does sometimes not receive a click event still persists even now where no meta keys are held during click. Only a directly following click(Mouse.at())
works.
Should I re-report that on a new issue since the issue title has changed?
from sikulix1.
app does sometimes not receive a click
There are possible reasons:
- GUI not (yet) ready (the click is ignored)
- app does not have focus (the click only brings the window to foreground)
Since SikuliX currently does not have a feature to "know", wether a click really happened against the intended app/GUI, it is very hard, to debug the reason behind.
So if it is really a problem for you, you should create your own click function, that divides the click into its possible steps and allows some intermediate waits and logs.
- search an image using wait(image, timeout)
- if found:
- hover (move mouse) to the target
- wait a short time if needed (up to 0.5 secs)
- click(Mouse.at())
- wait a short time if needed
- check for expected GUI changes (meaning click worked)
Should I re-report that on a new issue
If it only happens sometimes, but sometimes works, it surely is not a SikuliX problem.
So no extra issue.
But feel free, to post it on Launchpad as a question, so others have a chance, to talk about their experiences. Generally it is very hard to really help in such situations, since only you know/see what happens.
If you finally think it is a SikuliX problem: feel free to report it here or on Launchpad with as much information as possible about environment, app context and script snippets.
from sikulix1.
Many thanks @RaiMan for looking into this. I will test how far I can get with a specialized click()
function. I have the hope that waiting between hover and click(Mouse.at())
might help.
Is there already a delay in the standard click()
function between reaching the target position and issuing the click event?
from sikulix1.
Is there already a delay in the standard ...
No, not an intended wait, only some milliseconds caused by the internal handling (Java AWT Robot).
from sikulix1.
I can certainly confirm that this issue also exists in robotjs independent form sikulix. I can reproduce the same scenario in both applications.
However, with the earlier sikulix jar somehow it avoided the robotjs issue -- at that time I can reproduce this issue only with robotjs. With sikulix the same scenario magically worked.
But now it breaks in both.
The test scenario involves clicking the image of a drop down list. In my case is the Selenium console "Create a New Session" modal browser drop-down. See the picture in this link:
https://raw.githubusercontent.com/Yash-777/SeleniumDriverAutomation/master/images/wiki/NON_GRID.png
A few weeks ago using sikulix I am able to see the drop-down list. Now I cannot, because the click to display the drop-down does not register by the browser. It shows a red shadow as if the drop-down will display.
The same scenario has been failing with robotjs since the beginning. As I am digging more I will file this bug to robotjs.
I am not 100% sure if the change of behavior on sikulix side is due to change is sikulix jar file or due to I updated the npm and node environment. I will try to restore to the same npm/node and try out again.
Will keep you updated here.
from sikulix1.
Hey, I have confirmed that roll back java, npm and node versions doesn’t not help, so the only last resort is to get an older version of the 1.1.4 jar file.
@RaiMan, can you provide me a link to download a 1.1.4 jar file that was built around December time frame?
from sikulix1.
MP4 for passed on DISPLAY=:0
https://ufile.io/4ddoo
MP4 for failed on xvfb-run --server-num=99
https://ufile.io/24ny1
MP4 files can be played by chrome browser directly.
from sikulix1.
btw I have file a ticket to robotjs
octalmage/robotjs#453
from sikulix1.
Correction:
This issue does not exist at first when we run xvfb-run in single run. It occurs when we tried to execute xvfb-run command in background and in parallel (multiple xvfb-run). After that, the issue persist regardless single or parallel xvfb-run.
from sikulix1.
Related Issues (20)
- IDE: NameError after editing script
- IDE: Add utf-8 spec when running a selection of lines
- IDE: A region changed by clicking on region image is still the old one at next run - workaround
- App class: Windows: identify app by process name
- TERMINATING: problem with native library: libopencv_java430.dylib HOT 2
- UI Scaling for HDPI Screens does not work
- isValidImageFilename throws java.lang.IllegalArgumentException
- Issue when trying to connect a tiger vnc server HOT 1
- sikulixIDE 2.0.x ImportError HOT 1
- VS Code extension for SikuliX scripts HOT 1
- Is there a way to generate long clicks (10 seconds +) on actionable elements? HOT 2
- Screen.click(pattern) returns FindFailed error right after screen.has(pattern) returned true. HOT 2
- Latest sikulix1api library contains dependencies with reported security vulmeralibilies
- OCR(region.text()) is detecting wrong text, when text is underlined HOT 6
- Write access to repository's wiki HOT 1
- Wiki cannot be checked out in windows HOT 2
- Exception in thread "Thread-22" java.lang.ExceptionInInitializerError HOT 9
- How to connect database influxdb HOT 1
- Problem with transparency
- Easy way to ignore the background for image search? --- use images with transparent background HOT 7
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from sikulix1.