Code Monkey home page Code Monkey logo

sikuli-api's Introduction

Sikuli Java API

Sikuli API for Java provides image-based GUI automation functionalities to Java programmers. It is created and will be actively maintained by Sikuli Lab. The effort of making this API is a response to the desire of many Sikuli users to use Sikuli Script's functionalities directly in their Java programs, rather than writing Jython scripts. This new Java library has a re-designed API and includes several new functions that were not available in the original Sikuli Script.

News

  • April 30, 2013

  • April 8, 2013

  • Mar 6, 2013

    • [Sikuli Slides] (SikuliSlides) - 1.0.0 Release.
  • Nov 18, 2012 Version 1.0.2

    • Relative for defining region/location based on another region/location
    • Canvas for drawing graphical objects on screen
    • MultiStateTarget for finding targets with multiple state-dependent appearances (e.g., checkboxes)

Requirements

  • Java Runtime Environment (JRE) or Java Development Kit (JDK) version 1.6 or newer
  • (Optional) Apache Maven version 2.0.10 or newer, if you want to build it from source.

sikuli-api's People

Contributors

doubleshow avatar kalharbi avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

sikuli-api's Issues

NullPointerException

Original author: [email protected] (January 23, 2013 11:53:45)

What steps will reproduce the problem?

  1. Run the attached maven project

What is the expected output? Something
What do you see instead? Nothing

The output is

SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
Exception in thread "main" java.lang.NullPointerException
at org.sikuli.api.visual.Canvas.addLabel(Canvas.java:230)
at tk.ganian.sikuli.template.main(template.java:35)

Original issue: http://code.google.com/p/sikuli-api/issues/detail?id=22

Sikuli Java API: Why "match.getScore() is not equal to 1.00" when using same image

Original author: [email protected] (January 15, 2013 03:37:24)

Hi, Sikuli guys

I'm currently using some image search method from you library. The class name is "org.sikuli.core.search.ImageSearcher", and the method is "public List<T> search(Query query, Filter<T> filter, int n)".
I pasted some simple code below:
BufferedImage target = new BufferedImage( ); // read some prepared image from computer disk
ImageQuery query = new ColorImageQuery(target);
ImageSearcher searcher = new ImageSearcher(target);
List<RegionMatch> returned_list = searcher.search(query, filter, 2);
for (RegionMatch s : returned_list) {
System.out.println(s.getScore()); // Here is the problem
}

Problem: As you see, the searcher and query is using the same BufferedImage object. So the "s.getScore()" method should return "1.00". But when I tried some different BufferedImage, sometime the result is not equal to "1.00".
Is it a known issue in Sikuli? Or is there something I'm missing?
Please help me take a look at this bug, thanks in advance.

The attachment is one of the example image.

Original issue: http://code.google.com/p/sikuli-api/issues/detail?id=16

Java heap space exhausted with long running script

Original author: [email protected] (November 23, 2012 10:37:26)

What steps will reproduce the problem?

  1. Create a method that waits for a image to appear and then clicks on a position using: mouse.click(foundTarget.getCenter().getRelativeScreenLocation(x, y));.
  2. Run the method in a infinite loop

What is the expected output?
-A method that runs "forever"

What do you see instead?
-A mem usage in the task manager that increases every time the method is executed and eventually a crash with the following exception:
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at java.awt.image.DataBufferByte.<init>(Unknown Source)
at java.awt.image.ComponentSampleModel.createDataBuffer(Unknown Source)
at java.awt.image.Raster.createWritableRaster(Unknown Source)
at javax.imageio.ImageTypeSpecifier.createBufferedImage(Unknown Source)
at javax.imageio.ImageReader.getDestination(Unknown Source)
at com.sun.imageio.plugins.png.PNGImageReader.readImage(Unknown Source)
at com.sun.imageio.plugins.png.PNGImageReader.read(Unknown Source)
at javax.imageio.ImageIO.read(Unknown Source)
at javax.imageio.ImageIO.read(Unknown Source)
at org.sikuli.api.robot.desktop.DesktopScreen.getScreenshot(DesktopScreen.java:35)
at org.sikuli.api.DefaultScreenRegion.capture(DefaultScreenRegion.java:126)
at org.sikuli.api.ImageTarget.getUnordredMatches(ImageTarget.java:122)
at org.sikuli.api.DefaultTarget.doFindAll(DefaultTarget.java:93)
at org.sikuli.api.DefaultScreenRegion._find(DefaultScreenRegion.java:73)
at org.sikuli.api.DefaultScreenRegion.access$0(DefaultScreenRegion.java:69)
at org.sikuli.api.DefaultScreenRegion$RepeatFind.run(DefaultScreenRegion.java:110)
at org.sikuli.api.DefaultScreenRegion.wait(DefaultScreenRegion.java:119)
at OperationExecuter.CreateFastReport(OperationExecuter.java:30)
at BMSVerifierClient.main(BMSVerifierClient.java:61)

What version of the product are you using? On what operating system?
-sikuli-api-1.0.2-standalone.jar running on Windows XP SP3 with Java Version 7 Update 9

Please provide any additional information below.
-The problem is very similar to https://bugs.launchpad.net/sikuli/+bug/1013396

Original issue: http://code.google.com/p/sikuli-api/issues/detail?id=8

ColorImageTarget misses a constructor with File

Original author: [email protected] (December 06, 2012 11:57:30)

In V1.0.2, ColorImageTarget is a specialized ImageTarget but currently you can only create one instance if you have a URL or a BufferedImage.
On the other side, ImageTarget has an additional constructor that can take a File as argument.

For API consistency and for usage convenience, ColorImageTarget should also have a 3rd constructor, with a File argument.

Original issue: http://code.google.com/p/sikuli-api/issues/detail?id=10

The Sikuli 1.0.0 .jar does not contain the FindFailed class from sikuli-script.jar

Original author: [email protected] (October 24, 2012 17:40:58)

What steps will reproduce the problem?

  1. Create a very basic Java class that performs a Sikuli script and import org.sikuli.script.FindFailed;
  2. Wrap your Sikuli script actions in a try-catch block and catch the missing exception like so:
    catch( FindFailed e ) {
    Assert.fail( "Error with Sikuli script: " + e.getMessage() );
    e.printStackTrace();
    throw new AssertionError( e.getMessage() );
    }

What is the expected output? What do you see instead?
Currently this code works fine with the 1.0.0 version of sikuli .jar but in order for it to work I also need to include in my project the version 1.0rc3 Sikuli-script.jar resource. This is because the above mentioned class is missing from the 1.0.0 sikuli jar that was release in mid-October 2012.

What version of the product are you using? On what operating system?

Please provide any additional information below.

Original issue: http://code.google.com/p/sikuli-api/issues/detail?id=2

Can't find dependent libraries AppData\Local\Temp\javacpp173971675343433\jniopencv_core.dll

Original author: [email protected] (January 25, 2013 00:06:48)

What steps will reproduce the problem?

  1. Running the following code on 32 bit jvm
    DesktopCanvas canvas = new DesktopCanvas();
    ScreenRegion sr = new DesktopScreenRegion();
    Target image2 = new ImageTarget(new File("C:\path\to\image\file.png"));
   ScreenRegion fr = //sr;
    sr.find(image2);

2.
3.

What is the expected output? What do you see instead?
ImageTarget should be found

What version of the product are you using? On what operating system?
a) sikuli-api-1.0.2-standalone.jar
b) "c:\Program Files (x86)\Java\jre6\bin\java.exe" -version
java version "1.6.0_31"
Java(TM) SE Runtime Environment (build 1.6.0_31-b05)
Java HotSpot(TM) Client VM (build 20.6-b01, mixed mode, sharing)

  1. Windows 7 professional 64 bit
  2. RAM 8 gb
  3. AMD Phenom II X4 B95 Processor

Please provide any additional information below.

Exception in logs:

AppData\Local\Temp\javacpp173971675343433\jniopencv_core.dll: Can't find dependent libraries
at java.lang.ClassLoader$NativeLibrary.load(Native Method)
at java.lang.ClassLoader.loadLibrary0(Unknown Source)
at java.lang.ClassLoader.loadLibrary(Unknown Source)
at java.lang.Runtime.load0(Unknown Source)
at java.lang.System.load(Unknown Source)
at com.googlecode.javacpp.Loader.loadLibrary(Loader.java:418)
at com.googlecode.javacpp.Loader.load(Loader.java:368)
at com.googlecode.javacpp.Loader.load(Loader.java:315)
at com.googlecode.javacv.cpp.opencv_core.<clinit>(opencv_core.java:131)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Unknown Source)
at com.googlecode.javacpp.Loader.load(Loader.java:334)
at com.googlecode.javacpp.Loader.load(Loader.java:315)
at com.googlecode.javacv.cpp.opencv_core$CvArr.<clinit>(opencv_core.java:158)
at org.sikuli.core.cv.ImagePreprocessor.createGrayscale(ImagePreprocessor.java:52)
at org.sikuli.core.search.algorithm.TemplateMatcher.findMatchesByGrayscaleAtOriginalResolution(TemplateMatcher.java:18)
at org.sikuli.api.ImageTarget.getUnordredMatches(ImageTarget.java:122)
at org.sikuli.api.DefaultTarget.doFindAll(DefaultTarget.java:93)
at org.sikuli.api.DefaultScreenRegion._find(DefaultScreenRegion.java:73)
at org.sikuli.api.DefaultScreenRegion.find(DefaultScreenRegion.java:64)
at

Original issue: http://code.google.com/p/sikuli-api/issues/detail?id=25

Mouse assumes ScreenLocation.getScreen will be a DesktopScreen, but that's not true here

Original author: [email protected] (October 25, 2012 17:54:06)

Here's a snippet of code where the resulting Screen is an anonymous inner class from ScreenRegion.snapshot:

(This is written in Xtend)

    var s = new ScreenRegion
    var m = new Mouse
    m.click(s.find(new TextTarget('HelloWorldExample.java')).center)

Results in:

Exception in thread "main" java.lang.ClassCastException: org.sikuli.api.ScreenRegion$1 cannot be cast to org.sikuli.api.robot.DesktopScreen
at org.sikuli.api.robot.Desktop.getMouse(Desktop.java:111)
at org.sikuli.api.robot.Mouse.getDesktopMouse(Mouse.java:14)
at org.sikuli.api.robot.Mouse.click(Mouse.java:38)
at com.dingrocket.e2e.WebTests.run(WebTests.java:22)
at com.dingrocket.e2e.WebTests.main(WebTests.java:39)

My workaround is to create a new ScreenLocation with the existing one's x and y but the proper DesktopScreen instance

Original issue: http://code.google.com/p/sikuli-api/issues/detail?id=4

ISSUE - DeskTopScreenRegion and DeskTopMouse objects

Original author: [email protected] (January 07, 2013 08:35:26)

In My code i am using DeskTopScreenRegion and DeskTopMouse objects; If script is runninig on same machine functionality is working fine because these objects are refered to execution machine.

If i run the scripts on remote machine(Grid implementation) these objects are not sharing to remote machine hence DesktopMouse and ScreenRegion Actions are not working.

Please let me know if any one have solution to solve this problem.

Original issue: http://code.google.com/p/sikuli-api/issues/detail?id=14

Sikuli Java API: does not run on CentOS

Original author: [email protected] (January 16, 2013 15:06:39)

What steps will reproduce the problem?

  1. when I want run my test
    2.
    3.

What is the expected output? What do you see instead?
I have got error:

tmp/javacpp8897434569464444/libjniopencv_core.so: libopencv_core.so.2.4: cannot open shared object file: No such file or directory

What version of the product are you using? On what operating system?
sikuli-api: 1.0.2, operating system: CentOS 5.5

Please provide any additional information below.
my pom.xml file shortened:
<repositories>
<repository>
<id>javacpp</id>
<name>JavaCPP</name>
<url>http://maven2.javacpp.googlecode.com/git/&lt;/url>
</repository>
<repository>
<id>javacv</id>
<name>JavaCV</name>
<url>http://maven2.javacv.googlecode.com/git/&lt;/url>
</repository>
</repositories>
<dependency>
<groupId>org.sikuli</groupId>
<artifactId>sikuli-api</artifactId>
<version>1.0.2</version>
</dependency>
<dependency>
<groupId>org.sikuli</groupId>
<artifactId>sikuli-webdriver</artifactId>
<version>1.0.1</version>
</dependency>
<dependency>
<groupId>com.googlecode.javacpp</groupId>
<artifactId>javacpp</artifactId>
<version>${javacpp.version}</version>
</dependency>
<dependency>
<groupId>com.googlecode.javacv</groupId>
<artifactId>javacv</artifactId>
<version>${javacv.version}</version>
</dependency>

my test:
SikuliFirefoxDriver driver = new SikuliFirefoxDriver();
driver.get("some url");
Target imageTarget = new ImageTarget(new URL("some url"));
ScreenRegion screenRegion = new DesktopScreenRegion();
ScreenRegion s = screenRegion.find(imageTarget);
Mouse mouse = new DesktopMouse();
mouse.click(s.getRelativeScreenLocation(10, 10));
mouse.click(s.getRelativeScreenLocation(10, 30));
driver.close();

Original issue: http://code.google.com/p/sikuli-api/issues/detail?id=17

How to create an object of SikuliAction class and use it's methods in java

Original author: [email protected] (January 02, 2013 11:58:49)

What steps will reproduce the problem?
I want to create the SikuliAction class object.
SikuliAction sa = new SikuliAction(SikuliAction.ActionType type, Region source, PSC target, ScreenImage screenImage, Match match) ... to create this object requires 4 argument in this first argument is the nested class in protected....

SikuliAction sa = new SikuliAction(SikuliAction.ActionType type, new Screen(), img1,img, match);
what should be provided in the first argument,,,.,,.....

What version of the product are you using? On what operating system?
latest Sikuli and windows 7 os

Please provide any additional information below.
In the creation of the object requires the enum constant that will not be possible because that enum is protected one...

Original issue: http://code.google.com/p/sikuli-api/issues/detail?id=13

How to handle 'hover' in Sikuli Java API

Original author: [email protected] (January 23, 2013 22:51:41)

My question is: How to handle 'hover' in Sikuli Java API (not Sikuli Script)

I am actually trying to hover on a web element (clicking doesn't do the job). In the Sikuli script there used to be a function called 'hover' which did the job, but now after I have switched to Sikuli Java API, I am having difficulty.

Please help!

Thank you.

Original issue: http://code.google.com/p/sikuli-api/issues/detail?id=23

getting an error as at when select click..etc

Original author: [email protected] (March 28, 2013 14:11:21)

What steps will reproduce the problem?
1.In ubuntu it is getting an error as at when select click..etc java.awt.EventDispatchThread.rum(EventDispatchThread.java:138)
2.where as in mac is not showing any error
3.so how to get work in ubuntu

What is the expected output? What do you see instead?
screen should occur to select image or any .
we see only black screen .when we select the selecting box will appear

What version of the product are you using? On what operating system?

sikuil-r931-linux-i686 for linux

Original issue: http://code.google.com/p/sikuli-api/issues/detail?id=39

not working with Windows 64bit

Original author: [email protected] (February 12, 2013 10:05:52)

What steps will reproduce the problem?
1.try to configure sikuli in windows 64 bit
2.not able to find any solution
3.

What is the expected output? What do you see instead?
it has to work fine in 64 bit windows

What version of the product are you using? On what operating system?
window7 64 bit,2 gb ram, 256 gb hard disk

Please provide any additional information below.

Original issue: http://code.google.com/p/sikuli-api/issues/detail?id=31

java.lang.NumberFormatException being thrown when running sikuli slides - (copy and paste screenshot bug)

Original author: [email protected] (March 28, 2013 17:11:25)

What steps will reproduce the problem?
1.Downloaded sikuli-slides
2.Open cmd window
3.Typed java -jar sikuli-slides-1.0.0.jar sikuli.pptx

What is the expected output? What do you see instead?
I was expecting sikuli to at least find the images in the screen
I got this message:
Exception in thread "main" java.lang.NumberFormatException: null
at java.lang.Integer.parseInt(Unknown Source)
at java.lang.Integer.parseInt(Unknown Source)
at org.sikuli.slides.parsing.SlideParser.startElement(SlideParser.java:113)
at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.startElement(Unknown Source)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanStartElement(Unknown Source)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(Unknown Source)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(Unknown Source)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(Unknown Source)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(Unknown Source)
at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(Unknown Source)
at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(Unknown Source)
at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(Unknown Source)
at javax.xml.parsers.SAXParser.parse(Unknown Source)
at javax.xml.parsers.SAXParser.parse(Unknown Source)
at org.sikuli.slides.parsing.SlideParser.parseDocument(SlideParser.java:59)
at org.sikuli.slides.core.SikuliPowerPoint.parseSlideFile(SikuliPowerPoint.java:85)
at org.sikuli.slides.core.SikuliPowerPoint.runSikuliPowerPoint(SikuliPowerPoint.java:48)
at org.sikuli.slides.guis.MainUI.doSikuliPowerPoint(MainUI.java:33)
at org.sikuli.slides.guis.MainUI.main(MainUI.java:43)

What version of the product are you using? On what operating system?
sikuli-slides version 1.0.0. I am using Windows 7 64-bit and create the .pptx file using Microsoft Office Powerpoint 2007.

Please provide any additional information below.

Original issue: http://code.google.com/p/sikuli-api/issues/detail?id=40

Can't terminate application normally

Original author: [email protected] (April 02, 2013 22:58:34)

Rum application using Sikuli Jaava API and let the main thread terminate.

What is the expected output? What do you see instead?
The application should terminate, but since Sikuli uses AWT there is pending AWT thread. This thread won't finish unless all AWT component.

See this for more details: http://docs.oracle.com/javase/7/docs/api/java/awt/doc-files/AWTThreadIssues.html

sikuli-api-1.0.2 in 64-bit JVM under Win7

Sikuli API should provide way to dispose() any AWT components it creates so that application can finish normally and not by calling System.exit.

Ales

Original issue: http://code.google.com/p/sikuli-api/issues/detail?id=44

Feature ideas

Original author: [email protected] (November 22, 2012 22:02:48)

Hello,

I've been working on sikuli-clj (the Clojure wrapper) lately, and I've come across a couple things that would be handy to have in sikuli-api:

  1. Add Mouse click "duration" param
  • the click is too short to register for some apps, so being to specify something like 500 ms between the press and the release would be awesome
    2) Add Mouse.move(ScreenLocation screenLoc)
  • it would be nice to be able to move the mouse independently of pressing/releasing buttons

I can implement these and send a pull request, if you agree that these belong in sikuli-api.

(BTW, is this the best place for discussing features/development?)

Oh, and thank you for such an awesome lib!

-Charles

Original issue: http://code.google.com/p/sikuli-api/issues/detail?id=7

Implement OCR feature in JAVA API of sikuli

Original author: [email protected] (March 06, 2013 04:49:09)

What steps will reproduce the problem?
1.find a region
2.try to get text from that region using Sikuli X.
3.try to get text from that region using JAVA API of sikuli

What is the expected output? What do you see instead?
There is no JAVA API to get/read text from a region

What version of the product are you using? On what operating system?
version : sikuli-api-1.0.2-standalone.jar
OS : Windows XP

Please provide any additional information below.

Original issue: http://code.google.com/p/sikuli-api/issues/detail?id=33

sikuli-slides - not working on a secondary monitor

Original author: khalid3ahmed (March 26, 2013 16:27:11)

What steps will reproduce the problem?

  1. create sequence of slides.
  2. connect the computer to a secondary monitor.
  3. run sikuli-slides on the secondary monitor.

What is the expected output? What do you see instead?
It should find targets on the screen and execute the appropriate actions, but it gave an error message that says "Couldn't find target on the screen."

Please use labels and text to provide additional information.

Original issue: http://code.google.com/p/sikuli-api/issues/detail?id=38

How to handle Target Offset in Sikuli Java API

Original author: [email protected] (January 17, 2013 22:14:55)

My question is: How to handle Target Offset in Sikuli Java API (not Sikuli Script)

I am actually trying to click on the right most part of an image, but the target offset doesn't work. It just clicks on the center of the image instead.

I am trying to use the following code:

DesktopMouse mouse = new DesktopMouse();
ScreenRegion r;
mouse.click(r.getRelativeScreenLocation(121, 0));

Note: I got the target offset numbers from Sikuli IDE.

I would greatly appreciate your help. Thanks.

Original issue: http://code.google.com/p/sikuli-api/issues/detail?id=19

Type Java API does not work when used in Java swing

Original author: [email protected] (April 01, 2013 20:40:09)

What steps will reproduce the problem?

I am trying to use Sikuli stand alone in Java swing. When i was calling the sikuli java api directly i was getting an exception. After doing some google i found that i need to use the apis in thread. So i created a seperate thread for each sikuli api like click and type and used it in bloacking queue. When i call the new type functions say TypeAction by clicking a button in java swing the function stops abruptly in the sikuli type function. I dont get any exception or error. When i call the new TypeAction function via main() the entire function works as expected. In the new TypeAction function i search of nearby 50px region and then click and type. For click i use the first argument of sikuli's type function.

What is the expected output? What do you see instead?
Type api works as expected

What version of the product are you using? On what operating system?
RC3

Please provide any additional information below.

Original issue: http://code.google.com/p/sikuli-api/issues/detail?id=42

Dual Monitor Bug on Application Running on a Secondary Display

Original author: khalid3ahmed (October 21, 2012 00:31:11)

What steps will reproduce the problem?

  1. Connect your computer to a secondary display (Don't mirror the displays)
  2. Try to run the HelloWorld Example on the secondary display.

What is the expected output?
The application should work as it normally does on a single monitor.

What do you see instead?
The application crashes with NullPointerException.

Please use labels and text to provide additional information.

Original issue: http://code.google.com/p/sikuli-api/issues/detail?id=1

TargetEventListener.targetVanished() method bug

Original author: miglecz (January 24, 2013 21:28:47)

TargetEventListener.targetVanished()

That method is not receiving an event when:

  1. target is not on screen
  2. i register an event listener to watch for vanish
  3. it is never ever calling targetVanished() but it should imidiately do at creation

I recommend chaging file VisualEventManager.java to the attached file i revamped. Simpler logic for target change detection.

Original issue: http://code.google.com/p/sikuli-api/issues/detail?id=24

Sikuli java api does not support non english keyboard layout

Original author: [email protected] (April 12, 2013 11:47:54)

    &lt;groupId&gt;org.sikuli&lt;/groupId&gt;
    &lt;artifactId&gt;sikuli-api&lt;/artifactId&gt;
    &lt;version&gt;1.0.2&lt;/version&gt;

Running on Windows 7 64 bit.

I'm using an azerty layout (standard in Belgium):

new DesktopKeyboard().type("123");
it enters the following: &é"

These symbols are on the same keys as 123, but are normally activated by pressing SHIFT.

When I switch my keyboard layout in windows to English US everything works fine.

Philippe

Original issue: http://code.google.com/p/sikuli-api/issues/detail?id=47

memory leak?

Original author: miglecz (January 19, 2013 06:41:04)

I run my app for a few hours and memory runs out and terminates.

stacktrace:
java.lang.OutOfMemoryError: Java heap space
at java.awt.image.DataBufferByte.<init>(Unknown Source)
at java.awt.image.ComponentSampleModel.createDataBuffer(Unknown Source)
at java.awt.image.Raster.createWritableRaster(Unknown Source)
at javax.imageio.ImageTypeSpecifier.createBufferedImage(Unknown Source)
at javax.imageio.ImageReader.getDestination(Unknown Source)
at com.sun.imageio.plugins.png.PNGImageReader.readImage(Unknown Source)
at com.sun.imageio.plugins.png.PNGImageReader.read(Unknown Source)
at javax.imageio.ImageIO.read(Unknown Source)
at javax.imageio.ImageIO.read(Unknown Source)
at org.sikuli.api.robot.desktop.DesktopScreen.getScreenshot(DesktopScreen.java:35)
at org.sikuli.api.DefaultScreenRegion.capture(DefaultScreenRegion.java:126)
at org.sikuli.api.ImageTarget.getUnordredMatches(ImageTarget.java:122)
at org.sikuli.api.DefaultTarget.doFindAll(DefaultTarget.java:93)
at org.sikuli.api.MultiStateTarget.getUnordredMatches(MultiStateTarget.java:27)
at org.sikuli.api.DefaultTarget.doFindAll(DefaultTarget.java:93)
at org.sikuli.api.DefaultScreenRegion.findAll(DefaultScreenRegion.java:58)
...

In file DesktopScreen.java:35 sikuli source is:
InputStream is = new ByteArrayInputStream(os.toByteArray());
return ImageIO.read(is);

Sikuli is not closing the 'is' stream here around. Comment in ImageIO.read(InputStream input) says that:
"This method does not close the provided
InputStream after the read operation has completed
it is the responsibility of the caller to close the stream, if desired."

Is it closed elsewhere, it is intended as it is now or my memory leaks out elsewhere?

Original issue: http://code.google.com/p/sikuli-api/issues/detail?id=20

mvn test fails at testExtractInteger

Original author: [email protected] (January 31, 2013 13:54:11)

What steps will reproduce the problem?

  1. mvn test

What is the expected output? What do you see instead?

T E S T S

Running org.sikuli.api.ColorImageTargetTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.65 sec
Running org.sikuli.api.ImageTargetTest
Tests run: 14, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.759 sec
Running org.sikuli.api.MultiStateTargetTest
Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.433 sec
Running org.sikuli.api.ScreenRegionTest
Tests run: 2, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 0.909 sec <<< FAILURE!

Results :

Tests in error:
testExtractInteger(org.sikuli.api.ScreenRegionTest): For input string: ""

What version of the product are you using? On what operating system?
1.0.2

Please provide any additional information below.

Original issue: http://code.google.com/p/sikuli-api/issues/detail?id=27

"No X11 DISPLAY variable was set" error came on Linux using Sikuli Stand Alone api

Original author: [email protected] (March 30, 2013 16:19:18)

What steps will reproduce the problem?

  1. I Created a sample script like one below on my GNU/Linux machine using the Eclipse IDE. I added the sikuli standalone .jar.

import java.io.File;
import java.net.MalformedURLException;
import java.net.URL;

import javax.swing.plaf.basic.BasicSplitPaneUI.KeyboardEndHandler;

import org.sikuli.api.*;
import org.sikuli.api.robot.Keyboard;
import org.sikuli.api.robot.Mouse;
import org.sikuli.api.robot.desktop.DesktopKeyboard;
import org.sikuli.api.robot.desktop.DesktopMouse;
import org.sikuli.api.visual.Canvas;
import org.sikuli.api.visual.DesktopCanvas;

import static org.sikuli.api.API.*;

public class FirstStandAlone
{

    public static void main(String[] args) throws MalformedURLException, InterruptedException 
    {           

            // Open the main page of Google Code in the default web browser
            browse(new URL(&quot;MYAPP&quot;));


            // Create a screen region object that corresponds to the default monitor in full screen 
            ScreenRegion s = new DesktopScreenRegion();
            System.out.println(&quot;jiuiuiuddfcscdesi&quot;);

            ScreenRegion result = s.wait(new ImageTarget(new File(&quot;//home//dgangwar/LoginFields.PNG&quot;)), 5000);
    System.out.println(null==result);
    System.out.println(&quot;jiuiuiui&quot;);

// //Target imageTarget = new ImageTarget(new File(DataFile.USERIDTEXT));
Mouse mouse = new DesktopMouse();
mouse.click(result.getCenter());

    Mouse mouse = new DesktopMouse();
            mouse.click(result.getCenter());
            Keyboard kb = new DesktopKeyboard();
            kb.type(&quot;Hijhiih&quot;);

    }

}

  1. After running this program I got the below error.

    sikuli main&quot; java.lang.UnsatisfiedLinkError:
    
  2. I added the Path and Sikuli_Home variable to my Eclipse Run configuration, after this the above error did not come.

  3. Now I ran the Test case and got the below error. I have set up my VNC with 3 screen resolutions.

    Exception in thread "main" java.awt.HeadlessException:
    No X11 DISPLAY variable was set, but this program performed an operation which requires it.
    at java.awt.Desktop.getDesktop(Desktop.java:142)
    at org.sikuli.api.API.browse(API.java:20)
    at FirstStandAlone.main(FirstStandAlone.java:18)

What is the expected output? What do you see instead?

I used the same code on windows and it worked fine but on Linux I had the above problem.

What version of the product are you using? On what operating system?

GNU/LINUX , Sikuli-api-1.0.2-standalone.jar, Java SE 6

Please provide any additional information below.

Original issue: http://code.google.com/p/sikuli-api/issues/detail?id=41

java.lang.UnsatisfiedLinkError: /tmp/javacpp94459315877854/libjniopencv_core.so: libopencv_core.so.2.4: cannot open shared object file

Original author: [email protected] (February 01, 2013 06:06:34)

What steps will reproduce the problem?
1.Running the following in maven project gives the error
ScreenRegion screen = new DesktopScreenRegion();
ImageTarget target = new ImageTarget(compareImage);
List<ScreenRegion> r = screen.findAll(target);

What is the expected output? What do you see instead?
Works fine on Windows 7 64 bit.
Throws an error only on linux 64 bit java.lang.UnsatisfiedLinkError: /tmp/javacpp94459315877854/libjniopencv_core.so: libopencv_core.so.2.4: cannot open shared object file

What version of the product are you using? On what operating system?
Using linux 64 bit RedHat

Please provide any additional information below.
I see a special instruction link for linux(x64) but leads to a page without the special instructions. Can you please update the special instructions.

Original issue: http://code.google.com/p/sikuli-api/issues/detail?id=28

I can't get the hello world example to work.

Original author: [email protected] (March 23, 2013 06:10:47)

What steps will reproduce the problem?
1.Run the Quick start code
2. Wait for the browser to open webpage
3.Exception in thread "main" org.sikuli.api.SikuliRuntimeException: Image file can not be loaded from http://code.google.com/images/code_logo.gif because Can't get input stream from URL!
at org.sikuli.api.ImageTarget.<init>(ImageTarget.java:46)
at HelloWorldExample.main(HelloWorldExample.java:25)

What is the expected output? What do you see instead?

Displaying "Hello World" next to the target and then clicking on it.

What version of the product are you using? On what operating system?
windows 7 32bit
jdk1.7.0_09
sikuli-api-1.0.2-standalone.jar

Please provide any additional information below.

I am using the following code

--- Code

import java.net.MalformedURLException;
import java.net.URL;
import org.sikuli.api.Target;
import org.sikuli.api.;
import org.sikuli.api.robot.Mouse;
import org.sikuli.api.robot.desktop.DesktopMouse;
import org.sikuli.api.visual.Canvas;
import org.sikuli.api.visual.DesktopCanvas;
import org.sikuli.api.ImageTarget;
import static org.sikuli.api.API.
;

public class HelloWorldExample {

    public static void main(String[] args) throws MalformedURLException {

            // Open the main page of Google Code in the default web browser
           API.browse(new URL(&quot;http://code.google.com&quot;));

            // Create a screen region object that corresponds to the default monitor in full screen 
            ScreenRegion s = new DesktopScreenRegion();

            // Specify an image as the target to find on the screen
            URL imageURL = new URL(&quot;http://code.google.com/images/code_logo.gif&quot;);                
            Target imageTarget = new ImageTarget(imageURL);

            // Wait for the target to become visible on the screen for at most 5 seconds
            // Once the target is visible, it returns a screen region object corresponding
            // to the region occupied by this target
            ScreenRegion r = s.wait(imageTarget,5000);

            // Display &quot;Hello World&quot; next to the found target for 3 seconds
            Canvas canvas = new DesktopCanvas();
            System.out.println(&quot;r: &quot; + r); 
            canvas.addLabel(r, &quot;Hello World&quot;).display(5);

            // Click the center of the found target
            Mouse mouse = new DesktopMouse();
            mouse.click(r.getCenter());
    }

}

Original issue: http://code.google.com/p/sikuli-api/issues/detail?id=37

Mouse clicking wrong place in dual monitor

Original author: [email protected] (April 15, 2013 20:54:27)

What steps will reproduce the problem?

  1. Search for an image that exists on the secondary monitor (in my case, the one on the right side)
  2. Call mouse.click(region.getCenter())

What is the expected output? What do you see instead?
I expect the mouse to click that image in the center
The mouse does jump to the image on the second monitor (the one on the right), then you can see it move to the same coordinates on the first monitor (the one on the left) as if it thinks the button is on the first monitor

What version of the product are you using? On what operating system?
Windows 7 Ultimate, 64 bit

Please provide any additional information below.

  • Primary monitor is on the left. Secondary monitor is on the right.
  • This seems to not be an issue for Mac OSX
  • It will click the right button if I add an offset to region.getRelativeScreenLocation
  • Sikuli seems to think that my primary monitor is my secondary monitor, and vise versa...I had to call desktop_screen.setScreen(new DesktopScreen(1)) for it to detect the image on my primary monitor. My taskbar is on my left monitor. Also when I goto my monitor setup, windows recognize my left monitor as my main display.

Original issue: http://code.google.com/p/sikuli-api/issues/detail?id=48

Can't get the simple HelloWorldExample example working

Original author: [email protected] (November 30, 2012 11:18:15)

I can't get the hello world example to work.
The following line returns a null value:
ScreenRegion r = s.wait(imageTarget, 5000);

I downloaded two dependencies manually because java complained about missing slf4 and log4j. Why aren't theses mentioned in the dependencies?


---- What steps will reproduce the problem? ----

  1. Run the Quick start code
  2. Wait for the browser to open webpage
  3. wait for another 5 sec for s.wait(imageTarget, 5000);
  4. Se the NullPointerException in the log

--- What is the expected output? ---
Displaying "Hello World" next to the target and then clicking on it.


----What do you see instead? ----

Output:

r: null
Exception in thread "main" java.lang.NullPointerException
at org.sikuli.api.visual.Canvas.addLabel(Canvas.java:230)
at HelloWorldExample.main(HelloWorldExample.java:37)


--- What version of the product are you using? On what operating system?

windows 7 64bit
jdk6 64-bit
sikuli-api-1.0.2-standalone.jar


-- Please provide any additional information below. --

I am using the following code. The complete Eclipse project: http://ge.tt/7HBAtdS/v/0?c


--- Code

import java.net.MalformedURLException;
import java.net.URL;

import org.sikuli.api.API;
import org.sikuli.api.DesktopScreenRegion;
import org.sikuli.api.ImageTarget;
import org.sikuli.api.ScreenRegion;
import org.sikuli.api.Target;
import org.sikuli.api.robot.Mouse;
import org.sikuli.api.robot.desktop.DesktopMouse;
import org.sikuli.api.visual.Canvas;
import org.sikuli.api.visual.DesktopCanvas;

public class HelloWorldExample {

public static void main(String[] args) throws MalformedURLException {

    // Open the main page of Google Code in the default web browser
    API.browse(new URL(&quot;http://code.google.com&quot;));

    // Create a screen region object that corresponds to the default monitor
    // in full screen
    ScreenRegion s = new DesktopScreenRegion();

    // Specify an image as the target to find on the screen
    URL imageURL = new URL(&quot;http://code.google.com/images/code_logo.gif&quot;);
    Target imageTarget = new ImageTarget(imageURL);

    // Wait for the target to become visible on the screen for at most 5 seconds
    // Once the target is visible, it returns a screen region object corresponding
    // to the region occupied by this target
    ScreenRegion r = s.wait(imageTarget, 5000);

    // Display &quot;Hello World&quot; next to the found target for 3 seconds
    Canvas canvas = new DesktopCanvas();
    System.out.println(&quot;r: &quot; + r); 
    canvas.addLabel(r, &quot;Hello World&quot;).display(3);

    // Click the center of the found target
    Mouse mouse = new DesktopMouse();
    mouse.click(r.getCenter());
}

}

Original issue: http://code.google.com/p/sikuli-api/issues/detail?id=9

NullPointerException when doing findAll

Original author: miglecz (January 17, 2013 21:38:12)

Stacktrace:

java.lang.NullPointerException
at java.awt.image.ColorConvertOp.filter(Unknown Source)
at org.sikuli.core.cv.ImagePreprocessor.createGrayscale(ImagePreprocessor.java:51)
at org.sikuli.core.search.algorithm.TemplateMatcher.findMatchesByGrayscaleAtOriginalResolution(TemplateMatcher.java:18)
at org.sikuli.api.ImageTarget.getUnordredMatches(ImageTarget.java:122)
at org.sikuli.api.DefaultTarget.doFindAll(DefaultTarget.java:93)
at org.sikuli.api.MultiStateTarget.getUnordredMatches(MultiStateTarget.java:27)
at org.sikuli.api.DefaultTarget.doFindAll(DefaultTarget.java:93)
at org.sikuli.api.DefaultScreenRegion.findAll(DefaultScreenRegion.java:58)
...
ScreenRegion region = ...;
region.findAll(new ImageTarget(ImageIO.read(new File("img.png"))));

What is this, why is this?

Original issue: http://code.google.com/p/sikuli-api/issues/detail?id=18

Set up for Running Sikuli using Java API on Linux

Original author: [email protected] (April 02, 2013 04:46:28)

What steps will reproduce the problem?
1.I am using Sikuli Java API for Automation On Linux 64 Bits.
2. I am not able to execute the the Script. As X11 variable not set error is coming.
3. I added PATH and SIKULI_HOME to the Run Configuration.

What is the expected output? What do you see instead?

What version of the product are you using? On what operating system?
GNU/LINUX , Sikuli JAVA API, ECLIPSE

For Executing Sikuli from Java API,Do I need to install wmctrl and OpenCV 2.1 as a Pre requisites?
Please provide any additional information below.

Original issue: http://code.google.com/p/sikuli-api/issues/detail?id=43

Draw on Map-v3

Original author: [email protected] (March 18, 2013 09:26:37)

What steps will reproduce the problem?
1.Drawing on map view

What is the expected output? What do you see instead?
User should be able to draw on map-v3

What version of the product are you using? On what operating system?
XP-sp2 with Firefox 19.0.2

Please provide any additional information below.
System does able to click on specific pin point but fail to draw on map where custom programming done,
Working fine on manual process of drawing on map.

http://screencast.com/t/VSPE0a7y5EwK

Original issue: http://code.google.com/p/sikuli-api/issues/detail?id=35

ImageElement.click() doesn't do anything (although image is recognized)

Original author: [email protected] (March 17, 2013 17:32:56)

What steps will reproduce the problem?

  1. Create a simple application with Sikuli that recognizes an image in a page and clicks it

SikuliFirefoxDriver driver = new SikuliFirefoxDriver();
driver.get("http://www.wesite.com/");

ImageElement image = driver.findImageElement(new File("/image/path").toURI().toURL());
image.click();

  1. Run it
  2. Nothing happens

What is the expected output? What do you see instead?
By debugging, I can see that the ImageElement is recognized (it's x and y values are correct) but clicking on the element doesn't do anything.
I've tried several buttons in the same page (both HTML and within Flash content) but nothing never happens.

What version of the product are you using? On what operating system?
Sikuli webdriver 1.0.1 over Ubuntu 12.04 64 bit. OpenCV 2.4.

Original issue: http://code.google.com/p/sikuli-api/issues/detail?id=34

Implement SikuliWebDriver element finding as a "By" subclass

Original author: simon.m.stewart (January 31, 2013 10:50:11)

The mechanism that webdriver supplies for extending the set of locators it can use is to subclass the By class. It would be nice if sikuli-webdriver implemented this:

public class BySikuli {
public static By byImage(final String url) {
return new By() {
public WebElement findElement(SearchContext context) {
// Sikuli magic here
}

  public List&lt;WebElement&gt; findElements(SearchContext context) {
    // Sikuli magic here
  }
}

}
}

It could then be used like so:

driver.findElement(byImage(myUrl));

Original issue: http://code.google.com/p/sikuli-api/issues/detail?id=26

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.