The jpy JAR should be deployed as an OSGi bundle.
Check: how to deal with native code linked against Python / C API?

This is very important for the upcoming Sentinel Toolbox development.

jpy.array(init, type=jtype|name)

where init is a sequence (tuple, list) or an int providing the requested array length

• How to compile a 64-bit application using Visual C++ 2010 Express? - http://stackoverflow.com/questions/1865069/how-to-compile-a-64-bit-application-using-visual-c-2010-express
• 64-bit build on microsoft visual c++ express 2010 - http://stackoverflow.com/questions/8044385/64-bit-build-on-microsoft-visual-c-express-2010/8334985#8334985
• Configuring Programs for 64-Bit (Visual C++) - http://msdn.microsoft.com/en-us/library/h2k70f3s.aspx

The Java API looks up the (native) jpy Python module from a hidden file $HOME/.jpy. This file contains a link to the jpy shared library, respectively installed by the last jpy setup.
If users have multiple Python installations (e.g. versions 2.7/3.4 or 32-bit / 64-bit platforms), the Java API is unable to locate these as there is currently no configuration parameter for that.

We could either add a parameter to PyLib.startPython() or use a naming pattern for the $HOME/.jpy file, so that the Java PyLib class can select the appropriate Python interpreter interpolated from the JRE architecture and Python version, e.g. ".jpy-py27-amd64".

The current workaround is to simply edit the $HOME/.jpy file to make it point to the wanted jpy Python shared library file.

change source to 1.8 in maven. does not compile

Hi forman:

b = jpy.array('int', [1,2,3])
b.toString()
'[I@3f2a09d5'

But what I want to do is if I got string '[I@3f2a09d5', is it possible to get [1,2,3]. I have spent a lot of time on this, appreciate for your help!

• jpy.array(jtype|name, length)
• jpy.cast(jobj, jtype|name)

Hi,

I need to load a script from specified path and execute one particular function from it. How can I do this using jpy? Could you please give me syntax for the same?

jpy.array(type | type_name, length) ==>

jpy.array(type | type_name, length | initializer)

with initializer being any buffer or sequence (array, list, tuple, ...)

Hi,

I have loaded a script (module) into Python Interpreter using the below

PyModule.extendSysPath(Path_to_Script, true);
PyModule.importModule(Script_Name)

Now I have to remove this module.
How can I do this?
Is there any function in C-API?

Thanks a lot.

Installing jpy on OS X with Python 3.4 gives the following compile error:

src/main/c/jni/org_jpy_PyLib.c:793:42: error: passing 'PyObject'
(aka 'struct _object') to parameter of incompatible type 'PyObject _'
(aka 'struct _object *'); remove *
chars = PyBytes_AsString(_pyObjUtf8);

Move all PyInterpreter methods to PyLib class. Remove PyInterpreter.

Currently we get:

Error: Python error: <class 'AttributeError'>: 'module' object has no attribute 'get_class'
Traceback: <traceback object at 0x000000000CA676C8>

which is not helpful.

The Python interpreter crashes when we call from multiple JVM threads at the same time. See

• PyObjectTest.testCreateProxyAndCallSingleThreaded() --> ok
• PyObjectTest.testCreateProxyAndCallMultiThreaded() --> crash

It is for sure a GIL-related bug. Look here for very similar problems:

• http://stackoverflow.com/questions/15470367/pyeval-initthreads-in-python-3-how-when-to-call-it-the-saga-continues-ad-naus
• http://stackoverflow.com/questions/10625584/embedding-python-in-multithreaded-c-application
• http://stackoverflow.com/questions/4866701/python-pygilstate-ensure-release-causes-segfault-while-returning-to-c-from-p

... so that we can easily test for Java object identity

HI Norman,

Please suggest us as we are facing few Memory issues with usage of jpy in our application.

Context

We are building an Mutli-Language(Script oriented), distributed and muti-threading application in Java and we need to run user Python Scripts from our application (Application also supports R, Groovy).

One functionality flow in our application is that,

1. any can user uploads a python script and intimate the application to publish his python script, the script is loaded and run by a thread and here we are using jpy to handle User Python Scripts related tasks.
2. Now user can kill any thread (meaning, stop his running script) make modifications to it and reload the scripts to application and run them (basically with new threads)

what we have observed is, as user kills and publishes new threads, JVM memory is handled with out any hassle but since I have no way to cleanup the loaded modules and since user uses memory extensive modules like opencv etc, the memory is summing up to an extent where eventually at some point in time application is crashing with memory crunch.

Could you please suggest a right approach to use jpy in this context or is this rite use-case to use jpy?

Thanks in advance.

PyPI: https://pypi.python.org/pypi

Sphinx: http://sphinx-doc.org/

The following problem may or may not occur::

Running src/test/python/jpy_typeconv_test.py
..F
======================================================================
FAIL: test_ToPrimitiveArrayConversion (__main__.TestTypeConversions)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "src/test/python/jpy_typeconv_test.py", line 32, in test_ToPrimitiveArrayConversion
    self.assertEqual(fixture.stringifyIntArrayArg(a), 'int[](1,2,3)')
AssertionError: 'int[](0,0,0)' != 'int[](1,2,3)'
- int[](0,0,0)
?       ^ ^ ^
+ int[](1,2,3)
?       ^ ^ ^

The test code is::

def test_ToPrimitiveArrayConversion(self):
    fixture = self.Fixture()

    # Python int array to Java int array
    a = array.array('i', [1,2,3])
    self.assertEqual(fixture.stringifyIntArrayArg(a), 'int[](1,2,3)')
    ...

Use Py_Main(..), see http://docs.python.org/3/c-api/veryhigh.html

We currently use Py_Initialize() only, see http://docs.python.org/3/extending/embedding.html

Many users have a lot of legacy code depending on Python 2.7 and thus hesitate to move to Python 3:

• http://programmers.stackexchange.com/questions/63859/why-do-people-hesitate-to-use-python-3
• http://stackoverflow.com/questions/8102987/if-python-3-is-the-newest-python-why-is-python-2-still-around

Maven Repository: http://mvnrepository.com/

Currently Java API DOC HTML pages are generated into the doc/_static folder and referenced from a reference.rst. This works fine for a local Sphinx build but doesn't work for RTD, see http://jpy.readthedocs.org/en/latest/reference.html#java-api.

jpy can avoid copying memory from Python buffers to Java primitive array parameters if the parameter is known to be a mere output value of the method (and not read from by the method).

This would make Java method calls with mutable primitive array parameters more efficient.

We currently must explicitely call

jpy.create_jvm(jvm_options)

before we can import Java classes. To avoid this,

• we could let users provide 'jvm_options' in an INI file that is optionally loaded when the JVM is implictely created;
• or we load and execute a user defined 'jpy_conf.py' in order (to construct) to obtain 'jvm_options'.

For the latter, refer to PyRun_String as described here
http://docs.python.org/3/c-api/veryhigh.html?highlight=pyrun_string#PyRun_String

Another (or additional) possibility:

import jpy
jpy.jvm_options = ['-Xmx512M', '-Djava.class.path=target/classes']

jpy.jvm_options can be directly evaluated in the jpy C code before the JVM is implicitely created.

The following problem occurred on Linux (Ubuntu 12.04) when invoking the Python interpreter from Java (here from a BEAM extension module):

The Python extension 'numpy' obviously doesn't know about Python anymore when the Python interpreter is loaded from Java and the interpreter again loads numpy. Note that PyExc_SystemError is a symbol defined in practically all Python shared library versions.

On Windows, this works perfectly.

For example, the 3.3 function PyUnicode_AsUTF8() can be replaced as follows:

   PyObject* pyUTF8 = PyUnicode_AsEncodedString(pyUnicode, "utf-8", "replace");
   char* chars = PyBytes_AsString(pyUTF8);
   ...
   Py_DECREF(pyUTF8);

The following error occured on a Ubuntu 64-bit with JDK 8 and Python 3.4:

--> NULL assignement in JPy_ClearGlobalVars(), jpy_module.c

The installation process of jpy is too complicated and too prone to configuration errors.

This is mainly due to compilation of C and Java sources. Especially the C compilation on Windows is not straight forward. For Windows users, we could generate installers for Python 2.7/3.3/3.4 for both 32 and 64bit executables.They install the jpy Python module (shared library), copy the jpy.jar and write/modify the %HOME%.jpy file and ask for the JRE (or JVM shared library) location.

Users currently must also ensure that the parent directory of a compatible JVM shared library (libjvm.so on Unix, jvm.dll on Windows) is part of the LD_LIBRARY_PATH (Unix) or PATH (Windows) environment variable, so that it can be loaded by the jpy Python module. This step should be avoided or at least simplified, if somehow possible. Note that changing LD_LIBRARY_PATH or PATH from Python has no effect but maybe adding the JVM directory path using sys.path.append(jvm_dir) helps (see http://stackoverflow.com/questions/856116/changing-ld-library-path-at-runtime-for-ctypes) .

Other improvement ideas:

• We don't force users to compile Java at all. We simply bundle a precompiled jpy.jar in the source and binary distributions.
• Avoid setting JAVA_HOME env var: It could be set to JDK_HOME before os.system('mvn ...') is called: os.environ['JAVA_HOME'] = os.environ['JDK_HOME']

Library is a dark magic. No code examples, no stackoverflow questions.
Digging tests won't help - there is nothing

There is a problem with Java collections which all return java.lang.Object items, because Java generic type information is not available at runtime.

So before we convert to a corresponding Python value, we must inferr the actual Object type, beacause usually the Object value is meaningless to Python programmers (unless such values are passed through as arguments to other Java method calls).

In an Apache WSGI environment, jpy crashes when a new object is instantiated using the constructor. For example, the following chain of commands fails:

File = jpy.get_type('java.io.File')
File('somePath')

The first lines of the stacktrace read:

*** glibc detected *** /opt/apache2/bin/httpd: free(): invalid pointer: 0x00007fcb5a1435e0 ***
======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(+0x7eb96)[0x7fcb712d3b96]
/usr/local/lib/python2.7/dist-packages/jpy.so(JPy_AsJString+0x93)[0x7fcb59f30e63]
/usr/local/lib/python2.7/dist-packages/jpy.so(JMethod_CreateJArgs+0x133)[0x7fcb59f38983]
/usr/local/lib/python2.7/dist-packages/jpy.so(JObj_init+0xa5)[0x7fcb59f374a5]
/usr/lib/libpython2.7.so.1.0(+0x142108)[0x7fcb6dbaf108]
/usr/lib/libpython2.7.so.1.0(PyObject_Call+0x53)[0x7fcb6dbae083]
/usr/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x3fcd)[0x7fcb6db081fd]
/usr/lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x855)[0x7fcb6dac96b5]

The same chain of commands runs perfectly fine in a regular python shell.

We currently use the debug flag to switch on verbose mode. This is not useful because we dump to many stuff at once. We need a finer control, e.g.

• off=0
• all=0xff
• mem=0x01 // Python memory alloc/dealloc for Java objects
• type=0x02 // Java type resolution
• call=0x04 // calling of Java methods, Java field access
• meth=0x10 // finding matching Java methods

Use case: Extend Python by Java code.

We must unwrap Java objects (PyObject) passed to PyLib methods to obtain Python object pointers (PyObject*). Otherwise we cannot callback into Python when we call Java from Python and use the jpy Java API in our Java code.

The following code causes Python 2.7 to crash:

String = jpy.get_type('java.lang.String')
String(u'Bibo')

• Allow to pass Python a list, dict, or set where a Java method expects a List or array, a Map, or a Set.
• Implement the Python list, dict, and set protocols where a Java method returns a List, a Map, or a Set.

Here is a Python error caught by the jpy Java library:

And here the Java stacktrace:

The actual problem in this case was that the numpy module could not be found in Python client code.

Example:
group = product.getFlagCodingGroup()
group.add(FlagCoding('quality_flags'))

The first statement returns an ProductNodeGroup, If this class has not been resolved, e.g. via a call to jpy.get_class('org.esa.beam.framework.datamodel.ProductNodeGroup'), it's 'add' method remains unknown.

DoubleArray = jpy.get_class('[D')

will crash here!

arr = DoubleArray()

Am I doing something wrong?

When running mvn clean install I get the below error (after successful 64-bit python setup.py install) but it doesnt say what the missing dependent libraries are. Not sure at all what to do here?

org.jpy.PyLibTest Time elapsed: 0.001 sec <<< ERROR!
java.lang.UnsatisfiedLinkError: C:\workspaces\jpy\build: Can't find dependent libraries
at java.lang.ClassLoader$NativeLibrary.load(Native Method)
at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1929)
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1814)
at java.lang.Runtime.load0(Runtime.java:809)
at java.lang.System.load(System.java:1083)
at org.jpy.PyLib.loadLib(PyLib.java:352)
at org.jpy.PyLib.(PyLib.java:366)
at org.jpy.PyLibTest.setUpClass(PyLibTest.java:27)

this is after i changed the code that was occurring in my other issue about compiling with 1.8.

in pyobject:

public void setAttribute(String name, Object value) {
    assertPythonRuns();
    Class clazz;
    if (value != null) {
        clazz = value.getClass();
    } else {
        clazz = null;
    }
    PyLib.setAttributeValue(getPointer(), name, value, clazz);
}

also had to add this otherwise got the below exception:

            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>2.18.1</version>
                <configuration>
                    <systemPropertyVariables>
                        <jpy.jpyLib>C:\app\python\Lib</jpy.jpyLib> 
                    </systemPropertyVariables>
                </configuration>
            </plugin>


java.lang.ExceptionInInitializerError: null
        at org.jpy.PyLibConfig.getProperty(PyLibConfig.java:117)
        at org.jpy.PyLib.loadLib(PyLib.java:345)
        at org.jpy.PyLib.<clinit>(PyLib.java:366)
        at org.jpy.PyLibTest.setUpClass(PyLibTest.java:27)


python setup final output was:

OK
JPy_free: freeing module data...
JPy_free: done freeing module data
setting environment variable 'PYTHONPATH' to None
jpy binary build is 'build\jpy.win-amd64-3.4.zip'
jpy installed into 'C:\app\python\Lib\site-packages\'

We want to use

from java.io import File

We currently must use

File = jpy.get_class('java.io.File')

Add a new module function 'import' which calls 'get_class' but then creates new modules up the last dot in the class name. E.g. java.io.File will create the module 'java', than create 'io' and add this to 'java' and then add class 'File' to 'io'.

A Java method, e.g.
void someMethod(Object arg)

cannot be called from Python
obj.someMethod('abc')

--> RuntimeError: no matching Java method overloads found

See TODOs in jpy_jobj.c:

implement a special lookup: we need to override getattro of JType (--> JType_getattro) as well so that we know if a method is called on a class rather than on an instance. Using PyObject_GenericGetAttr will also call JType_getattro, but then we loose the information that a method is called on an instance and not on a class.

Let org.jpy.PyLib expose important python process-wide parameters:
http://docs.python.org/3/c-api/init.html#process-wide-parameters
This is important e.g. for retrieving meta-info about the Python version used.

Allow modifying Python sys.path variable (and other sys variables):
http://docs.python.org/3/c-api/sys.html#system-functions
This is important for setting up Python plug-in loading from Java code.