andyli / pyextern Goto Github PK

View Code? Open in Web Editor NEW

60.0 8.0 8.0 58.47 MB

Extern generator for the Haxe Python target.

Haxe 99.98% Shell 0.02% Earthly 0.01%

haxe python externs

pyextern's Introduction

pyextern

Extern generator for the Haxe Python target. This repository also includes a set of generated externs.

How to generate extern for a Python lib

Install the Python lib, probably using pip3.
Install the Python dependencies of the generator by pip3 install -r requirements.txt.
Install the Haxe dependencies of the generator by haxelib install build.hxml.
Build the generator by haxe build.hxml
Generate externs by python3 Main.py moduleName1,moduleName2,moduleName3 path/to/output_dir

Generated externs included in this repository

The list of libraries and their versions of the generated externs can be found in requirements-externs.txt. The generated externs can be found in the out directory.

docutils

Docutils is a modular system for processing documentation into useful formats, such as HTML, XML, and LaTeX. For input Docutils supports reStructuredText, an easy-to-read, what-you-see-is-what-you-get plaintext markup syntax.

importlib

The purpose of the importlib package is two-fold. One is to provide the implementation of the import statement (and thus, by extension, the import() function) in Python source code. This provides an implementation of import which is portable to any Python interpreter. This also provides an implementation which is easier to comprehend than one implemented in a programming language other than Python.

Two, the components to implement import are exposed in this package, making it easier for users to create their own custom objects (known generically as an importer) to participate in the import process.

inspect

The inspect module provides several useful functions to help get information about live objects such as modules, classes, methods, functions, tracebacks, frame objects, and code objects. For example, it can help you examine the contents of a class, retrieve the source code of a method, extract and format the argument list for a function, or get all the information you need to display a detailed traceback.

There are four main kinds of services provided by this module: type checking, getting source code, inspecting classes and functions, and examining the interpreter stack.

matplotlib

matplotlib is a python 2D plotting library which produces publication quality figures in a variety of hardcopy formats and interactive environments across platforms. matplotlib can be used in python scripts, the python and ipython shell (ala MATLAB®* or Mathematica®†), web application servers, and six graphical user interface toolkits.

numpy

NumPy is the fundamental package for scientific computing with Python.

pandas

Python Data Analysis Library

pandas is an open source, BSD-licensed library providing high-performance, easy-to-use data structures and data analysis tools for the Python programming language.

pkgutil

This module provides utilities for the import system, in particular package support.

scipy

SciPy (pronounced “Sigh Pie”) is a Python-based ecosystem of open-source software for mathematics, science, and engineering.

seaborn

Seaborn is a Python visualization library based on matplotlib. It provides a high-level interface for drawing attractive statistical graphics.

textwrap

The textwrap module provides two convenience functions, wrap() and fill(), as well as TextWrapper, the class that does all the work, and a utility function dedent(). If you’re just wrapping or filling one or two text strings, the convenience functions should be good enough; otherwise, you should use an instance of TextWrapper for efficiency.

pyextern's People

Contributors

Stargazers

Watchers

Forkers

matthijskamstra carlolson pyzh zaxebo1 clemos skial sonygod a-prokopyev-resume

pyextern's Issues

Infer types based on argument names?

Really glad to stumble across this library! I'm just thinking out loud here, wanted to start a discussion.

I'm realizing that in most dynamic language libraries, the argument name winds up acting like a type of some sort. E.g. an arg of "name" is always a string type in my experience. It might be possible to define some simple rules for defining named argument types on a per-extern basis.

Some examples:

pandas : arguments with name "axis" are always Integer, "skipna" is Boolean, etc.
seaborn : "data" is usually a pandas dataframe or an ndarray type.
scipy : "border_value" is usually an integer (0)

Also, should it be possible to extract all of the base level object methods into some other extendable extern definition? It would really reduce a lot of duplication. I'm talking about init, and operators like le, etc.

Switch the declaration of arguments to python.Kwargs ?

Hello,
This tool is awesome I test it for two days now and it is a relief not to have to code externs each time a new library is added to a project.

I wanted to discuss a point.

Lets say I have generated externs with optional arguments as it is usually the case with python. Most arguments generated by the tool are typed as Dynamic. If I pass an argument to the function in Haxe it will automatically pass it to the first argument of type Dynamic, which in many case you don't want. More precisely you could like to target the second optional argument.

Usually done by passing the null variable to the optional arguments one wants to skip.
However passing the null value to the python function is passing a value nonetheless and it overrides the default value specified in the Python library.
Thus if said function does not have null checking it will end in error.

Here is the way such process could be handled:

Either use Kwargs and type them in the code that consume the externs.

var browser_args:KwArgs<{executable_path:Dynamic, options:Dynamic}> = {executable_path: chromedriver_path, options: chrome_options};
        browser = new WebDriver(browser_args);

having such generated externs:

@:pythonImport("selenium.webdriver.chrome.webdriver", "WebDriver") extern class WebDriver {
	public function new(?executable_path:Dynamic, ?port:Dynamic, ?options:Dynamic, ?service_args:Dynamic, ?desired_capabilities:Dynamic,
		?service_log_path:Dynamic, ?chrome_options:Dynamic, ?keep_alive:Dynamic):Void;
}

Or change the extern like so:

typedef WebDriver_options = {
	var ?executable_path:Dynamic;
	var ?port:Dynamic;
	var ?options:Dynamic;
	var ?service_args:Dynamic;
	var ?desired_capabilities:Dynamic;
	var ?service_log_path:Dynamic;
	var ?chrome_options:Dynamic;
	var ?keep_alive:Dynamic;
}

@:pythonImport("selenium.webdriver.chrome.webdriver", "WebDriver") extern class WebDriver {
	public function new(?options:Kwargs<Webdriver_options>):Void;
}

Would you consider this modification maybe a flag in the externs generation as interesting?

Thanks again for the good tool.

Have a great day.

getting error while creating extern

I wanted to create externs for Flask.
That failed, so I tried something you already did (docutils)

And I got this error

pip3 install docutils
python3 Main.py docutils out
Traceback (most recent call last):
  File "Main.py", line 11, in <module>
    from docutils.frontend import OptionParser as docutils_frontend_OptionParser
  File "/usr/local/lib/python3.5/site-packages/docutils/frontend.py", line 41, in <module>
    import docutils.utils
  File "/usr/local/lib/python3.5/site-packages/docutils/utils/__init__.py", line 20, in <module>
    import docutils.io
  File "/usr/local/lib/python3.5/site-packages/docutils/io.py", line 18, in <module>
    from docutils.utils.error_reporting import locale_encoding, ErrorString, ErrorOutput
  File "/usr/local/lib/python3.5/site-packages/docutils/utils/error_reporting.py", line 47, in <module>
    locale_encoding = locale.getlocale()[1] or locale.getdefaultlocale()[1]
  File "/usr/local/Cellar/python3/3.5.2_1/Frameworks/Python.framework/Versions/3.5/lib/python3.5/locale.py", line 577, in getlocale
    return _parse_localename(localename)
  File "/usr/local/Cellar/python3/3.5.2_1/Frameworks/Python.framework/Versions/3.5/lib/python3.5/locale.py", line 486, in _parse_localename
    raise ValueError('unknown locale: %s' % localename)
ValueError: unknown locale: UTF-8

when will there be a release

till date there is no release and there is nothing on haxelib.org about it.
When it will be released a version formally and uploaded on haxelib or pip3

Attempting to generate externs for Blender's python module throws an error...

Hello...

I run this: python3 Main.py bpy /output/folder

and I get this:

trying to import module: bpy
process module: bpy
process module: bpy.path
:9: (ERROR/3) Unknown interpreted text role "class".
process module: bpy.props
process module: bpy.utils
process module: bpy.utils.units
Error: Not freed memory blocks: 8, total unfreed memory 0.008392 MB

This is the Blender API if you need to see what this is about.

I'm on MacOS.

I built the bpy.so module myself. Here it is if you need it for testing purposes. I couldn't upload here because of the file size, it's 26 MB and zipped. You should put bpy.so into pyextern folder.

And there's this Resources folder, it has to be placed in the pyextern folder's parent directory because bpy.so needs it there. Here it is, Resources.zip I can't upload it here, because it's 19,7 MB.

Thank you very much.

Disabling DCE fixes: 'HxOverrides' has no attribute 'arrayGet'

I was getting this error every single time:

Traceback (most recent call last):
  File "Main.py", line 4547, in <module>
    pyextern_Main.main()
  File "Main.py", line 3188, in main
    main.processModule(modname1,modname1)
  File "Main.py", line 2930, in processModule
    proc.processModule(module,moduleName,self)
  File "Main.py", line 3453, in processModule
    td2 = main.getTd(moduleName,"")
  File "Main.py", line 3052, in getTd
    _g22 = HxOverrides.arrayGet(Reflect.field(td,"meta"), 0)
AttributeError: type object 'HxOverrides' has no attribute 'arrayGet'

After doing some google research I found out it could be related to DCE.
I added "-dce no" to build.hxml and that seems to fix it but I'm still opening an issue because I suspect this could be related to my specific environment, I'm using Anaconda on Mac OS X.

How about add blender python api extern?

how about add blender python api extern?

how to call externs with keyword args

I tried to use pyextern with boto3. it generates the externs but when I try to use them I get "TypeError: ... only accepts keyword arguments" for many of the methods. I suspect this is because pyextern generates dynamic interfaces but boto3 requires KwArgs.

example haxe:

        var session = new Session();
        var dynamo = session.client("dynamodb");
        var request = {Limit: 3};
        var result = dynamo.list_tables(request);

results in this python:

        request = _hx_AnonObject({'Limit': 3})
        result = Reflect.field(self.dynamo,"list_tables")(request)

but it needs to be

        result = Reflect.field(self.dynamo,"list_tables")(Limit=3)

am I doing something wrong? is there a workaround?

AttributeError: type object 'python_Boot' has no attribute 'keywords'

Hi there 👋

I've been trying pyextern and stumbled upon an initialization error (this is raised whatever the arguments):

Traceback (most recent call last):
  File "/home/clement/PERSO/pyextern/Main.py", line 12336, in <module>
    pyextern_Processor.docDefaults = _hx_init_pyextern_Processor_docDefaults()
  File "/home/clement/PERSO/pyextern/Main.py", line 12335, in _hx_init_pyextern_Processor_docDefaults
    return _hx_local_0()
  File "/home/clement/PERSO/pyextern/Main.py", line 12333, in _hx_local_0
    Reflect.setField(v,"report_level",5)
  File "/home/clement/PERSO/pyextern/Main.py", line 692, in setField
    setattr(o,(("_hx_" + field) if ((field in python_Boot.keywords)) else (("_hx_" + field) if (((((len(field) > 2) and ((ord(field[0]) == 95))) and ((ord(field[1]) == 95))) and ((ord(field[(len(field) - 1)]) != 95)))) else field)),value)
AttributeError: type object 'python_Boot' has no attribute 'keywords'

The error seems to occur because statics are initialized in a "wrong" order in the generated file:

def _hx_init_pyextern_Processor_docDefaults():
    def _hx_local_0():
        p = docutils_frontend_OptionParser([docutils_parsers_rst_Parser])
        v = p.get_default_values()
        Reflect.setField(v,"report_level",5)
        return v
    return _hx_local_0()
pyextern_Processor.docDefaults = _hx_init_pyextern_Processor_docDefaults()
pyextern_Process_numpy.__meta__ = _hx_AnonObject({'obj': _hx_AnonObject({'process_modules': ["numpy"]})})
pyextern_Process_pandas.__meta__ = _hx_AnonObject({'obj': _hx_AnonObject({'process_modules': ["pandas"]})})
pyextern_Process_pyqt5.__meta__ = _hx_AnonObject({'obj': _hx_AnonObject({'process_modules': ["PyQt5", "PySide2"]})})
python_Boot.keywords = set(["and", "del", "from", "not", "with", "as", "elif", "global", "or", "yield", "assert", "else", "if", "pass", "None", "break", "except", "import", "raise", "True", "class", "exec", "in", "return", "False", "continue", "finally", "is", "try", "def", "for", "lambda", "while"])
python_Boot.prefixLength = len("_hx_")
python_Lib.lineEnd = ("\r\n" if ((Sys.systemName() == "Windows")) else "\n")

pyextern_Main.main()

_hx_init_pyextern_Processor_docDefaults() is called before python_Boot.keywords is populated.

I'm not sure how to workaround the issue; changing the line
https://github.com/andyli/pyextern/blob/master/src/pyextern/Processor.hx#L37
to

		var v: {report_level: Int} = p.get_default_values();

seems to prevent the failing call to Reflect.setField, so maybe it is enough.

But IMO it is a problem that one can't use Reflect.setField in a static initialization block, so maybe the issue should be raised to the python target 🤷
Anyway, I'll do a PR with the workaround I found.

Thanks 👍