python binding about ev3dev-lang HOT 29 CLOSED

Do you want to join our discussion in #4?

It would be great if you could adjust your code, too, and we then pull your stuff into this repo.

from ev3dev-lang.

I've written my own short python binding. It's incomplete, but it does follow closely to the API spec

https://github.com/G33kDude/pyev3

from ev3dev-lang.

@G33kDude I think having a Python binding in this repo following the API spec would be great; would you be willing to move yours over here (no pressure if you don't want to)? Either way, having another person following the spec means that we have a bigger capacity to catch errors (like you found with the _modes attributes). I browsed your source tree and it looks like all the basics have already been implemented with very little code.

from ev3dev-lang.

I'm not a very experienced python developer, I just do what I can. I'm not opposed to moving my repo over here, I just can't promise the same level of competence and reliability the other libraries have.

from ev3dev-lang.

Feel free to submit a PR if you want to. Once we get the auto-generation system set up, basic maintenance tasks should get a lot easier, which means less work for you.

from ev3dev-lang.

I have looked at autogen - nice bit of work @WasabiFan! With respect to the Python bindings, would it be helpful to have a separate pure Python binding? As I understand it the current one builds on top of the C++ file handlers - or am I mistaken?

from ev3dev-lang.

The python bindings in the official repo are indeed based on the C++ bindings. I gave a bit of motivation for this in #34. The only comparison between my and @topikachu's versions of python bindings I could find is this one: https://github.com/ev3con/python-ev3con/tree/master/linienverfolgung/alternativ. It seems to second my initial guess that C++ based bindings should be a bit faster than pure python (see the comment headers for those two files). Its in German though and I don't speak the language.

from ev3dev-lang.

OK, so do you mind if I also put together a pure Python set of bindings (the API will of course be the same anyways) I am doing some experiments with caching file handles to get around performance issues. Typically you access a bunch of attributes once for setup, then read/write the same handful during normal operation, so caching based on frequency or last used time may give us some real benefits.

from ev3dev-lang.

There's a very large difference between caching and not caching the file handles. Without caching I couldn't get my gyro bot to balance. With caching everything ran much faster, and it balanced much more easily.

from ev3dev-lang.

OK, so do you mind if I also put together a pure Python set of bindings

Sure, why not. If the performance turns out to be adequate, I would not even mind replacing the C++ base with pure python implementation. The development cycle of anything C++ based for ev3dev turned out to be a bit of a PITA due to hardware limitations and performance issues of cross-compilation environment.

Since the majority of the code today is generated with @WasabiFan's autogen script, switching the base should be painless.

from ev3dev-lang.

What do we define as adequate though? I'd imagine we would want the best performance reasonably possible, and I would assume that wrapping the C++ binding would give that. If not for very straightforward actions like enable motor A, but at least for more complicated actions like drawing to the LCD.

If we do switch to "adequate" pure python bindings, it'd be a shame if it turns out the C++ wrapper was more performant and it falls out of development.

from ev3dev-lang.

Another point for the pure python solution is the ability to easily install the bindings anywhere without the need for C++ compiler. Several users raised this question recently having desktop IDEs in mind.

from ev3dev-lang.

What do we define as adequate though?

Since the interfaces would be interchangeable, it should be easy to provide some performance tests. Then I think it would be easy to decide either way.

from ev3dev-lang.

@ddemidov Now that @rhempel is making a pure-python binding, I think we should rename your submodule in this repo to something that differentiates it from the other. I'm thinking that the name native-python would work, to go along with @rhempel's pure-python. Do you have any other ideas? I just want to make the differences clear so that someone attempting to decide what to use won't be confused when looking at the repo.

from ev3dev-lang.

To my ear native-python sounds the same as pure-python. May be cpp-python would be more accurate, or we could just describe each version clearly in the README.
Also, I would not mind merging the versions if @rhempel's experiments with caching file access turn successful, as most of the points I made in #34 are moot since the introduction of the autogen script.

from ev3dev-lang.

We could also merge the cpp and python (the cpp-based ones) bindings into a single submodule, since changing the one usually leads to a change (or at least to rebuilding) of another. Then we would have a cpp binding that incidentally also provides a python interface.

from ev3dev-lang.

I agree native-python and pure-python sound like they might mean the same thing. Let me see how the file caching goes - the whole point of this ev3dev exercise was to use the scripting language's native file i/o capabilities so we would not need low level C bindings for each language :-)

from ev3dev-lang.

I've created a simple test (measure time needed to take 1000 readings of a motor position) for the C++, python, and pure-python bindings. The pure-python bindings were at adeb4bc at the time of the test. The source code of the test versions: https://gist.github.com/ddemidov/e27fdf63f3ff6bb98dcf.

Here are the results:

C++

$ ./perf_motor
lego-ev3-l-motor, outA
1000 readings in 160784 us

Python

$ ./perf_motor.py 
lego-ev3-l-motor, outA
1000 readings in 0.365907 seconds

Pure Python

$ ./perf_motor_pp.py 
Got a match outA <-> outA
/sys/class/tacho-motor/motor0
lego-ev3-l-motor, outA
1000 readings in 1.701115 seconds

So currently Python is 2.27 times slower than C++, and Pure Python is 4.65 times slower than Python (and 10.58 times slower that C++).

from ev3dev-lang.

I still have some debug code in there (obviously) but the results are not too terrible :-) I'm not so much concerned with how much slower pure-python is. It's more important to be useable.
But it's good to have the performance benchmarks anyways!

from ev3dev-lang.

Is it OK to provide some feedback at this point, or should I wait for a pull request?

It seems you have single file handle cache both for reading and writing attributes. When you have an IOError, you reopen the file:

So I think if I modify the test to measure both read and write of the same attribute, pure python will loose the performance gain it got from file caching. It will probably be even slower than straightforward implementation without any caching (due to error handling). A simple solution would be to have two caches: one for reading and the other for writing.

EDIT: I just noticed that when you reopen the file, you do it both for reading and writing, so an IOError is probably a one time thing.

from ev3dev-lang.

Feedback is fine and appreciated - I don't normally program in Python.
What I was trying to accomplish was to isolate knowledge of the read/write nature of the property from the getters and setters. the first time you get an attribute the file is opened read only and is cached. Later, if you want to write to the attribute, the file is clodes and opened in w+ mode. That way there is only a penalty the first time, after that the file is always read/writeable.
I guess we could consider two caches, but then you might run into a file handle limit. I may still need to deal with that.

from ev3dev-lang.

Yes, I got the intention (see the edit to my above comment).

Re file handle limit: there is a builtin decorator functools.lru_cache which looks very nice, but is only available in python3. cachetools is a backport for python2.

from ev3dev-lang.

It looks like _get_string_array_attribute() returns single string instead of string array. return self._get_attribute( attribute ).split() would return an iterable list of strings.

from ev3dev-lang.

_get_string_array_attribute does return the entire string returned from the file - is it supposed to be iterable on return? If yes, then I'll split it before the return

from ev3dev-lang.

A list of strings could be used like this:

# test sensor modes:
for m in sen.modes:
  m.mode = m

# check if command is supported:
if 'run-timed' in mot.commands:
  mot.command = 'run-timed'
else:
  mot.command = 'run-forever'
  sleep(42)
  mot.command = 'stop'

from ev3dev-lang.

There is no doubt it's useful - the question is whether the spec says that we want to return an iterable data structure or the entire string :-)
We could have rewritten the above example using sen.modes.split() - so I think we need consensus on what our language API needs to return for string array properties. My preference is to return the entire string and let the caller handle manipulating the result - but I am happy to discuss options.

from ev3dev-lang.

Looks like the __EV3_MODULE_connected__ dict is never cleared. What happens when a class holding a connected device is destroyed (e.g. goes out of scope)? Do you loose the ability to control the device until program restart?

from ev3dev-lang.

Yes, I'm running into that now after running tests :-) I need to have a destructor that iterates through the cache and closes the file handles.

from ev3dev-lang.

Re string array splitting: I agree this may be a matter of interpretation. I just assumed that the strings should be splitted because that is the way all other bindings are working.

from ev3dev-lang.