How to document (automatically if possible) names to be used for supported languages. about ev3dev-lang HOT 14 CLOSED

This presents an interesting question. On one hand, having the actual names documented as in your example would probably make it easier to use. But on the other hand, that means more maintenance work and a bigger spec.

I think that, as long as it is manually maintained, we can't feasibly maintain the names in the tables. But if we had the spec auto-generated, we could have the system handle it, and organize it as needed. So, maybe this is a good time to discuss auto-generation of both the human-readable spec sheet and, possibly, the tedious parts of the bindings themselves.

If we set up the spec in a machine-readable way, we could set up some sort of a framework. It could generate both markdown so that it is useful to readers and the individual property getters and setters where appropriate. So, is this something that we want to work on? I would be happy to take a shot at writing a generator if that's what we decide to do.

@fdetro what do you think?

from ev3dev-lang.

IMHO we should not add such implementation details to the general spec. What about using tools like doxygen to create the documentation for the specific languages and then add the result to the repo Wiki?

Regarding the machine-readable spec, I'm currently experimenting with a xml-based spec definition and a xslt document to update my lua binding to the latest version.

If you are interested, I can share the current state with you (it's in a very early state).

from ev3dev-lang.

@fdetro I'd love to see what you have. If we can find a framework that will build both the code and human-readable documentation in one go, I'd say that's the best-case scenario. Then, we could also add auto-updated inline comment docs to the code, which I think would be helpful.

I really don't have any experience with the "standards" for doing this sort of thing, so I'll have to do some research myself to figure out what's available.

from ev3dev-lang.

@WasabiFan I have published a codegen_experiments branch with my current state of code generation.

I do not know of any standard tools for this kind of task, I usually use XSLT (for simple cases) or script languages like lua.

The branch contains a XML version of the spec (plus the DC and servo motor classes), please ignore the pre/post-lua tags. It seems that XSLT is too simple for our use case here, at least for the cpp part of the spec. It should be easy to have the .md version of the spec generated using XSLT.

Due to the experiences I made during my experiments I think that it does not make sense to put more efforts into this, but maybe you come to another conclusion.

from ev3dev-lang.

Due to the experiences I made during my experiments I think that it does not make sense to put more efforts into this, but maybe you come to another conclusion.

Although I don't think that this format (XSLT) will be best in the end, I do think that it's a very good demonstration.

My base thinking is this: if we can auto-generate the core of each class (the getters/setters mostly), it could greatly speed up our turn-around time for updates, and require much less work from us. So, the question is: "how do we accomplish this?". Here's what I am looking for in a system that does this:

• Extensibility. I want it to be able to read a spec declaration, and output whatever we want, templated using the spec's values (no human intervention for that part).
• In-place generation. I think the easiest way to use a system like this is if you can run a script, and it will update the contents of the pre-existing file. Then we can still manually change the files, as long as we steer clear of the auto-generated sections.

I am thinking that we should write our own script (we can chose a language later) that does this. I imagine adding some simple markup in comments to each of our files, that specifies where we want the script to put things. Something like this (pseudo-code):

class Motor {
    constructor(...) {
        ...
    }

    //<autogen section="motor-properties">
    //</autogen>
}

Then, we can define what we want in each section in some sort of definition file (one for each language). We run the script, it finds the tags, deletes what was inside before, and repopulates the section with new content. This is a very high-level thought process, so I expect that it won't be quite as simple as that in the end. But I'd be happy to give it a try when I have some time.

from ev3dev-lang.

OK, so I had some free time and decided to take a shot at this (it's a slightly modified syntax from the one described above). I wrote my test in JavaScript because I am familiar with it, but we could probably write it in a different language instead if we want to. Basically, I set up a syntax like this:

//The next line starts the "autogen" block. This identifier could be anything, as long as it can be put in a comment
//~autogen
/*
    // Everything inside this block comment is parsed as Liquid (it's a templating engine, same as we use on our ev3dev website). Along with some other syntax such as "if" statements, you can inject information that my script loads from the spec like this: {{ meta.version }}.
*/
    // Everything inside this block comment is parsed as Liquid (it's a templating engine, same as we use on hour ev3dev website). Along with some other syntax such as "if" statements, you can inject information that my script loads from the spec like this: 0.9.1-pre.
//~autogen
// Everything outside of the block comment above was generated from the liquid content. It is replaced each time the autogen script is run.

You can take a look at the code that I pushed to this branch to get a better idea of how it works. In that test, I auto-generated the LED class from a JSON spec. It won't win any style awards, but it functions pretty well.

from ev3dev-lang.

@fdetro I have been experimenting with various versions of this idea for the past few days, and am seeing some good results. What do you think about doing something like this? Should I continue developing this concept, or would you prefer a different language and/or design?

from ev3dev-lang.

In general your approach looks OK to me, JSON as machine-readable spec format is as good as XML.

Regarding the gen language JavaScript seems to be a very good compromise, every developer should at least have basic knowledge of it, otherwise it's very easy to learn.

What I do not like is that the autogen instructions do stay redundantly in the source code files. Can't we have a syntax like

//~autogen:version
//~autogen:property.header:LED
//~autogen:property.source:Motor

and then put the autogen code into another file, like cpp.autogen, js.autogen etc.? My original approach also covered constants, and maybe in the future we can cover default implementations for constructors, too. Does this sound reasonable?

from ev3dev-lang.

What I do not like is that the autogen instructions do stay redundantly in the source code files.

I had the same thought. I have some changes that are pretty much done that fix this problem. The syntax that I used is:

//~autogen js_myfile classes.led>currentClass

The first part specifies the file name that it's looking for; when the parser reads that, it appends an extension to the end and then loads that file as a template. The rest is a space-separated list of variable mappings, for arguments. In this example, the template would always look at the variable "currentClass", and the source comment would specify which spec object that would be populated with.

My original approach also covered constants, and maybe in the future we can cover default implementations for constructors, too.

Yes, sounds good. We can put whatever data we want in the spec, including constant info and descriptions, so that we can generate more code and some human-friendly documentation in the future (my plan is to check the file extension to determine what comment style to use, so we can add html-style comments for markdown).

I should be able to push my changes in an hour or two so that you can try it out.

P.S. I have been doing a lot of testing in my experiments branch, so whenever we merge it I want to squash the commits before bringing them to our main branch.

from ev3dev-lang.

I have just pushed my changes in commit 86c4a90. It includes the template files instead of inline comments, and manages whitespace better. In official liquid syntax, there is no way to keep code looking reasonable while not adding a bunch of whitespace to the output. To circumvent this, I forked the liquid library that I was using and slightly modified it so that tags can go on to separate lines -- not the best solution, but it works without breaking anything. Ex:

{% if myvar == true %}{%
    assign foo = 7 %}{%
else %}{%
    assign foo = 8 %}{%
endif %}

from ev3dev-lang.

@fdetro Now that we have released, I'd say it's time to start thinking about merging the autogen branch in to develop. Are there any changes that you would like to make before we do this?

And when we do merge, would you rather that we throw away the history from that branch (which is fairly ugly) or just do a normal merge?

from ev3dev-lang.

Sorry for my delayed response, I have very little time at the moment. Yes, please do a normal merge of your changes to the develop branch. Can you add a small HowTo?

from ev3dev-lang.

OK, done!

There is now a readme file in the autogen directory that explains how to run the script and includes an overview on including templates in code.

from ev3dev-lang.

@fdetro If you have some free time over the next few days/weeks, can you try to get some of your bindings transferred over to the autogen system? Once the core of our libraries is generated off of the same specification, we should be able to adjust to kernel changes and add new classes fairly quickly without much work from us.

I think I am done with all classes as of now in my JS binding, so I'm moving on to my other one.

from ev3dev-lang.