nanopass / nanopass-framework-scheme Goto Github PK

@dybvig requests:

would be nice to autogenerate when the output type is *

Should be doable and would simplify some common use cases. We will also need to check that there are not extra return values.

Now that stex and Chez Scheme are available on github, the docs makefile should be updated to allow others to build these docs.

In the meantime the user-guide.pdf is committed in this directory.

I've defined two simple languages:

(define-language N
  (terminals (number (n)))
  (Num ()
    n))

(define-language M
  (terminals (number (n)))
  (Math (e)
    n
    (+ e0 e1)
    (* e0 e1)
    (- e0 e1)
    (/ e0 e1)))

the pass between the two is similarly simple:

(define-pass M->N : M (e) -> N ()
  (Eval : Math (e) -> Num ()
    ((+ ,(e0) ,(e1)) (+ e0 e1))
    ((* ,(e0) ,(e1)) (* e0 e1))
    ((- ,(e0) ,(e1)) (- e0 e1))
    ((/ ,(e0) ,(e1)) (/ e0 e1))
    (,n n)))

But, I get this error:

Exception in N: meta not found e0 in N

I can easily fix this by adding e to the meta-variable list for the nonterminal Num, but I don't think I should have to.
Is this behavior intentional?

We would like to have a way to create a "template" language with a "template" pass, and then extend both the language and the pass to create a concrete pass.

We would like to be able to extend both the existing transformers and the list of definitions, expressed using something like the - and + syntax that we extend passes with.

The idea is to have something like:

(define-pass foo-template : L-template (x) -> L-template ()
  (definitions
     (define f1 (lambda (x) ---))
     (define f2 (lambda (y) ---))
    ---)
  (A : A (x) -> A ()
    [(a) `(a)]
    [(b ,x ,y ,z) `(b ,x ,y ,z)]
    ---)
  (B : B (x) -> B ()
    [(qux ,x) `(qux ,x)]
    ---)
  body)

and modify it as (syntax subject to change):

(define foo : (L (extends L-template)) (x) -> (L (extends L-template)) ()
  (extends foo-template)
  (definitions
    (- (define f2 ...))
    (+ (define f3 (lambda (z) ---)))
  (A : A (x) -> A ()
    (- [(b ,x ,y ,z) ...])
    (+ [(c ,w) `(c ,w)]
        [(d ,n) `(d ,n)]))
  (+ (C : C (x) -> C ()
        ---)
  (- body)
  body)

For testing purposes it is useful to be able to define a language as:

(define-language L)

and a pass as:

(define-pass test-pass : L (x) -> L ())

Right now this produces:

Exception in define-pass: unrecognized non-terminal #f

@dybvig reports:

listing the same production twice for the same nonterminal should 
cause an error 

This should be checked and an error raised.

not an issue, i figure it out, an stupid question

As of right now trace-define-pass output the trace in stdout, it should use stderr.

Right now the unparser simply recurs to the top-level unparser on sub-parts. This means that if more than one terminal has the same domain, e.g., if we use symbols to represent both variables and constant symbols, differentiated by context, we will see all of them printed out the same way, even if the language specifies different unparsers for these two terminals.

One caveat to note with this is that if the unparser is called directly on one of these terminals, there is no way to determine which one it is (since the only way it can differentiate these things is using the terminal predicate, and two predicates will return true for the same actual terminal.)

Originally reported by @soegaard on nanopass framework google group with the following test program (in Racket):

#lang racket
(require nanopass/base)

(define id?              identifier?)
(define (unparse-id id)  (list 'id (syntax-e id)))

(define property?            identifier?)
(define (unparse-property p) (list 'prop (syntax-e p)))

(define-language L
  (entry Foo)
  (terminals ((id       (x)) . => . unparse-id)
             ((property (p)) . => . unparse-property))
  (Foo (f)
    (foo x p)))

(unparse-L
 (with-output-language (L Foo)
   `(foo ,#'a ,#'b)))

Found an example where we had a pass like this:

(define-pass ---
  (Foo : Foo (x) -> Foo ()
     ---)
  (Bar : Bar (x y) -> Bar ()
    [(form ,[foo y -> foo]) `(form ,foo)])

And it was matching. It may also require extra return values, since the example pass had that form.

Hi, I'm implementing a Language Protocol Server named scheme-langserver and adding some type inference features. It would be convenient if this repository was published on Akku.
Akku.scm is a language package manager for Scheme. It grabs hold of code and vigorously shakes it until it behaves properly.

In the current version of the nanopass framework, passes with always auto-generate missing transformers and assume that that is what was intended by the programmer, even if the programmer simply missed a case, or misspecified a cata-morphism.

It should be possible to disable this when the programmer wishes to see instead receive an error for a missing transformer.

(requested, most recently, by Tim Zakian--but also by Kent Dybvig, Eric Holk, etc.)

We were playing around with a tree example, where we stuck in extra non-terminals for the heck of it:

(define-language LTree2
  (entry Tree)
  (Tree (t) n l)
  (Node (n) (node t0 t1))
  (Leaf (l) (leaf i))
  (terminals (number (i))))

We wanted to be able to handle the Tree non-terminal with patterns like this:

[,[n -> h] h]
[,[l -> h] h]

These currently result in a "no top-level cata" error. But, would it be possible to permit this? The "n" and "l" identify the target for the cata right? Pattern matching on ,n and ,l without catas works fine.

Platform: win10 1903, chez scheme installed from chocoloty
How to reproduce:
1. clone nanopass from github
2. scheme test-all.ss
What you expect to see:
all test passes.
What you got:

Running error-messages test suite...
  Testing run-time-error-messages ...
!!! expected error message "Exception in with-output-language: expected list of symbol but recieved x in field x* of (lambda (x* ...) body* ... body) from expression (quote x) at character position 31709 of tests/unit-tests.ss" does not match "Exception in with-output-language: expected list of symbol but recieved x in field x* of (lambda (x* ...) body* ... body) from expression (quote x) at character position 32575 of tests/unit-tests.ss"
#f
Ran 1 tests with 0 successes, 1 failures, and 0 exceptions

The failed test is this one

Hi @akeep

I've been intensely engaged in understanding and exploring the nanopass framework and I am curious whether this can be used for non-Lispy languages as well. I do sense that it's, of course, possible.

I've come up with a very basic flow, I wish to benchmark against a single source file for now.

1. Pass the abc.x source file via standard input or via explicit in-scheme command
2. Parse ( into what form?) the contents of the file read in as a Unicode string.
3. Use the nanopass framework based transformations to slowly move towards the target language.

So, as you can see it's the very initial part that's hazy.

Could you give a general sketch of

1. How to use the lexer-phase
2. How to pass the chunked form to the form expected by nanopass.

I know it's not an issue with the framework but something like a basic tutorial for non-Lispy source lang. would be really helpful, apart from the current docs and references.

Currently, there are several issues concerning checking duplicated terminals, alternatives and non-terminals. To check duplicates, we need a procedure to check whether two things compare equal. But with only a equal? procedure, we can only use O(n^2) algorithms to de-duplicate (n being the number of terms/alts/nonterms/meta-vars). If we could have another procedure to hash terms/alts/nonterms, we can de-duplicate in O(n) on average with hash-tables. But the current specification of nanopass prevents us from doing this, because two term's type/name/predicate should be compared by free-identifier=?.
Perhaps we can strengthen the requirements to require two term's type/name to be symbolically equal as well as free-identifier=?, or we can allow the user to specify a symbolic name(compared by eq?) and a predicate (compared by free-identifier=?), only when the two things are both equal can two terminals be considered equal.
I think when terms become hashable, alts also become hashable.
Is this approach acceptable?

It is often the case that we want to fold a transformer over a list of nonterminals and produce the resulting list of nonterminals and another folded result.

It is not entirely clear how this should work, since in some cases we'd want to just create a list of nonterminals (like the current map does), however, sometimes we would like to fold over that list as well.

This is a boilerplate elimination enhancement, so it is worth considering what the right point in the space is.

Greetings,

I am trying to track down a compile problem in Chez Scheme 9.5.3 on Raspberry Pi 3/Raspian-Linux.

My attempt to make the code generation visible was to

 (trace-define-pass np-generate-code : L16 (ir) -> * (code)

This dies in the pretty-printer during the bootstrap compile.

Can some kind person help me out to get a cross-compiled bootstrap which displays the code being generated?

Thanks a bunch!
-KenD [Ken (dot) Dickey (at) Whidbey (dot) com]

nanoPassTraceProblem.txt

@dybvig reported that:

listing a terminal twice in the same or a different clause should 
cause an error. listing it in two clauses silently causes one to win 
and the other to lose.

Should add checking to avoid this.

Use a prefix matcher to identify meta-variables along with a parameterizable suffix-ok? checker to allow people to determine their own meta-variable suffixes.

I wrote a let expression like the one in the sample document and tried to execute the parsing procedure created by define-parser, but it didn't work.
I'm new to this software, so I might have misunderstood something.

(import (nanopass) (rnrs))

(define-language foo
  (terminals
    (symbol (x))
    (integer (i)))
  (Expression (e body)
    x
    i
    (let ((x* e*) ...) body)))

(define-parser parse-foo foo)
(parse-foo '(let ((a 1)) a))

@LeifAndersen reported that a nanopass language with a simple list of expressions does not handle creating a list with no elements in it, even though this should be handled without a problem.

@LeifAndersen has a solution in the nanopass/nanopass-framework-racket, however, things have drifted enough that I'll need to adapt the solution a bit.

Here is the example:

(define-language Lsrc 
  (terminals
    (symbol (s))) 
  (Expr (e) 
    s
    (e* ...)))

(with-output-language (Lsrc Expr) `()) => '()

But should produce an Expr with an empty list in it.

I'm seeing the Harlan compiler regularly use about 3GB of memory to do all the Nanopass macro expansions. This is kind of a lot, and it also means we have to use the 64-bit version of Petite or Chez to compile programs. Is there a way to significantly reduce this?

When tracing a multiple return value transformer, we seem to be discarding some type of extra values.

Tim Zakian is working on a gentle tutorial document.

Another thing I'd like to put together would be a little cheatsheet in the form of a table or spreadsheet that shows:

• which forms bind which names
• which forms expect which names

Anything else that should go in there?

One thing I'm wondering about is how we can best clarify to students where the magic ends and the plain ol' Scheme code begins.