sweet-js / sweet-core Goto Github PK

View Code? Open in Web Editor NEW

4.6K 131.0 207.0 24.14 MB

Sweeten your JavaScript.

Home Page: https://www.sweetjs.org

License: BSD 2-Clause "Simplified" License

JavaScript 100.00%

javascript macros parser

sweet-core's People

Contributors

Stargazers

Watchers

Forkers

disnet haohuichen lujaw alexsnet rwaldron ryanseddon parndt zamuro57 dherman truji cpc26 magnetised samhunta kevinkelley soulhakr bharath1097 dahabit tml carlqlange can3p barnabyc iammerrick hallco978 uxscripts open-source-gis netconstructor leadsplus robkuz chinafather ryanramage voila weidagang bmeck mpmedia geofffilippitwc mmcdermo tuchida sunbox stealthycoin chiehwen theosis zmilan gabriano philw9 smillaedler chaosim natefaubion jlongster youthlab galexcode jjedmorianktah tcr mbijon raisch montanaflynn lantiga nijikokun paulog axyoma ananavati iknowl willemdegroef nvdnkpr lifuzu lukehoban andreypopp cybernetics yilab jimlyndon matthewrobb scivi ifrahim r3m0t tegeek escalant3 srikumarks jasuperior binarymuse desa alexfincham sd031 macressler prodigeni jayphelps aitoroses michaelfeitosa ozten eiriklv hax simoko cicorias bradparks dennieallen phamthaithinh wheercool neosocra marctmiller mtco neuroradiology volkovasystems

sweet-core's Issues

macro with empty case fails

The following macro with no patterns in the case fails.

macro m {
  case => { foo }
}
m

Optional tokens

Hi, Ive been trying to implement a "def" macro that supports default parameters. What I could come up with is this:

macro def {
    case ($($p:ident = $v:expr) (,) ...) { $body:SourceElements } => {
        def($p (,) ...) {
            $($p = typeof $p === 'undefined' ? $v : $p) (;) ...;
            $body
        };
    }

    case ($params:ident (,) ...) { $body:SourceElements } => {
        $(function($params (,) ...) {
            $body
        });
    }
}

So, it works for def(a, b) { } and def(a = 4, b = a) { } but not def(a, b = 4) { }. What would be a good way to do this? I thought if I could denote the = $v:expr part as optional, it would be ok but have no idea about the syntax.

Is there a better way to do this? Maybe a recursive solution?

documentation - tutorial

It would be great to have a nice tutorial/walkthrough of how you write macros. Perhaps patterned after the excellent Racket macro documentation.

restrict expansion positions

The following:

    var m();

where m is a macro should not be allowed, will cause problems. Where else do we need to restrict?

Sanitized symbol names inconsistent in generated code

In this example

macro module {
  case $body => {
    define(function (require, exports, module) {
      $body
    });
  }
}

module {
  var x = require('x');
  exports = function foo() {
    return 'foo' + x
  }
}

// outputs:

define(function (require$1, exports$2, module$3) {
    {
        var x$6 = require('x');
        exports = function foo() {
            return 'foo' + x$6;
        };
    }
});

... the sanitized parameter names are not being carried forward properly into the body of the generated function. x$6, which is declared and referenced only inside the body of the function is consistent, but require and exports will be referencing variables declared further up the scope (or undefined) rather than in the function's parameters.

There's a high likelihood that I'm doing this wrong :) Can you confirm if this is a bug or expected behavior, and if it's expected behavior, point me in the right direction? Thanks!

using `macro` as an object property causes errors

Using the identifier macro as an object property will cause an error to be thrown

var o = {
  macro: "foo"
}
o.macro;

add simplified single case form

Proposed syntax:

macro m <form> ... => <body>

Unclosed brackets and parens cause an infinite loop

All of these files will cause sjs to go to an infinite loop:

{
[
(

Expected behavior would probably be either to throw an error or ignore it. I'd maybe ignore with a warning.

macro names cannot be symbols

Examples inspired by coffeescript or ecma6

macro -> not allowed

macro => not allowed

add literal syntax for patterns

Proposed syntax is $[] so the macro:

macro m {
  case ($x:expr $[...]) => { ... }
}

will match an expression followed by an ellipses, not multiple exprs.

Arthimetic in macros

Hi Guys,

is it possible to somehow do arithimetic operations in sweet.js macros?

For example:
macro count {
case $x => { console.log($x); count $($x-1) }
case 0 => { }
}
count(3)
=>
console.log(3);
console.log(2)
console.log(1);

Thanks.
Jonas

add syntax-case form

Proposed syntax:

macro m (stx) {
  case <form> ... => {
     ...
     return #{ function(foo) {} }
  }

So perhaps use #{} as the syntax quote?

reader bugs

Some known bugs in the reader:

a = function () {}
/4/
7

and

function a() { return /42/; }

And:

for (var a = 7 in /7/ in 9 in b);

If I run that in node I get:

TypeError: Cannot use 'in' operator to search for '/7/' in 9

As this code is vaguely equivalent to:

var a = 7; for (a in ((/7/ in 9) in b));

However, it gets converted by sjs to:

for (var a = 7 in (/7/ in (9 in b)));

That's obviously different, and gives this error instead:

ReferenceError: b is not defined

some examples are broken

looks really interesting, but some of your examples are broken here
http://voila.github.com/sweet.js/

insome cases try-it produces results that look like the belong to a different example, and the last one just errors outright

TypeError: Cannot call method 'getValue' of undefined

package.json file

Just so it's easier to get all the deps for this project it'd be good to have a package.json file so people can just run npm install --dev to pull them all down. I can do a pull request for this if you'd like?

integrate with source maps for good line numbers

It might "just work" with the latest version of escodegen but I have a feeling expansion introduced some problems.

An example macro outputs extra characters

Hey, awesome work! I've also experimented with JS and macros a bit [1] [2], and this is a really expressive way of defining my macros even though I can't (yet) convert my current macro-dependent projects to use it because they depend on being able to execute arbitrary code at compile-time.

But to the point, the following example taken from the website:

macro class {
  case $className:ident { 
    constructor $constParam $constBody
    $($methodName:ident $methodParam $methodBody) ...
  } => {
    function $className $constParam $constBody

    $($className.prototype.$methodName 
      = function $methodName $methodParam $methodBody; ) ...

  }
}

class Person {
  constructor(name) {
    this.name = name;
  }

  say(msg) {
    console.log(this.name + " says: " + msg);
  }
}
var bob = new Person("Bob");
bob.say("Macros are sweet!");

produces

function Person(name$1) {
    this.name = name$1;
}
Person.prototype.say = function say(msg$3) {
    console.log(this.name + ' says: ' + msg$3);
};
var bob = new Person('Bob');
bob.say('Macros are sweet!');

What are those dollar signs followed by numbers doing there? Is this a bug or is there use case that requires them to be inserted?

[1] https://github.com/jussi-kalliokoski/inceptionscript
[2] https://github.com/jussi-kalliokoski/macnjs

fix nested ellipses

Only single level of ellipses is currently working.

hygiene issues at the top level

Hygiene isn't working right at the toplevel. In particular variable names are only getting renamed inside functions.

Add remaining hooks into the parser

We have :expr and :ident but need the rest too.

make sure multiple cases in macro definitions are working

Code to support is in place but not well tested yet.

documentation request: explain what features from coffeescript can be implemented

I can shorten a function to def, but can I use an arrow instead? Can I re-bind this like with the fat arrow?
can I make functions automatically return a value
is destructuring assignment possible?

Non-identifier macros

For some cases I would like to include non-identifier macros

macro $ {
  case $x "$" => {
    $x
  }
}

$(123)$

works

macro ` {
  case $x "`" => {
    $x
  }
}

`123`

Would be useful.

Is there anything in this category planned for sweet.js?

performance fixes

Slow for large files. There should be some fairly obvious fixes to get it back to an acceptable speed.

restrict definition positions to top of scope

Need to actually signal error, currently it will just silently cause problems for var statements.

Embed an editor for trying things on sweetjs.org

This project is pretty sweet, although I think having editor for trying out thinks embeded on the website would really make a big difference, in terms of people giving it a try. I'm talking about something along this lines: http://jeditoolkit.com/wisp/

Last code block demos macros system, seeing output while you type is super useful IMO.

allow macros to override reserved keywords

Currently allowing macros to shadow the future reserved keywords (class) but not the current reserved words (var, function). Need to make sure we actually can do this since the reader makes some assumptions about the form of function that might break in the presence of macros overriding its meaning.

Fix recursive/mutually recursive macro definitions

expand when seeing a parse annotation

We need to do an implicit local-expand when seeing a parse annotation. Eg.

macro m {
 case (e:expr) => { ... }
}

The syntax matched inside the parens need to be expanded before the parse is attempted.

Need to make sure we don't expand multiple times when trying to match a multi-case macro (since future syntax-case can cause side effects).

Multiple Cases Not Working

Or I missing something fundamental. I have not used macros before (but have always been intrigued) and so when i saw sweet.js I was thrilled at the chance to see what the fuss was about. To start out lite I figured I would take the class example and expand on it to handle more cases.

Here is the macro definition I came up with to handle the case of a constructor only, normal class, and one with a super class.

macro class {
  case $className:ident {
     constructor $constParam $constBody } => {
        function $className $constParam $constBody
     }

  case $className:ident {
    constructor $constParam $constBody
    $($methodName:ident $methodParam $methodBody) ... } => {

    function $className $constParam $constBody

    $($className.prototype.$methodName
      = function $methodName $methodParam $methodBody; ) ...
  }

  case $className:ident < $super:lit {
    constructor $constParam $constBody
    $($methodName:ident $methodParam $methodBody) ... } => {

    function $className $constParam $constBody
    $className.prototype = new $super.prototype();

    $($className.prototype.$methodName
      = function $methodName $methodParam $methodBody; ) ...
  }
}

Here are some usages:

class Thing {
    constructor() {
      console.log('look you made a thing');
    }
}

class Person {
  constructor(name) {
     this.name = name;
  }

  func() {
     console.log(func);
  }
}

class Me < Person {
  constructor(name) {
     this.name = name;
  }

  anotherFunc(arg1, arg2) {
     console.log(arg1, arg2)
  }
}

If I run sjs with only the first case and Thing class it works. The same for the second and third respectively macro/class pairs. But when I run it all together I get the Macro invocation does not match error.

I fully expect I am doing it wrong and if so a simple pointer in the right direction would be great. Otherwise, if this is valid usage here is a test case.

implement infix macro forms

Using ideas from honu I think we can implement infix operator macros.

Cannot create anonymous functions.

macro lambda {
  case $params $body => {
    function $params $body
  }
}

lambda (x) { console.log(x * x); }

This code produces the error:

/usr/local/share/npm/lib/node_modules/sweet.js/lib/sweet.js:5038
            throw e;
                  ^
TypeError: Cannot read property '0' of undefined
    at throwError (/usr/local/share/npm/lib/node_modules/sweet.js/lib/sweet.js:1173:38)
    at throwUnexpected (/usr/local/share/npm/lib/node_modules/sweet.js/lib/sweet.js:1233:9)
    at parseVariableIdentifier (/usr/local/share/npm/lib/node_modules/sweet.js/lib/sweet.js:2172:13)
    at parseFunctionDeclaration (/usr/local/share/npm/lib/node_modules/sweet.js/lib/sweet.js:2966:14)
    at parseSourceElement (/usr/local/share/npm/lib/node_modules/sweet.js/lib/sweet.js:3119:24)
    at parseSourceElements (/usr/local/share/npm/lib/node_modules/sweet.js/lib/sweet.js:3159:29)
    at Object.parseProgram (/usr/local/share/npm/lib/node_modules/sweet.js/lib/sweet.js:3173:19)
    at parse_stx (/usr/local/share/npm/lib/node_modules/sweet.js/lib/sweet.js:5023:49)
    at Object.parse (/usr/local/share/npm/lib/node_modules/sweet.js/lib/sweet.js:5088:16)
    at Object.exports.run (/usr/local/share/npm/lib/node_modules/sweet.js/lib/sjs.js:21:41)

EDIT: Apparently this code works:

macro lambda {
  case $params $body => {
    (function $params $body)
  }
}

lambda (x) { console.log(x * x); }

I assume this is expected behavior, then?

Deleted

preserve comments

Esprima/escodgen already do this I think. Just need to hook up the sjs binary I think.

add juxtaposition macro wrappers to support CoffeeScript style function application

CoffeeScript (among other languages) allows function application via juxtaposition (ie foo a means foo(a)). Currently there's no way to write a macro that would provide that functionality in sweet.js but @samth pointed out [1] a way of doing this by wrapping juxtapositions in a macro that could then be re-bound.

[1] https://groups.google.com/d/msg/sweetjs/sp8kC-jFOFI/IZDeX6EMWjAJ

How to generate jsdoc ?

Hi,

Is it possible to generate documentation (override existing or merge) ?

use case 1. : generate closure annotation

def foo(arg : number )

/**
 * @param {number}
 */
var foo = function(arg)

sweet.js needs a community discussion group or an irc channel

Where can we talk about all the cool things we're doing with Sweet js? :O

handling rebinding of `this` and `arguments`

Will case problems for hygiene.

browser version of sweet.js

Currently we have some node dependencies but would be nice to run in the browser. Need to factor out the require calls and do a little cleaning.

Make www.sweetjs.org work

I don't know why but my Awesomebar auto completion led me to http://www.sweetjs.org/ which shows a scary placeholder page. Redirecting should be fairly easy, no?

Can't create strings from tokens

macro:

macro magic {
    case ($objName:expr) {
        $(function $methodName:ident $methodParam $methodBody) ... } => {
        $(
            Object.defineProperty($objName, "$methodName", {
                value: (function $methodName $methodParam $methodBody)
            });
        ) ...
    }
}

code:

magic(obj){
    function fn(){
    }
}

expected output:

Object.defineProperty(obj, "fn", {value: function fn() {
    }});

actual output:

Object.defineProperty(obj, fn, {value: function fn() {
    }});

handle `with`

Will probably cause problems for hygiene. Throw warning?

Cannot match ()

Hello,

I'm trying to write a macro that generates a lisp-like linked list.
I got desired output for _L(1,2) and _L(1), but sjs raises error when I tried _L().

Code

macro _L {
  case () => {
    nil
  }
  case ($car) => {
    new Pair($car, nil)
  }
  case ($car, $rest ...) => {
    new Pair($car, _L($rest ...))
  }
}

var nil = new Object;
function Pair(){};

_L();
//_L(1);
//_L(1, 2);

Expected

_L(); is compiled to nil

Actual

/usr/local/share/npm/lib/node_modules/sweet.js/lib/sweet.js:4447
                            assert(env[bodyStx.token.value].level === 1, "elli
                                                           ^
TypeError: Cannot read property 'level' of undefined
    at transcribe (/usr/local/share/npm/lib/node_modules/sweet.js/lib/sweet.js:4447:60)
    at Array.reduce (native)
    at Function._.reduce._.foldl._.inject (/usr/local/share/npm/lib/node_modules/sweet.js/node_modules/underscore/underscore.js:110:28)
    at wrapper.(anonymous function) [as reduce] (/usr/local/share/npm/lib/node_modules/sweet.js/node_modules/underscore/underscore.js:1021:26)
    at transcribe (/usr/local/share/npm/lib/node_modules/sweet.js/lib/sweet.js:4394:28)
    at transcribe (/usr/local/share/npm/lib/node_modules/sweet.js/lib/sweet.js:4452:55)
    at Array.reduce (native)
    at Function._.reduce._.foldl._.inject (/usr/local/share/npm/lib/node_modules/sweet.js/node_modules/underscore/underscore.js:110:28)
    at wrapper.(anonymous function) [as reduce] (/usr/local/share/npm/lib/node_modules/sweet.js/node_modules/underscore/underscore.js:1021:26)
    at transcribe (/usr/local/share/npm/lib/node_modules/sweet.js/lib/sweet.js:4394:28)

A particular case overshadows other cases

Compiling the following macro

macro val {
    case $var:ident = $expr => {
        var $var = $expr
    }

    case $var:ident = $x:lit .. $y:lit => {
        var $var = (function() {
            var arr = []
            for (var i = $x; i < $y; i++)
                arr.push(i)
            return arr
        })();
    }
}

// This snippet below doesn't match
val a = 10;
console.log(a);

// This works nicely
val r = 10 .. 25
console.log(r)

produces the error:

/usr/local/lib/node_modules/sweet.js/lib/sweet.js:189
            throw new Error('ASSERT: ' + message);
                  ^
Error: ASSERT: no macro cases matched
    at assert (/usr/local/lib/node_modules/sweet.js/lib/sweet.js:189:19)
    at Object.macroCases [as transformer] (/usr/local/lib/node_modules/sweet.js/lib/sweet.js:4352:13)
    at expand (/usr/local/lib/node_modules/sweet.js/lib/sweet.js:4666:53)
    at Object.parse (/usr/local/lib/node_modules/sweet.js/lib/sweet.js:5077:32)
    at Object.exports.run (/usr/local/lib/node_modules/sweet.js/lib/sjs.js:19:55)
    at Object.<anonymous> (/usr/local/lib/node_modules/sweet.js/bin/sjs:7:23)
    at Module._compile (module.js:449:26)
    at Object.Module._extensions..js (module.js:467:10)
    at Module.load (module.js:356:32)
    at Function.Module._load (module.js:312:12)

It looks like the case

case $var:ident = $x:lit .. $y:lit => {
        var $var = (function() {
            var arr = []
            for (var i = $x; i < $y; i++)
                arr.push(i)
            return arr
        })();
    }

is somehow overshadowing any other cases in the macro.

add capability to the bin/sjs compiler

To add:

compile multiple files
readline support (repl that expands macros)
options to adjust output formatting (spacing etc.)

hygiene for var

Hygiene for vars is working in some cases but not all. The two failing tests here show the non working cases.

Unexpected boolean change in :expr

macro unless {
  case ($condition:expr) $body => {
    if(!($condition)) $body else {}
  }
}

unless (true) {
  go(10);
}

Becomes "sweeter" this way:

if (!false) {
    go(10);
} else {
}

:expr doesn't allow function calls

This fails to match:

macro m {
  case ($x:expr) => {$x}
}
function id (x) { return x; }
var x = m( id(4) )

This is related to the outside-in/inside-out expansion stuff. Right now when trying to match the :expr the syntax is completely unexpanded but what we really need to do is a local-expand sort of thing.

restrict tokens that a macro can consume

Comma in particular can't be allowed.

preprocessed include (support for modules)

I think it would be useful to have support for including other sjs files from within a sjs program.

That will allow to share macro definitions from more than one file without having to write them on every file using it.