w3c / webidl2.js Goto Github PK

View Code? Open in Web Editor NEW

236.0 236.0 60.0 3.86 MB

WebIDL parser

Home Page: https://w3c.github.io/webidl2.js/checker/

License: MIT License

HTML 1.74% JavaScript 78.37% WebIDL 19.89%

webidl2.js's People

Contributors

Stargazers

Watchers

Forkers

xuyuhan ayg btford trueinteractions constellation xinliux imclab manishearth tobie suto jyasskin othree ms2ger teoli2003 motiz88 tmpvar modulexcite artillery mcanthony markandrus halindrome halton mkwtys kleopatra999 arronei amilajack timothygu meylah alexxnica tqeto prozanne charlieyqin ricea kenrussell kainino0x corner-sky jpmedley dyna-dot lukebjerring aleksrutins ihgazni2 tg1999 stevedorries eb-forks doc22940 opencollective monkeywithacupcake cscott foolip isabella232 lucacasonato cybai nickmeinhold pyoor isaac-mason bob420-svg

webidl2.js's Issues

Support for ExtendedAttributeIdentList (current editor's draft)

While parsing:

[Constructor(DOMString type, optional CustomEventInit eventInitDict),
 Exposed=(Window,Worker)]
interface CustomEvent : Event {
  readonly attribute any detail;

  void initCustomEvent(DOMString type, boolean bubbles, boolean cancelable, any detail);
};

(from https://dom.spec.whatwg.org/#customevent)

I'm getting the error

No right hand side to extended attribute assignment, line 1 (tokens: '(Window,Worker)')
[
    {
        "type": "other",
        "value": "("
    },
    {
        "type": "identifier",
        "value": "Window"
    },
    {
        "type": "other",
        "value": ","
    },
    {
        "type": "identifier",
        "value": "Worker"
    },
    {
        "type": "other",
        "value": ")"
    }
]

It seems like the parser doesn't support identifier lists as specified in the current editor's draft: http://heycam.github.io/webidl/#prod-ExtendedAttributeIdentList.

Are there plans to support the editor's draft?

Numerous tests are failing

Numerous tests are failing when running the test suite using the npm test command, despite running a version of the test harness patched with #6.

Dump is 15k line longs, care to see it or can you reproduce?

--tobie

Can't represent union types in typedefs

Currently webidl2 reports "Unterminated operation" when encountering union types in typedefs.

For example, see this from the webgl 1.0 spec idl:

  typedef (ImageData or
           HTMLImageElement or
           HTMLCanvasElement or
           HTMLVideoElement) TexImageSource;

and these from the webgl 2.0 spec idl:

typedef (Uint32Array or sequence<GLuint>) UniformUIVSource;
...
typedef (Float32Array or sequence<GLfloat>) UniformMatrixFVSource;
...
typedef (Int32Array or sequence<GLint>) VertexAttribIVSource;
// etc

But according to the WebIDL grammar this should be valid, as a Typedef contains a Type:

[24]    Typedef → "typedef" ExtendedAttributeList Type identifier ";"

And a Type can be a SingleType or a UnionType:

[57]    Type    → SingleType 
 | UnionType TypeSuffix
...
[59]    UnionType   → "(" UnionMemberType "or" UnionMemberType UnionMemberTypes ")"

'npm install' fails on building microtime

$ npm install
npm WARN deprecated [email protected]: Jade has been renamed to pug, please install the latest version of pug instead of jade
npm WARN deprecated [email protected]: graceful-fs v3.0.0 and before will fail on node releases >= v7.0. Please update to graceful-fs@^4.0.0 as soon as possible. Use 'npm ls graceful-fs' to find it in the tree.
npm WARN deprecated [email protected]: Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue

> [email protected] install /home/halton/work/projects/github/webidl2.js/node_modules/microtime
> node-gyp rebuild

make: Entering directory '/home/halton/work/projects/github/webidl2.js/node_modules/microtime/build'
  CXX(target) Release/obj.target/microtime/src/microtime.o
In file included from ../src/microtime.cc:11:0:
../../nan/nan.h:261:25: error: redefinition of ‘template<class T> v8::Local<T> _NanEnsureLocal(v8::Local<T>)’
 NAN_INLINE v8::Local<T> _NanEnsureLocal(v8::Local<T> val) {
                         ^
../../nan/nan.h:256:25: note: ‘template<class T> v8::Local<T> _NanEnsureLocal(v8::Handle<T>)’ previously declared here
 NAN_INLINE v8::Local<T> _NanEnsureLocal(v8::Handle<T> val) {
                         ^
../../nan/nan.h:661:13: error: ‘node::smalloc’ has not been declared
     , node::smalloc::FreeCallback callback
             ^
../../nan/nan.h:661:35: error: expected ‘,’ or ‘...’ before ‘callback’
     , node::smalloc::FreeCallback callback
                                   ^
../../nan/nan.h: In function ‘v8::Local<v8::Object> NanNewBufferHandle(char*, size_t, int)’:
../../nan/nan.h:665:50: error: ‘callback’ was not declared in this scope
         v8::Isolate::GetCurrent(), data, length, callback, hint);
                                                  ^
../../nan/nan.h:665:60: error: ‘hint’ was not declared in this scope
         v8::Isolate::GetCurrent(), data, length, callback, hint);
                                                            ^
../../nan/nan.h: In function ‘v8::Local<v8::Object> NanNewBufferHandle(const char*, uint32_t)’:
../../nan/nan.h:672:67: error: no matching function for call to ‘New(v8::Isolate*, const char*&, uint32_t&)’
     return node::Buffer::New(v8::Isolate::GetCurrent(), data, size);
                                                                   ^
In file included from ../../nan/nan.h:25:0,
                 from ../src/microtime.cc:11:
/usr/include/nodejs/src/node_buffer.h:31:40: note: candidate: v8::MaybeLocal<v8::Object> node::Buffer::New(v8::Isolate*, v8::Local<v8::String>, node::encoding) <near match>
 NODE_EXTERN v8::MaybeLocal<v8::Object> New(v8::Isolate* isolate,
                                        ^
/usr/include/nodejs/src/node_buffer.h:31:40: note:   conversion of argument 3 would be ill-formed:
In file included from ../src/microtime.cc:11:0:
../../nan/nan.h:672:67: error: invalid conversion from ‘uint32_t {aka unsigned int}’ to ‘node::encoding’ [-fpermissive]
     return node::Buffer::New(v8::Isolate::GetCurrent(), data, size);
                                                                   ^
In file included from ../../nan/nan.h:25:0,
                 from ../src/microtime.cc:11:
/usr/include/nodejs/src/node_buffer.h:43:40: note: candidate: v8::MaybeLocal<v8::Object> node::Buffer::New(v8::Isolate*, char*, size_t) <near match>
 NODE_EXTERN v8::MaybeLocal<v8::Object> New(v8::Isolate* isolate,
                                        ^
/usr/include/nodejs/src/node_buffer.h:43:40: note:   conversion of argument 2 would be ill-formed:
In file included from ../src/microtime.cc:11:0:
../../nan/nan.h:672:67: error: invalid conversion from ‘const char*’ to ‘char*’ [-fpermissive]
     return node::Buffer::New(v8::Isolate::GetCurrent(), data, size);
                                                                   ^
../../nan/nan.h: In function ‘v8::Local<v8::Object> NanNewBufferHandle(uint32_t)’:
../../nan/nan.h:676:29: error: could not convert ‘node::Buffer::New(v8::Isolate::GetCurrent(), ((size_t)size))’ from ‘v8::MaybeLocal<v8::Object>’ to ‘v8::Local<v8::Object>’
     return node::Buffer::New(v8::Isolate::GetCurrent(), size);
                             ^
../../nan/nan.h: In function ‘v8::Local<v8::Object> NanBufferUse(char*, uint32_t)’:
../../nan/nan.h:683:12: error: ‘Use’ is not a member of ‘node::Buffer’
     return node::Buffer::Use(v8::Isolate::GetCurrent(), data, size);
            ^
microtime.target.mk:90: recipe for target 'Release/obj.target/microtime/src/microtime.o' failed
make: *** [Release/obj.target/microtime/src/microtime.o] Error 1
make: Leaving directory '/home/halton/work/projects/github/webidl2.js/node_modules/microtime/build'
gyp ERR! build error 
gyp ERR! stack Error: `make` failed with exit code: 2
gyp ERR! stack     at ChildProcess.onExit (/usr/share/node-gyp/lib/build.js:269:23)
gyp ERR! stack     at emitTwo (events.js:87:13)
gyp ERR! stack     at ChildProcess.emit (events.js:172:7)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:200:12)
gyp ERR! System Linux 4.4.0-24-generic
gyp ERR! command "/usr/bin/nodejs" "/usr/bin/node-gyp" "rebuild"
gyp ERR! cwd /home/halton/work/projects/github/webidl2.js/node_modules/microtime
gyp ERR! node -v v4.2.6
gyp ERR! node-gyp -v v3.0.3
gyp ERR! not ok 
[email protected] /home/halton/work/projects/github/webidl2.js
├─┬ [email protected] 
│ ├── [email protected] 
│ └── [email protected] 
├── [email protected] 
├─┬ [email protected] 
│ └─┬ [email protected] 
│   ├── [email protected] 
│   ├── [email protected] 
│   ├─┬ [email protected] 
│   │ └── [email protected] 
│   ├── [email protected] 
│   └── [email protected] 
├─┬ [email protected] 
│ ├── [email protected] 
│ ├─┬ [email protected] 
│ │ └── [email protected] 
│ ├── [email protected] 
│ ├── [email protected] 
│ ├─┬ [email protected] 
│ │ ├── [email protected] 
│ │ ├── [email protected] 
│ │ └─┬ [email protected] 
│ │   ├── [email protected] 
│ │   └── [email protected] 
│ ├── [email protected] 
│ ├─┬ [email protected] 
│ │ ├── [email protected] 
│ │ └── [email protected] 
│ ├─┬ [email protected] 
│ │ └── [email protected] 
│ └── [email protected] 
└── [email protected] 

npm ERR! Linux 4.4.0-24-generic
npm ERR! argv "/usr/bin/nodejs" "/usr/bin/npm" "install"
npm ERR! node v4.2.6
npm ERR! npm  v3.5.2
npm ERR! code ELIFECYCLE

npm ERR! [email protected] install: `node-gyp rebuild`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the [email protected] install script 'node-gyp rebuild'.
npm ERR! Make sure you have the latest version of node.js and npm installed.
npm ERR! If you do, this is most likely a problem with the microtime package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     node-gyp rebuild
npm ERR! You can get information on how to open an issue for this project with:
npm ERR!     npm bugs microtime
npm ERR! Or if that isn't available, you can get their info via:
npm ERR!     npm owner ls microtime
npm ERR! There is likely additional logging output above.

npm ERR! Please include the following file with any support request:
npm ERR!     /home/halton/work/projects/github/webidl2.js/npm-debug.log

Exception when parsing test/syntax/idl/typedef.widl

When running browser test

should produce the same AST for 50 ‣
Error: the object {
  "input": "<Point> PointSequence"
  "line": 14
  "message": "Invalid operation"
  "tokens": [
    {
      "type": "other"
      "value": "<"
    }
    {
      "type": "identifier"
      "value": "Point"
    }
    {
      "type": "other"
      "value": ">"
    }
    {

parent of things

When processing IDL, it would be nice have a back reference to the parent/owner of things from the parse tree.

Like, "operation" would have the parent "interface" as a reference. Enum values would have the Enum as the parent. Params would have the "operation" as the parent... and so on up the tree.

Interfaces, Enums, Dictionaries would just be null as the parent.

Comment order parsing bug

Give the following input (top level):

/* first */
// second

The parser is returning:

// second
/* first */

As json:

[
  {
    "type": "line-comment",
    "value": " second"
  },
  {
    "type": "multiline-comment",
    "value": " first "
  },
  {
    "type": "ws",
    "value": "\n"
  }
]```

Extended Attributes `rhs` should always be there

It's somewhat annoying to have to check for missing properties, as is the case for rhs on extended attributes.

We should make sure we don't conditionally add/remove members - and where they are null, they should just be null.

No Licensing Information

Awesome project! I'm contemplating using this for something, but I noticed you haven't decided upon a license yet.

Could you do this, and add a LICENSE file to the project? Thanks!

Enum values should be objects

Given:

enum Things {
    "foo",
    "bar"
};

Enum values current result in,

[
  {
    "type": "ws",
    "value": "\n          "
  },
  "foo",
  {
    "type": ","
  },
  {
    "type": "ws",
    "value": "\n          "
  },
  "bar",
  {
    "type": "ws",
    "value": "\n        "
  }
]

This switching of types makes things really inconsistent. Would have expected an Object like:

  {
    "type": "enum-value",
    "value": "foo",
  }

And maybe, for compat:

  {
    "type": "enum-value",
    "value": "foo",
    toString(){
       return this.value;
    }
  }

Add mixin support

Seems we don't support mixins yet, right?

interface mixin Observable {
  void addEventListener(DOMString type,
                        EventListener listener,
                        boolean useCapture);
  // ...
};

Keep supporting allowNestedTypedefs or not?

We already issued several breaking changes, users probably removed the nonstandard syntax by now, so I think we can just remove it. What do you think?

Is array syntax dead?

The type_suffix function parses array syntax (type[]) but I cannot find a relevant reference on Web IDL spec, is it dead?

Type ::
    SingleType
    UnionType Null

Question: Convert WebIDL -> Javascript

Does webidl2 support converting WebIDL definitions to javascript? WebIDL to planned to do this so I'm wondering if webidl2 does.

Add support for Annotated Types

See Annotated Types in WebIDL Spec

We need this for ReSpec for the Media Capture & Streams spec

Looking at the git logs, the ws: true option is to enable the writing feature. It will be great to write a whitespace test with the writer (as we can just parse, write, and check the text is intact) but writer.js is currently not being maintained anymore. Should we maintain it or not?

Export is not AMD compatible

Should export to exports object correctly, not bind to window object.

How can I parse just a function?

I'm facing [NamedConstructor=MyClass(ArgType argument)] and I'm wondering how I should parse the constructor definition in terms of operation.

For now, I'm using the following work-around: parsing "interface TEMP { void " + constructorDefinition + "; }" where I recover the constructorDefinition from its currently parsed state, but there should probably exist a way to do this directly from inside the API.

Or is there already a way I'm unaware of?

Parsing error for annonated inner types of generic types

Parsing the below throws the following: Error parsing generic type Promise.

interface Foo {
  Promise<[XAttr] long> doStuff();
};

This should be supported per spec.

Parser throws on nullable array of nullable array

The following valid IDL:
interface Foo {
attribute Bar[]?[]? array;
};

makes webidl2.js throw:
Can't nullable more than once, line 2 (tokens: ' array;
}')

Add support for records

https://heycam.github.io/webidl/#idl-record

Whitespace issues

@dontcallmedom, I'm getting breakages in the ReSpec test suite relating to white space going missing. Could you possibly help debug that?

I'm won't have time to look at it for 2 weeks (going on vacation) - so can't release an updated ReSpec to support the latest changed recently added.

Ping me on IRC if you want to discuss... otherwise, just run the test runner in ReSpec and you will see the errors.

E.g.,

Expected '        typedef DOMString string;
      ' to equal 'typedef DOMString string;'.

Remove implements statement support

I think "implements" has been dropped from the IDL spec... probably when "mixin" happened...

Affected specs: whatwg/webidl#472

Can't represent large constants

WebIDL allows constants of the 64-bit integer type unsigned long long. However, this library tries to represent them as a javascript number, which means that large numbers get rounded. jyasskin@48a2400 demonstrates this.

Remove exceptions support

webidl2.js includes a function for parsing exceptions (I even went so far as to type these as IDLException in my typed branch); however, upon closer inspection, I don't believe WebIDL allows exceptions to be declared. See here:

An exception is a type of object that represents an error and which can be thrown or treated as a first class value by implementations. Web IDL does not allow exceptions to be defined, but instead has a number of pre-defined exceptions that specifications can reference and throw in their definition of operations, attributes, and so on.

I also don't see any reference to "exception" in the grammar, so I think this code should be removed (originally introduced here).

module not supported?

Excuse me, I try the this parser on https://github.com/RicoP/idl2typings/blob/master/idl/webgl.idl, and get Exception of
"Unrecognised tokens, line 14 (tokens: 'module webgl {')"

Does module not supported yet？

Remove serializer-related productions

Dear all,

In whatwg/webidl#323, we removed serializers from WebIDL and replaced it with regular toJSON operations (and the new [Default] extended attribute).

This means that we were able to remove all of the serializer-related productions from the WebIDL grammar. And in particular the following non-terminals:

and the following terminals:

serializer in ArgumentNameKeyword.

Because we're relying on existing grammar for the replacement solution (operations and extended attributes), there is no new grammar to support. 💃

As usual, please feel free to reach out if anything needs clarification or if you have questions.

Thanks!

Type conversion on default values is destructive

Given

interface X {
  const unsigned long BIT_MASK = 0x0000fc00;
}

Becomes:

"{
  "type": "number",
  "value": 64512
}"

I think the only fix here is not to do the number conversion.

@saschanaz, thoughts?

Add support for missing types in ServiceWorker

See w3c/ServiceWorker#259

Remove creators support

whatwg/webidl#44 removed it back in 2015. It should be safe to remove it here too.

Parser throws on nullable array of any

The following valid IDL:
interface Foo {
attribute any[]? array;
};

makes webidl2.js throw:
Type any cannot be made nullable, line 2 (tokens: 'array;
};')

Should report error for using duplicate names

Consider the following example idl:

typedef int Test;

interface Test {
  void foo();
};

The generated output is as

[
    {
        "extAttrs": [],
        "idlType": {
            "generic": null,
            "idlType": "int",
            "nullable": false,
            "sequence": false,
            "union": false
        },
        "name": "Test",
        "type": "typedef"
    },
    {
        "extAttrs": [],
        "inheritance": null,
        "members": [
            {
                "arguments": [],
                "creator": false,
                "deleter": false,
                "extAttrs": [],
                "getter": false,
                "idlType": {
                    "generic": null,
                    "idlType": "void",
                    "nullable": false,
                    "sequence": false,
                    "union": false
                },
                "legacycaller": false,
                "name": "foo",
                "setter": false,
                "static": false,
                "stringifier": false,
                "type": "operation"
            }
        ],
        "name": "Test",
        "partial": false,
        "type": "interface"
    }
]

In this case, the name Test is used twice: once for the interface and once for the typedef.

Should the library report this as an error?

extended attribute structure missing type

Expected:

{
   type: "extendedAttribute",
   // other props
}

Otherwise, they can't be distinguished without somehow walking up the tree.

What is lib/writer.js

I love your library. i have some questions about it.

can you explain shortly what is lib/writer.js ?

types of types

For interfaces, const, attributes etc. that define a idlType, those types should themselves have a type: property saying what they are (e.g., "attribute-type", "constant-type").

Add support for namespaces

We added them in whatwg/webidl@df8c9c4, yay! Would be good to support them in the JS Web IDL parser everyone knows and loves.

Add support for iterable<>, maplike<>, setlike<> declarations

[NoInterfaceObject] on Dictionary types

We should probably abort if we see [NoInterfaceObject] on a Dictionary. Just saw this in spec I was reviewing.

Remove legacy caller support

Legacy callers will be removed in whatwg/webidl#412. Corresponding idlharness.js bug: web-platform-tests/wpt#7258.

Wrong jsondiffpatch location

Run browser tests and get following errors:

Parses all of the IDLs to produce the correct ASTs
should produce the same AST for 0 ‣
ReferenceError: jsondiffpatch is not defined
    at Context.<anonymous> (run-tests.js:11:46)
...

Get Robin to give us push permissions on npm

Waiting on @darobin to grant me admin on npm.

Help migrate people on recent Web IDL extended attribute changes

Recently we introduced the concept of extended attributes applying to types. The extended attributes [Clamp], [EnforceRange], and [TreatNullAs] were moved into this category. This means that the following syntactically-allowed Web IDLs are not correct:

  [TreatNullAs=EmptyString] attribute DOMString data;

should be

  attribute [TreatNullAs=EmptyString] DOMString data;

and

  void close([Clamp] optional unsigned short code, optional USVString reason);

should be

  void close(optional [Clamp] unsigned short code, optional USVString reason);

To get people to migrate, it would be very helpful if ReSpec could emit an error on such now-invalid Web IDL, telling people about the change. I'm updating a variety of specs now, but I'm sure to miss some.

Records don't seem to be working right

Given:

dictionary Foo {
  record<DOMString, USVString> keys;
};

Getting:

base-runner.js:47 TypeError: Cannot read property 'nullable' of undefined
    at idlType2Text (webidl.js:235)
    at idlType2Text (webidl.js:270)
    at idlType2Text (webidl.js:267)
    at webidl.js:536
    at Array.forEach (<anonymous>)
    at writeDefinition (webidl.js:527)
    at webidl.js:503
    at Array.map (<anonymous>)
    at makeMarkup (webidl.js:502)
    at HTMLPreElement.<anonymous> (webidl.js:1171)

It's affecting spec:
https://w3c.github.io/push-api/#dom-pushsubscription

Drop support for Node 4, move to 6 LTS

In October, we should drop Node 4, and start targeting.

It might also be a good chance to rewrite this is ES6.

Would like to be part of that effort so I can learn the code base a bit better.

question: where to find w3c idls ?

i failed to find a central place with the idl defined, or a doc explaining how to regenerate them
i got piece here and there but no central place
- e.g. http://www.w3.org/TR/DOM-Level-2-Core/idl-definitions.html

question how to preprocess them to get actual idl ?

they contains cpp preprocessor directive apparently
how to preprocess them to get actual pure idl ?

Unable to parse annotated types in generics

interface I {
    void f1(sequence<[XAttr] long> arg);
};

(Also happens with other types using generics like records.)

tokenise() assumes a specific property enumeration order

In order to run webidl2.js in a JS_MORE_DETERMINISTIC build of SpiderMonkey, I had to change

var ..., types = []
for (var k in re) types.push(k);

to:

var types = ["float", "integer", "identifier", "string", "whitespace", "other"];

document white space

We seem to be missing documentation for white space types... in particular, not sure what "ws-pea" and "ws-tpea" are (they sound cute tho 🥜 ).