Code Monkey home page Code Monkey logo

engine262's Introduction

engine262

An implementation of ECMA-262 in JavaScript

Goals

  • 100% Spec Compliance
  • Introspection
  • Ease of modification

Non-Goals

  • Speed at the expense of any of the goals

This project is bound by a Code of Conduct.

Join us in #engine262:matrix.org.

Why this exists

While helping develop new features for JavaScript, I've found that one of the most useful methods of finding what works and what doesn't is being able to actually run code using the new feature. Babel is fantastic for this, but sometimes features just can't be nicely represented with it. Similarly, implementing a feature in one of the engines is a large undertaking, involving long compile times and annoying bugs with the optimizing compilers.

engine262 is a tool to allow JavaScript developers to have a sandbox where new features can be quickly prototyped and explored. As an example, adding do expressions to this engine is as simple as the following diff:

--- a/src/evaluator.mts
+++ b/src/evaluator.mts
@@ -232,6 +232,8 @@ export function* Evaluate(node) {
     case 'GeneratorBody':
     case 'AsyncGeneratorBody':
       return yield* Evaluate_AnyFunctionBody(node);
+    case 'DoExpression':
+      return yield* Evaluate_Block(node.Block);
     default:
       throw new OutOfRange('Evaluate', node);
   }
--- a/src/parser/ExpressionParser.mts
+++ b/src/parser/ExpressionParser.mts
@@ -579,6 +579,12 @@ export class ExpressionParser extends FunctionParser {
         return this.parseRegularExpressionLiteral();
       case Token.LPAREN:
         return this.parseParenthesizedExpression();
+      case Token.DO: {
+        const node = this.startNode<ParseNode.DoExpression>();
+        this.next();
+        node.Block = this.parseBlock();
+        return this.finishNode(node, 'DoExpression');
+      }
       default:
         return this.unexpected();
     }

This simplicity applies to many other proposals, such as optional chaining, pattern matching, the pipeline operator, and more. This engine has also been used to find bugs in ECMA-262 and test262, the test suite for conforming JavaScript implementations.

Requirements

To run engine262 itself, a engine with support for recent ECMAScript features is needed. Additionally, the CLI (bin/engine262.js) and test262 runner (test/test262/test262.js) require a recent version of Node.js.

Using engine262

Use it online: https://engine262.js.org

You can install the latest engine262 build from GitHub Packages.

If you install it globally, you can use the CLI like so:

$ engine262

Or, you can install it locally and use the API:

import { Agent, setSurroundingAgent, ManagedRealm, Value, CreateDataProperty, inspect, CreateBuiltinFunction } from '@engine262/engine262';

const agent = new Agent({
  // onDebugger() {},
  // ensureCanCompileStrings() {},
  // hasSourceTextAvailable() {},
  // loadImportedModule() {},
  // onNodeEvaluation() {},
  // features: [],
});
setSurroundingAgent(agent);

const realm = new ManagedRealm({
  // promiseRejectionTracker() {},
  // getImportMetaProperties() {},
  // finalizeImportMeta() {},
  // randomSeed() {},
});

realm.scope(() => {
  // Add print function from host
  const print = CreateBuiltinFunction((args) => {
    console.log(...args.map((tmp) => inspect(tmp)));
    return Value.undefined;
  }, 1, Value('print'), []);
  CreateDataProperty(realm.GlobalObject, Value('print'), print);
});

realm.evaluateScript(`
'use strict';

async function* numbers() {
  let i = 0;
  while (true) {
    const n = await Promise.resolve(i++);
    yield n;
  }
}

(async () => {
  for await (const item of numbers()) {
    print(item);
  }
})();
`);

// a stream of numbers fills your console. it fills you with determination.

Testing engine262

This project can be run against test262, which is particularly useful for developing new features and/or tests:

$ # build engine262
$ npm run build

$ # update local test262 in test/test262/test262
$ git submodule update --init --recursive

$ # update local test262 to a pull request
$ pushd test/test262/test262
$ git fetch origin refs/pull/$PR_NUMBER/head && git checkout FETCH_HEAD
$ popd

$ # run specific tests
$ npm run test:test262 built-ins/AsyncGenerator*

$ # run all tests
$ npm run test:test262

The output will indicate counts for total tests, passing tests, failing tests, and skipped tests.

Related Projects

Many people and organizations have attempted to write a JavaScript interpreter in JavaScript much like engine262, with different goals. Some of them are included here for reference, though engine262 is not based on any of them.

engine262's People

Contributors

ahabhgk avatar bakkot avatar bendtherules avatar caub avatar devsnek avatar exe-boss avatar gibson042 avatar jack-works avatar jakechampion avatar jamiebuilds avatar jridgewell avatar jugglinmike avatar lightmare avatar mischnic avatar nicolo-ribaudo avatar rbuckton avatar rwaldron avatar targos avatar timothygu avatar yaffle avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

engine262's Issues

Figure out how to do parenthesized LHS correctly

Back story (estree/estree#194): ESTree/Acorn doesn't provide us enough information to distinguish between

fn = function () {};

and

(fn) = function () {};

causing tc39/test262 language/expressions/assignment/fn-name-lhs-cover.js to fail. As estree/estree#194 attempts to find a solution in ESTree, we should find a way to patch Acorn so that we have this information for ourselves right now.

@devsnek assigning this to you as you've done some Acorn work recently.

Fail to build

$ npx rollup -c

./src/api.mjs → dist/engine262.js, dist/engine262.mjs...
[!] Error: Unexpected token (Note that you need plugins to import files that are not JavaScript)
src/intrinsics/RegExpPrototype.mjs (163:14)
161: 
162:   const first = S.stringValue().charCodeAt(index);
163:   if (first < 0xD800 || first > 0xDBFF) {
                   ^
164:     return new Value(index + 1);
165:   }
Error: Unexpected token (Note that you need plugins to import files that are not JavaScript)
    at error (/home/timothy-gu/dev/engine262/node_modules/rollup/dist/rollup.js:3597:30)
    at Module.error (/home/timothy-gu/dev/engine262/node_modules/rollup/dist/rollup.js:14376:9)
    at tryParse (/home/timothy-gu/dev/engine262/node_modules/rollup/dist/rollup.js:14039:16)
    at Module.setSource (/home/timothy-gu/dev/engine262/node_modules/rollup/dist/rollup.js:14095:35)
    at /home/timothy-gu/dev/engine262/node_modules/rollup/dist/rollup.js:17409:20
    at process.runNextTicks [as _tickCallback] (internal/process/next_tick.js:40:5)
    at Function.Module.runMain (internal/modules/cjs/loader.js:774:11)
    at findNodeScript.then.existing (/usr/local/lib/node_modules/npm/node_modules/libnpx/index.js:268:14)

Built-in functions should explicitly declare whether it is a constructor

CreateBuiltinFunction makes no mention of whether the created function object is a constructor or not (i.e., whether or not it has a [[Construct]] internal method). It only says

  1. Let func be a new built-in function object that when called performs the action described by steps.

However, the oft-missed Clause 18 says:

Built-in function objects that are not identified as constructors do not implement the [[Construct]] internal method unless otherwise specified in the description of a particular function.

So to identify whether each built-in function object is a constructor or not, we need some additional information.

Currently, what we do is a heuristic that searches for the string NewTarget in the body of the function implementation, and if it is present that we treat it as a constructor: https://github.com/devsnek/engine262/blob/5427f25f9134fb8793aa7e4f4d94ac14dd2bc060/src/value.mjs#L287

However, it doesn't work well with all cases, such as %TypedArray%. For now we employ a hack that has the string NewTarget in a comment: https://github.com/devsnek/engine262/blob/5427f25f9134fb8793aa7e4f4d94ac14dd2bc060/src/intrinsics/TypedArray.mjs#L21-L26 but we should just pass an additional flag to CreateBuiltinFunction.

Built-in functions shouldn't automatically fill in Value.undefined arguments

See https://github.com/devsnek/engine262/blob/5427f25f9134fb8793aa7e4f4d94ac14dd2bc060/src/value.mjs#L262-L271

This turns out to not work with a bunch of methods that accept less than their declared arity, like Array.prototype.unshift which accepts 0 args but has length property be 1.

We can either make every function explicitly set the default value for a required argument to Value.undefined, or make Bootstrap functions take an additional option for how many parameters the function requires. I vote for the former, as Bootstrap methods already take a lot of arguments (name, value, length, descriptor). Additionally, it's nice to group everything about how a function is evaluated at one place rather than scattered between the function itself and the Bootstrapper.


For reference, the spec says:

Unless otherwise specified in the description of a particular function, if a built-in function or constructor is given fewer arguments than the function is specified to require, the function or constructor shall behave exactly as if it had been given sufficient additional arguments, each such argument being the undefined value. Such missing arguments are considered to be “not present” and may be identified in that manner by specification algorithms.

inspect() is doing something wrong with custom error object

While implementing first class host support for engine262 in eshost, I came across this issue:

$ engine262
> function MyError(msg) {this.message = msg;} MyError.prototype.name = 'MyError';MyError.prototype.toString = function() {return 'MyError: ' + this.message;};throw new MyError('FAIL!');
{ message: 'FAIL!' }
> var f = new MyError("wtf");
undefined
> print(f)
{ message: 'wtf' }
undefined
> f
{ message: 'wtf' }
> String(f)
MyError: wtf
>

Implement Date

Implement https://tc39.github.io/ecma262/#sec-date-object

Remaining TODOs:

Some tests are failing

$ node test/test262.js test/language/statements/class/strict-mode/arguments-callee.js test/built-ins/ThrowTypeError/unique-per-realm-non-simple.js
(node:10069) ExperimentalWarning: Readable[Symbol.asyncIterator] is an experimental feature. This feature could change at any time
[00:00|:    1|+    0|-    1|»    0]: test/language/statements/class/strict-mode/arguments-callee.js
{ message: 'Expected a TypeError to be thrown but no exception was thrown at all' }
$ node test/test262.js test/language/statements/class/strict-mode/arguments-callee.js test/language/expressions/object/dstr/async-gen-meth-dflt-ary-ptrn-elem-ary-elision-init.js
(node:9999) ExperimentalWarning: Readable[Symbol.asyncIterator] is an experimental feature. This feature could change at any time
[00:00|:    1|+    0|-    1|»    0]: test/language/statements/class/strict-mode/arguments-callee.js
{ message: 'Expected a TypeError to be thrown but no exception was thrown at all' } 

crash with Object.keys(String.prototype)

> Object.keys(String.prototype)
TypeError: Assert failed
    at Evaluate_CallExpression$$1 (C:\Users\Michael\Documents\git\targos\engine262\dist\engine262.js:4233:18)
    at EvaluateCall$$1.next (<anonymous>)
    at EvaluateCall$$1 (C:\Users\Michael\Documents\git\targos\engine262\dist\engine262.js:4176:19)
    at Call$$1 (C:\Users\Michael\Documents\git\targos\engine262\dist\engine262.js:12810:13)
    at BuiltinFunctionValue.Call (C:\Users\Michael\Documents\git\targos\engine262\dist\engine262.js:19599:21)
    at nativeCall (C:\Users\Michael\Documents\git\targos\engine262\dist\engine262.js:19553:13)
    at BuiltinFunctionValue.Object_keys [as nativeFunction] (C:\Users\Michael\Documents\git\targos\engine262\dist\engine262.js:22304:19)
    at EnumerableOwnPropertyNames$$1 (C:\Users\Michael\Documents\git\targos\engine262\dist\engine262.js:13071:20)
    at StringExoticObjectValue.OwnPropertyKeys (C:\Users\Michael\Documents\git\targos\engine262\dist\engine262.js:19685:8)
    at Assert (C:\Users\Michael\Documents\git\targos\engine262\dist\engine262.js:35:12)

Issues disambiguating stdout contents

All shells that eshost supports currently provide some distinguishable error stack output that can be easily disambiguated through some regular expression pattern matching process. Presently, engine262 prints all process output to process.stdout, so eshost must implement a normalizeResult method in the ConsoleAgent subclass for engine262. Unfortunately, there is no way to reliably disambiguate between, for example, the following values:

foo\n
SyntaxError\n

I'm going to re-use the format and method that I used in a previous issue, as I think it was helpful to illustrate the issue:

// code.js

eval("\'\u000Astr\u000Aing\u000A\'");

In node repl:

$ node
> child_process.spawnSync("engine262", [ "./code.js" ]).stdout.toString()
'SyntaxError\n'

Compared to:

...JSC:

> child_process.spawnSync("jsc", [ "./code.js" ]).stdout.toString()
'Exception: SyntaxError: Unexpected EOF\nat ./code.js:7\neval@[native code]\nglobal code@./code.js:7:5\n'

...d8:

> child_process.spawnSync("d8", [ "./code.js" ]).stdout.toString()
'undefined:1: SyntaxError: Invalid or unexpected token\n\'\n^\nSyntaxError: Invalid or unexpected token\n    at ./code.js:7:1\n\n'

All of the others have similar information that can be extracted and used to disambiguate.

Now, if I had some code that produces stdout that is not an error:

// code.js

print(1 === 1);

In node repl:

$ node
> child_process.spawnSync("engine262", [ "./code.js" ]).stdout.toString()
'true\n'

I think there a few possible fixes:

  1. Move error output to stderr (but I think this might require quite a bit of refactoring work, I'm not sure)
  2. Make all error output include certain standard pieces of information:
Error: message
    at file:N:N

For example, a SyntaxError on line 1, at character 1, in "foo.js"

SyntaxError: Invalid or unexpected token
    at foo.js:1:1

The same error, but from eval():

SyntaxError: Invalid or unexpected token
    at <eval>:1:1

I don't even think it has to be exactly like the examples I've given, but there needs to be some way to know the difference between the output of these:

function MyError() {}
MyError.prototype.name = 'MyError';
throw new MyError();
print('MyError');

Presently, both produce 'MyError\n' in engine262, whereas eshost needs something like:

MyError: undefined
  at code.js:3:1

Publish to npm?

It would be great if this could be installed as a package from npm

ecma262 changes

Some more tests are failing

[00:25|: 3508|+ 1868|-    1|» 1639]: test/language/arguments-object/10.6-10-c-ii-1.js
{ message: 'foo(10,"sss",1) !== true' } 

[00:25|: 3509|+ 1868|-    2|» 1639]: test/language/arguments-object/10.6-10-c-ii-2.js
{ message: 'foo(10,"sss",1) !== true' } 

[01:33|:11175|+ 8261|-    3|» 2911]: test/built-ins/Object/defineProperty/15.2.3.6-4-292-1.js
{ message: 'Expected a === 20, actually 0' } 

[01:33|:11181|+ 8266|-    4|» 2911]: test/built-ins/Object/defineProperty/15.2.3.6-4-293-2.js
{ message: 'Expected "a === 10", actually 0' } 

[01:33|:11182|+ 8266|-    5|» 2911]: test/built-ins/Object/defineProperty/15.2.3.6-4-293-3.js
{ message: 'Expected "a === 10", actually 0' } 

[01:33|:11186|+ 8269|-    6|» 2911]: test/built-ins/Object/defineProperty/15.2.3.6-4-294-1.js
{ message: 'Expected "a === 10", actually 0' } 

[01:33|:11187|+ 8269|-    7|» 2911]: test/built-ins/Object/defineProperty/15.2.3.6-4-295-1.js
{ message: 'Expected "a === 10", actually 0' } 

[01:33|:11192|+ 8273|-    8|» 2911]: test/built-ins/Object/defineProperty/15.2.3.6-4-296-1.js
{ message: 'Expected "a === 10", actually 0' } 

[02:13|:15982|+11754|-    9|» 4219]: test/language/arguments-object/mapped/mapped-arguments-nonconfigurable-2.js
{ message: 'Expected SameValue(«1», «2») to be true' } 

[02:13|:15983|+11754|-   10|» 4219]: test/language/arguments-object/mapped/mapped-arguments-nonconfigurable-3.js
{ message: 'Expected SameValue(«1», «2») to be true' } 

[02:13|:15984|+11754|-   11|» 4219]: test/language/arguments-object/mapped/mapped-arguments-nonconfigurable-4.js
{ message: 'Expected SameValue(«1», «2») to be true' } 

[02:13|:15986|+11755|-   12|» 4219]: test/language/arguments-object/mapped/mapped-arguments-nonconfigurable-delete-2.js
{ message: 'Expected SameValue(«1», «2») to be true' } 

[02:13|:15987|+11755|-   13|» 4219]: test/language/arguments-object/mapped/mapped-arguments-nonconfigurable-delete-3.js
{ message: 'Expected SameValue(«1», «2») to be true' } 

[02:13|:15988|+11755|-   14|» 4219]: test/language/arguments-object/mapped/mapped-arguments-nonconfigurable-delete-4.js
{ message: 'Expected SameValue(«1», «2») to be true' } 

[02:13|:15991|+11757|-   15|» 4219]: test/language/arguments-object/mapped/mapped-arguments-nonconfigurable-nonwritable-3.js
{ message: 'Expected SameValue(«1», «2») to be true' } 

[02:13|:15992|+11757|-   16|» 4219]: test/language/arguments-object/mapped/mapped-arguments-nonconfigurable-nonwritable-4.js
{ message: 'Expected SameValue(«1», «2») to be true' } 

[02:13|:15993|+11757|-   17|» 4219]: test/language/arguments-object/mapped/mapped-arguments-nonconfigurable-nonwritable-5.js
{ message: 'Expected SameValue(«1», «2») to be true' } 

[02:13|:15995|+11758|-   18|» 4219]: test/language/arguments-object/mapped/mapped-arguments-nonconfigurable-strict-delete-2.js
{ message: 'Expected SameValue(«1», «2») to be true' } 

[02:13|:15996|+11758|-   19|» 4219]: test/language/arguments-object/mapped/mapped-arguments-nonconfigurable-strict-delete-3.js
{ message: 'Expected SameValue(«1», «2») to be true' } 

[02:13|:15997|+11758|-   20|» 4219]: test/language/arguments-object/mapped/mapped-arguments-nonconfigurable-strict-delete-4.js
{ message: 'Expected SameValue(«1», «2») to be true' } 

[02:13|:16003|+11763|-   21|» 4219]: test/language/arguments-object/mapped/nonconfigurable-descriptors-set-value-by-arguments.js
{ message: 'Expected SameValue(«1», «2») to be true' } 

[02:13|:16004|+11763|-   22|» 4219]: test/language/arguments-object/mapped/nonconfigurable-descriptors-set-value-with-define-property.js
{ message: 'Expected SameValue(«1», «2») to be true' } 

[02:13|:16005|+11763|-   23|» 4219]: test/language/arguments-object/mapped/nonconfigurable-descriptors-with-param-assign.js
{ message: 'Expected SameValue(«1», «2») to be true' } 

[02:13|:16007|+11764|-   24|» 4219]: test/language/arguments-object/mapped/nonconfigurable-nonenumerable-nonwritable-descriptors-set-by-arguments.js
{ message: 'Expected SameValue(«1», «2») to be true' } 

[02:13|:16008|+11764|-   25|» 4219]: test/language/arguments-object/mapped/nonconfigurable-nonenumerable-nonwritable-descriptors-set-by-param.js
{ message: 'Expected SameValue(«1», «2») to be true' } 

[02:13|:16011|+11766|-   26|» 4219]: test/language/arguments-object/mapped/nonconfigurable-nonwritable-descriptors-set-by-arguments.js
{ message: 'Expected SameValue(«1», «2») to be true' } 

[02:13|:16012|+11766|-   27|» 4219]: test/language/arguments-object/mapped/nonconfigurable-nonwritable-descriptors-set-by-param.js
{ message: 'Expected SameValue(«1», «2») to be true' } 

[03:04|:21632|+16765|-   28|» 4839]: test/language/expressions/yield/formal-parameters-after-reassignment-non-strict.js
{ message: 'First result `value` Expected SameValue(«23», «32») to be true' } 

[03:09|:22430|+17224|-   29|» 5177]: test/language/statements/async-function/evaluation-mapped-arguments.js
Test262:AsyncTestFailure:Test262Error: Test262Error: Expected SameValue(«1», «2») to be true 

[03:20|:25703|+18084|-   30|» 7589]: test/language/statements/for-of/arguments-mapped-aliasing.js
{ message: 'argument at index 1 Expected SameValue(«2», «3») to be true' }

@devsnek

Fixing non-do transform

Things that need to be fixed (that i know about):

  • Nested calls Q(Thing(Q(Thing2())))
  • Things in blocks
    { const a = Q(Thing()); } becomes:
    let a;
    if (a instanceof AbruptCompletion) { return a; }
    if (a instanceof Completion) { a = a.Value; }
    { a = Thing(); }
    notice how a is used before its set
  • Concise body for arrow functions () => Q(Thing()). This one seems easily fixable.
  • Conditional expressions boo ? Q(DoNotExecute()) : fallback. DoNotExecute() is currently always executed.

Syntax issues with strict mode and eval

There are a number of issues related to strict mode and eval and I'm not sure how we should proceed to fix them.

I already monkey patched acorn to allow forcing the strict flag to true (used when eval is called in a strict context).

Some problems that I saw with the current state of things:

  • we cannot eval code that is only valid in a specific context
  • there are new constructs, like classes, that are always strict, even in sloppy mode. The current way our runtime strict checking is implemented (strict flag in AST nodes) doesn't work with that.

CLI support for -m or --module to evaluate .js as module

Nearly all of the shells that eshost works with provides support for module evaluation via a flag, eg. -m or --module. I've written a small patch for engine262 to support both of those flags (while preserving the .mjs behavior), but wanted to open an issue before I went ahead with a PR.

This is necessary because there is actually no specified requirement that modules must have .mjs, so all of Test262's tests, including module code, are in .js files.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.