Code Monkey home page Code Monkey logo

parser's Introduction

@observablehq/parser

To parse a cell:

import {parseCell} from "@observablehq/parser";

const cell = parseCell(`hello = "world"`);

Examples

(In these examples, the node.start and node.end indexes into the input string are not shown for brevity. If options.locations is true, node.loc will also be populated with the line and column numbers.)

An expression cell (where cell.body is a type of expression):

1 + 2
{
  "type": "Cell",
  "id": null,
  "async": false,
  "generator": false,
  "body": {
    "type": "BinaryExpression",
    "left": {
      "type": "Literal",
      "value": 1,
      "raw": "1"
    },
    "operator": "+",
    "right": {
      "type": "Literal",
      "value": 2,
      "raw": "2"
    }
  }
}

A block cell (where cell.body is a BlockStatement):

{
  return 1 + 2;
}
{
  "type": "Cell",
  "id": null,
  "async": false,
  "generator": false,
  "body": {
    "type": "BlockStatement",
    "body": [
      {
        "type": "ReturnStatement",
        "argument": {
          "type": "BinaryExpression",
          "left": {
            "type": "Literal",
            "value": 1,
            "raw": "1"
          },
          "operator": "+",
          "right": {
            "type": "Literal",
            "value": 2,
            "raw": "2"
          }
        }
      }
    ]
  }
}

An empty cell (where cell.body is null):

{
  "type": "Cell",
  "id": null,
  "async": false,
  "generator": false,
  "body": null
}

A named expression cell (where cell.id is an Identifier):

foo = 42
{
  "type": "Cell",
  "id": {
    "type": "Identifier",
    "name": "foo"
  },
  "async": false,
  "generator": false,
  "body": {
    "type": "Literal",
    "value": 42,
    "raw": "42"
  }
}

A named block cell (where cell.id is an Identifier):

foo = {
  return 42;
}
{
  "type": "Cell",
  "id": {
    "type": "Identifier",
    "name": "foo"
  },
  "async": false,
  "generator": false,
  "body": {
    "type": "BlockStatement",
    "body": [
      {
        "type": "ReturnStatement",
        "argument": {
          "type": "Literal",
          "value": 42,
          "raw": "42"
        }
      }
    ]
  }
}

An asynchronous expression cell (where cell.async is true):

2 * await value
{
  "type": "Cell",
  "id": null,
  "async": true,
  "generator": false,
  "body": {
    "type": "BinaryExpression",
    "left": {
      "type": "Literal",
      "value": 2,
      "raw": "2"
    },
    "operator": "*",
    "right": {
      "type": "AwaitExpression",
      "argument": {
        "type": "Identifier",
        "name": "value"
      }
    }
  }
}

A generator expression cell (where cell.generator is true):

yield* [1, 2, 3]
{
  "type": "Cell",
  "id": null,
  "async": false,
  "generator": true,
  "body": {
    "type": "YieldExpression",
    "delegate": true,
    "argument": {
      "type": "ArrayExpression",
      "elements": [
        {
          "type": "Literal",
          "value": 1,
          "raw": "1"
        },
        {
          "type": "Literal",
          "value": 2,
          "raw": "2"
        },
        {
          "type": "Literal",
          "value": 3,
          "raw": "3"
        }
      ]
    }
  }
}

A viewof expression cell (where cell.id is a ViewExpression):

viewof x = DOM.range()
{
  "type": "Cell",
  "id": {
    "type": "ViewExpression",
    "id": {
      "type": "Identifier",
      "name": "x"
    }
  },
  "async": false,
  "generator": false,
  "body": {
    "type": "CallExpression",
    "callee": {
      "type": "MemberExpression",
      "object": {
        "type": "Identifier",
        "name": "DOM"
      },
      "property": {
        "type": "Identifier",
        "name": "range"
      },
      "computed": false
    },
    "arguments": []
  }
}

A viewof reference within an expression cell (where cell.body contains a ViewExpression):

viewof x.tagName
{
  "type": "Cell",
  "id": null,
  "async": false,
  "generator": false,
  "body": {
    "type": "MemberExpression",
    "object": {
      "type": "ViewExpression",
      "id": {
        "type": "Identifier",
        "name": "x"
      }
    },
    "property": {
      "type": "Identifier",
      "name": "tagName"
    },
    "computed": false
  }
}

An import cell (where cell.body is an ImportDeclaration):

import {foo} from "module"
{
  "type": "Cell",
  "id": null,
  "async": false,
  "generator": false,
  "body": {
    "type": "ImportDeclaration",
    "specifiers": [
      {
        "type": "ImportSpecifier",
        "view": false,
        "imported": {
          "type": "Identifier",
          "name": "foo"
        },
        "local": {
          "type": "Identifier",
          "name": "foo"
        }
      }
    ],
    "source": {
      "type": "Literal",
      "value": "module",
      "raw": "\"module\""
    }
  }
}

Importing a view (where specifier.view is true):

import {viewof foo} from "module"
{
  "type": "Cell",
  "id": null,
  "async": false,
  "generator": false,
  "body": {
    "type": "ImportDeclaration",
    "specifiers": [
      {
        "type": "ImportSpecifier",
        "view": true,
        "imported": {
          "type": "Identifier",
          "name": "foo"
        },
        "local": {
          "type": "Identifier",
          "name": "foo"
        }
      }
    ],
    "source": {
      "type": "Literal",
      "value": "module",
      "raw": "\"module\""
    }
  }
}

Importing a view imports both the view symbol (viewof foo) and the value symbol (foo). Likewise, if the specified view is renamed during import (e.g., viewof foo as bar), both the view symbol and the value symbol are renamed (e.g., viewof bar and bar).

Importing with injection (where declaration.injections is present):

import {chart} with {sales as data} from "@mbostock/d3-bar-chart"
{
  "type": "Cell",
  "id": null,
  "async": false,
  "generator": false,
  "body": {
    "type": "ImportDeclaration",
    "specifiers": [
      {
        "type": "ImportSpecifier",
        "view": false,
        "imported": {
          "type": "Identifier",
          "name": "chart"
        },
        "local": {
          "type": "Identifier",
          "name": "chart"
        }
      }
    ],
    "injections": [
      {
        "type": "ImportSpecifier",
        "view": false,
        "imported": {
          "type": "Identifier",
          "name": "sales"
        },
        "local": {
          "type": "Identifier",
          "name": "data"
        }
      }
    ],
    "source": {
      "type": "Literal",
      "value": "@mbostock/d3-bar-chart",
      "raw": "\"@mbostock/d3-bar-chart\""
    }
  }
}

For an injection, specifier.imported and specifier.local are reversed compared to a normal specifier: they are from the perspective of the imported module rather than the importing module. So in the example above, the importing module’s variable sales is injected into the imported module (the chart), replacing the variable data.

Injecting a view injects both the view symbol (viewof foo) and the value symbol (foo). Likewise, if the specified view is renamed during injection (e.g., viewof foo as bar), both the view symbol and the value symbol are renamed (e.g., viewof bar and bar).

API Reference

# parseCell(input[, options]) <>

Returns a cell.

# peekId(input) <>

Tries to find the ID of a cell given a snippet of its contents, and returns it as a string if found.

Cell

# cell.id

The name of the cell: null if the cell is anonymous; otherwise an Identifier or a ViewExpression.

# cell.body

The body of the cell: null for an empty cell; an ImportDeclaration for an import cell; otherwise a BlockStatement or an expression node.

# cell.async

A boolean indicating whether the cell body is asynchronous (i.e., whether it contains an await statement). False for import and empty cells.

# cell.generator

A boolean indicating whether the cell body is a generator (i.e., whether it contains a yield statement). False for import and empty cells.

ViewExpression

# view.id

The view identifier: an Identifier.

ImportDeclaration

# declaration.injections

An array of ImportSpecifier nodes, if the import declaration has a with clause, and otherwise null.

ImportSpecifier

# specifier.view

A boolean indicating whether the import specifies a view.

parser's People

Contributors

asg017 avatar dependabot-preview[bot] avatar greenkeeper[bot] avatar jashkenas avatar mbostock avatar mourner avatar nebrius avatar sydneypalumbo avatar tmcw avatar tophtucker avatar visnup avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

parser's Issues

Special message for unexpected eof?

Using the generic unexpected token message in this case isn’t as helpful as it could be, because eof is a special sentinel token. For example:

42, // I am a comment.

The error is highlighted at the end of the comment rather than at the comma.

permit use of keywords in javascript expressions

currently, there are valid javascript expressions that use observables keywords mutable and viewof as identifiers that the observable parser will reject. the goal would be to allow these expressions where possible.

here are some examples of these identifiers that could be defined in a local cell scope:

{
  const viewof = 3;
}
function foo(viewof) { ... }

and at the notebook scope where cells can be named mutable and viewof:

function viewof() { ... }
viewof = 3

additionally, import semantics need to be worked out. what happens if a cell named viewof is imported from another notebook that defines it as a cell:

import { viewof } from "..."

this change is likely to alter logic in parse.js, and the Hello, Observable Parser notebook can be used to aid in testing.

broadly this change would be a preamble to possibly adding a builtin keyword to provide a namespace for all the built-in identifiers like width.

Promise for cell operator (promiseof)

It’d be nice if there were a way to opt-in to getting a promise of a cell’s next value. This would allow two things:

  1. A cell could start running before a cell it references has fulfilled. This would allow some operations to proceed in parallel, without needing to break these operations out into another cell.

  2. A cell could handle an error if a cell it references errors. For example, promiseof foo.catch(() => "default") or try { return await promiseof foo; } catch { return "default"; }. Related observablehq/runtime#286.

Add a CLI

For development, I've found the following to be extremely useful:

#!/usr/bin/env node -r esm

import {parseCell} from "./src/index";
import {readFileSync} from "fs";

const actual = parseCell(readFileSync(process.argv[2], "utf8"), {
  globals: null
});

actual.fileReferences = Array.from(actual.fileReferences.entries());
console.log(JSON.stringify(actual, null, 2));

So that I can write compact test files and re-run them without test noise. I think we should add something similar to the repo to assist in development for people with similar preferences.

An in-range update of acorn-walk is breaking the build 🚨

The dependency acorn-walk was updated from 6.1.1 to 6.2.0.

🚨 View failing branch.

This version is covered by your current version range and after updating it in your project the build failed.

acorn-walk is a direct dependency of this project, and it is very likely causing it to break. If other packages depend on yours, this update is probably also breaking those in turn.

Status Details
  • ci/circleci: Your tests failed on CircleCI (Details).

FAQ and help

There is a collection of frequently asked questions. If those don’t help, you can always ask the humans behind Greenkeeper.


Your Greenkeeper Bot 🌴

TypeScript support?

Do we have plan for TypeScript support or even consider it?
Not sure if this is the right place to ask questions like this though, my apologies!

An in-range update of eslint is breaking the build 🚨

The devDependency eslint was updated from 6.0.1 to 6.1.0.

🚨 View failing branch.

This version is covered by your current version range and after updating it in your project the build failed.

eslint is a devDependency of this project. It might not break your production code or affect downstream projects, but probably breaks your build or test tools, which may prevent deploying or publishing.

Status Details
  • ci/circleci: Your tests failed on CircleCI (Details).

Release Notes for v6.1.0
  • 8f86cca Upgrade: eslint-scope@^5.0.0 (#12011) (Kevin Partington)
  • d08683e Fix: glob processing (fixes #11940) (#11986) (Toru Nagashima)
  • bfcf8b2 Fix: dot-location errors with parenthesized objects (fixes #11868) (#11933) (Milos Djermanovic)
  • 79e8d09 Fix: add parens for sequence expr in arrow-body-style (fixes #11917) (#11918) (Pig Fang)
  • 105c098 Docs: update docs for object-curly-spacing (fixes #11634) (#12009) (Chiawen Chen)
  • c90a12c Chore: update release script for new website repo (#12006) (Kai Cataldo)
  • e2c08a9 Sponsors: Sync README with website (ESLint Jenkins)
  • b974fcb Update: Check computed property keys in no-extra-parens (#11952) (Milos Djermanovic)
  • 222d27c Update: Add for-in and for-of checks for props in no-param-reassign (#11941) (Milos Djermanovic)
  • e4c450f Fix: no-extra-parens autofix with in in a for-loop init (fixes #11706) (#11848) (Milos Djermanovic)
  • 2dafe2d Fix: prefer-const produces invalid autofix (fixes #11699) (#11827) (Milos Djermanovic)
  • cb475fd Fix: Cache file error handling on read-only file system. (fixes #11945) (#11946) (Cuki)
  • 89412c3 Docs: Fixed a typo (fixes #11999) (#12000) (Eddie Olson)
  • 6669f78 Fix: --init with Vue.js failed (fixes #11970) (#11985) (Toru Nagashima)
  • 93633c2 Upgrade: Upgrade lodash dependency (fixes #11992) (#11994) (Cyd La Luz)
  • 776dae7 Docs: fix wrong Node.js version in getting started (#11993) (Toru Nagashima)
  • 4448261 Docs: some typos and optimization points (#11960) (Jason Lee)
  • 2a10856 Chore: Add temporary test files to .gitignore (#11978) (Milos Djermanovic)
  • d83b233 Chore: update path for release bundles (#11977) (Kai Cataldo)
  • 1fb3620 Fix: creating of enabledGlobals object without prototype (fixes #11929) (#11935) (finico)
  • c2f2db9 Docs: Replace global true and false with writable and readonly in rules (#11956) (Milos Djermanovic)
  • 19335b8 Fix: actual messageId and expected messageId are switched in rule tester (#11928) (Milos Djermanovic)
  • 8b216e0 Docs: Fix incorrect example comments for unicode-bom rule (fixes #11937) (#11938) (Brandon Yeager)
  • cc3885b Chore: add v8-compile-cache to speed up instantiation time (#11921) (薛定谔的猫)
  • d8f2688 Upgrade: deps (#11904) (薛定谔的猫)
  • e5f1ccc Docs: add 'stricter rule config validating' in migrating docs (#11905) (薛定谔的猫)
Commits

The new version differs by 28 commits.

  • 02d7542 6.1.0
  • 5997f7a Build: changelog update for 6.1.0
  • 8f86cca Upgrade: eslint-scope@^5.0.0 (#12011)
  • d08683e Fix: glob processing (fixes #11940) (#11986)
  • bfcf8b2 Fix: dot-location errors with parenthesized objects (fixes #11868) (#11933)
  • 79e8d09 Fix: add parens for sequence expr in arrow-body-style (fixes #11917) (#11918)
  • 105c098 Docs: update docs for object-curly-spacing (fixes #11634) (#12009)
  • c90a12c Chore: update release script for new website repo (#12006)
  • e2c08a9 Sponsors: Sync README with website
  • b974fcb Update: Check computed property keys in no-extra-parens (#11952)
  • 222d27c Update: Add for-in and for-of checks for props in no-param-reassign (#11941)
  • e4c450f Fix: no-extra-parens autofix with in in a for-loop init (fixes #11706) (#11848)
  • 2dafe2d Fix: prefer-const produces invalid autofix (fixes #11699) (#11827)
  • cb475fd Fix: Cache file error handling on read-only file system. (fixes #11945) (#11946)
  • 89412c3 Docs: Fixed a typo (fixes #11999) (#12000)

There are 28 commits in total.

See the full diff

FAQ and help

There is a collection of frequently asked questions. If those don’t help, you can always ask the humans behind Greenkeeper.


Your Greenkeeper Bot 🌴

Scoped package?

This is currently acorn-observable, but should probably be @observablehq/acorn-observable.

SyntaxError when destructuring onto instance properties

Describe the bug

Demo: https://observablehq.com/d/1fb03f01444425d2

Given a cell foo with

foo = "bar"

when defining a cell where the value is referenced as a destructuring default for an instance property:

function Test1(opts = {}) {
  ({a: this.a = foo} = opts);
}  

Observable will erroneously report

SyntaxError: Assignment to constant variable foo

To avoid the error, the cell value has to be assigned to an intermediate variable.

For example, the following pattern works:

function Test2(opts = {}) {
  const _foo = foo;
  ({a: this.a = _foo} = opts);
}  

as does:

function Test3(opts = {}, bar = foo) {
  ({a: this.a = bar} = opts);
}  

An in-range update of acorn is breaking the build 🚨

The dependency acorn was updated from 6.2.0 to 6.2.1.

🚨 View failing branch.

This version is covered by your current version range and after updating it in your project the build failed.

acorn is a direct dependency of this project, and it is very likely causing it to break. If other packages depend on yours, this update is probably also breaking those in turn.

Status Details
  • ci/circleci: Your tests failed on CircleCI (Details).

FAQ and help

There is a collection of frequently asked questions. If those don’t help, you can always ask the humans behind Greenkeeper.


Your Greenkeeper Bot 🌴

`import with` should allow constant as LHS of `as`

See for example https://observablehq.com/@airbornemint/units

I would like to import it using

import { m } with { "," as thousands_separator } from "@airbornemint/units"

But that gives me a parse error and I have to use two cells, one to define new_thousands_separator = "," and the other to import { m } with { new_thousands_separator as thousands_separator } from "@airbornemint/units", which seems unnecessarily verbose for what is basically a module-level configuration parameter specified at the time of import.

Support escapes in tag parsing

Use case: Display an inline code block containing the literal text <div ...${props} /> using a Markdown cell

Expected behavior: escape the ${} with a \: <div ...\${props} /> or <div ...$\{props} /> or <div ...\$\{props} />

Actual results:

  • <div ...\${props} /> tries to evaluate props
  • <div ...$\{props} /> outputs <div ...$\{props} />
  • <div ...\$\{props} /> outputs <div ...\$\{props} />

An in-range update of rollup-plugin-commonjs is breaking the build 🚨

The devDependency rollup-plugin-commonjs was updated from 10.0.1 to 10.0.2.

🚨 View failing branch.

This version is covered by your current version range and after updating it in your project the build failed.

rollup-plugin-commonjs is a devDependency of this project. It might not break your production code or affect downstream projects, but probably breaks your build or test tools, which may prevent deploying or publishing.

Status Details
  • ci/circleci: Your tests failed on CircleCI (Details).

Commits

The new version differs by 3 commits.

See the full diff

FAQ and help

There is a collection of frequently asked questions. If those don’t help, you can always ask the humans behind Greenkeeper.


Your Greenkeeper Bot 🌴

Mark `id` as derived

Sometimes this module yields an id node that's derived from a function or class name, and other times it returns an id from id = at the beginning of the node. For clarity, for printing its AST and choosing when to prepend id =, we should indicate in the AST which is which.

An in-range update of rollup is breaking the build 🚨

The devDependency rollup was updated from 1.14.2 to 1.14.3.

🚨 View failing branch.

This version is covered by your current version range and after updating it in your project the build failed.

rollup is a devDependency of this project. It might not break your production code or affect downstream projects, but probably breaks your build or test tools, which may prevent deploying or publishing.

Status Details
  • ci/circleci: Your tests failed on CircleCI (Details).

Release Notes for v1.14.3

2019-06-06

Bug Fixes

  • Generate correct external imports when importing from a directory that would be above the root of the current working directory (#2902)

Pull Requests

Commits

The new version differs by 4 commits.

See the full diff

FAQ and help

There is a collection of frequently asked questions. If those don’t help, you can always ask the humans behind Greenkeeper.


Your Greenkeeper Bot 🌴

An in-range update of husky is breaking the build 🚨

The devDependency husky was updated from 3.0.2 to 3.0.3.

🚨 View failing branch.

This version is covered by your current version range and after updating it in your project the build failed.

husky is a devDependency of this project. It might not break your production code or affect downstream projects, but probably breaks your build or test tools, which may prevent deploying or publishing.

Status Details
  • ci/circleci: Your tests failed on CircleCI (Details).

Release Notes for v3.0.3

Fix: prevent old hooks (husky < 1.0) to be run if new ones are defined (husky >= 1.0 ) #556

Commits

The new version differs by 12 commits.

See the full diff

FAQ and help

There is a collection of frequently asked questions. If those don’t help, you can always ask the humans behind Greenkeeper.


Your Greenkeeper Bot 🌴

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.