denoland / deno_ast Goto Github PK

View Code? Open in Web Editor NEW

137.0 22.0 40.0 489 KB

Source text parsing, lexing, and AST related functionality for Deno

Home Page: https://crates.io/crates/deno_ast

License: MIT License

Rust 97.25% TypeScript 2.75%

deno_ast's Introduction

deno_ast

Source text parsing, lexing, and AST related functionality for Deno.

use deno_ast::parse_module;
use deno_ast::MediaType;
use deno_ast::ParseParams;
use deno_ast::SourceTextInfo;

let source_text = "class MyClass {}";
let text_info = SourceTextInfo::new(source_text.into());
let parsed_source = parse_module(ParseParams {
  specifier: deno_ast::ModuleSpecifier::parse("file:///my_file.ts").unwrap(),
  media_type: MediaType::TypeScript,
  text_info,
  capture_tokens: true,
  maybe_syntax: None,
  scope_analysis: false,
}).expect("should parse");

// returns the comments
parsed_source.comments();
// returns the tokens if captured
parsed_source.tokens();
// returns the module (AST)
parsed_source.module();
// returns the `SourceTextInfo`
parsed_source.text_info();

swc upgrades

We upgrade swc about once a month. Upgrading swc is a very involved process that often requires many changes in downstream Deno crates. We also test the new version of swc in all downstream crates before merging a PR into deno_ast that updates swc.

Please do not open a PR for upgrading swc unless you have stated you are going to work on it in the issue tracker and have run the tests on all downstream crates.

To upgrade swc:

Checkout the following repositories in sibling directories to this repository (ex. /home/david/dev/deno_graph, /home/david/dev/deno_ast):
Ensure they all have upstream remotes set for the urls specified above. For example, your deno_graph repo should have git remote add upstream https://github.com/denoland/deno_graph.
Run ./scripts/update_swc_deps.ts
Run ./scripts/01_setup.ts
Run ./scripts/02_build.ts and fix any build errors.
Run ./scripts/03_test.ts and fix any failing tests.
- At this point, all the tests should be passing in all the repositories.
Open a PR for upgrading deno_ast.
Merge the PR and publish a new version of deno_ast using the release workflow.
At this point, bump the version of deno_ast and open PRs for deno_graph, deno_lint, and dprint-plugin-typescript (note: ./scripts/04_confirm.ts might be helpful to automate some of this. Read its source code to understand it so that you can deal with any problems that may arise).
Merge the PR deno_graph and publish using its release workflow.
Open PRs to deno_emit, deno_doc, and eszip.
Merge those PRs and do releases. Also merge and release deno_lint and dprint-plugin-typescript (Bartek and David have access to publish dprint-plugin-typescript).
Open a PR to Deno with the versions bumped and merge the PR.

deno_ast's People

Contributors

Stargazers

Watchers

Forkers

dsherret isgasho magurotuna bartlomieju hoangpq happysmile12321 maxxcs ah-yu odenlang littledivy wrnrlr isabella232 cd-work gabssnake phial3 clownsw leodog896 cre3per whymidnight losfair fairhopeweb 4812571 mayhemheroes rossdylan alyti await-ovo sido378 jollytoad seanpm2001 florianlebrun shabbirhasan1 ahuigo unyt-org melbourne2991 eryue0220 runelk alshdavid alshdavid-forks chriss-0x01 petamoriken

deno_ast's Issues

Suggestion: Move transpiling and bundling to deno_ast?

I think it would be beneficial to:

Move all transpiling and bundling out of the CLI and to deno_ast. It would be in a feature flag. Obviously this would be bundling without deno_graph so there would be some adapter that deno_graph would be plugged into.
Add a ton of bundling and transpiling tests in here.
Pin all swc deps to a specific patch version so everything upstream gets tested on the same swc deps (might be good to do this anyway)

I think this would allow us to:

Ensure everything is in a good state before we upgrade all the upstream crates. (We don't find something is wrong after integrating into the cli).
Help us refactor out Deno.emit to a Wasm module that could connect this with deno_graph.

@kitsonk thoughts? I think this would be quick to do. I could start soon if you agree.

Enable JSX Namespace by default

I'm raising this here, as I've just discovered that this repo is where this issue could be fixed: denoland/deno#20345

deno_ast/src/transpiling/mod.rs

Line 338 in 2910cf3

throw_if_namespace: None,

I believe throw_if_namespace could just be set to Some(false) to solve this?

Unless you want to provide it as an option, but I don't think that's necessary. TypeScript doesn't make this restriction, so I don't know why swc should.

Add hashing on EmitOptions

https://github.com/denoland/deno/pull/15220/files#r923841773

Move deno_lint scope analysis into deno_ast

It would be useful in dnt.

https://github.com/denoland/deno_lint/blob/main/src/scopes.rs

`MediaType::as_ts_script_kind()` returns the wrong value for JSON

It currently returns 5 and should actually return 6.

Improve diagnostic to have the ability to format with source code text

For denoland/deno#8529

This would allow taking a Diagnostic and then formatting it showing the position in the source file text the diagnostic occurred for.

Update swc_ecmascript

denoland/deno#10861 and denoland/deno#12532 have been fixed by swc-project/swc#2530, but swc_ecmascript hasn't been updated in a while.

cc @dsherret

CommonJS analysis gets confused for a bundled file

Reported in: denoland/deno#23265 (reply in thread)

Our CJS analysis gets confused by code like this:

/***/ "uuid":
/*!***********************!*\
  !*** external "uuid" ***!
  \***********************/
/***/ ((module) => {

module.exports = require("uuid");

/***/ }),

/***/ "stream":
/*!*************************!*\
  !*** external "stream" ***!
  \*************************/
/***/ ((module) => {

module.exports = require("stream");

/***/ }),

/***/ "zlib":
/*!***********************!*\
  !*** external "zlib" ***!
  \***********************/
/***/ ((module) => {

module.exports = require("zlib");

/***/ })

The CJS analysis in this case tells that zlib is a reexport (but not "uuid" or "stream"). I think this is completely wrong as it's not a reexport of the actual file, because it's inside a closure that provides module variable.

Complete file: https://gist.github.com/bartlomieju/a00a74d32947eef6a0811483d8619a0e

Reproduction in Deno:

import * as dicomStream from "@exini/dicom-streams-js";

$ deno run script.js

Upgrade swc after resolution of swc issues

The new OptCall in swc makes a lot of our code more error prone. We will upgrade once this issue is resolved or alternatively change dprint-swc-ecma-ast-view to combine these two nodes together in the wrapper: swc-project/swc#4010

Add functionality equivalent for "cjs-module-lexer"

https://github.com/nodejs/cjs-module-lexer

Basically we need a function, that given a CJS module source will return a list of exports and reexports.

We need this functionality for denoland/deno#12648 and denoland/deno#13200

deno-ast 0.15 does not compile

My guess is there are some swc feature flags that deno-ast is depending on implicitly, but I have not debugged yet.

$ rustc --version    
rustc 1.61.0 (fe5b13d68 2022-05-18)

I created a new cargo project and added the following dependency:

deno_ast = "0.15.0"

The body of the code is:

use deno_ast::parse_module;
use deno_ast::MediaType;
use deno_ast::ParseParams;
use deno_ast::SourceTextInfo;
use std::sync::Arc;

fn main() {
    let source_text = Arc::new("class MyClass {}");
    let text_info = SourceTextInfo::new(source_text);
    let parsed_source = parse_module(ParseParams {
      specifier: "file:///my_file.ts".to_string(),
      media_type: MediaType::TypeScript,
      text_info,
      capture_tokens: true,
      maybe_syntax: None,
      scope_analysis: false,
    }).expect("should parse");
    
    // returns the comments
    parsed_source.comments();
    // returns the tokens if captured
    parsed_source.tokens();
    // returns the module (AST)
    parsed_source.module();
    // returns the `SourceTextInfo`
    parsed_source.source();
}

Compiling:

   Compiling swc_ecma_parser v0.104.2
error[E0599]: no method named `span_lo` found for struct `Box<swc_ecma_ast::Expr>` in the current scope
   --> /home/carllerche/.cargo/registry/src/github.com-1ecc6299db9ec823/swc_ecma_parser-0.104.2/src/parser/class_and_fn.rs:323:36
    |
323 |             span: span!(self, expr.span_lo()),
    |                                    ^^^^^^^ method not found in `Box<swc_ecma_ast::Expr>`

error[E0599]: no method named `span_lo` found for struct `Box<swc_ecma_ast::Expr>` in the current scope
  --> /home/carllerche/.cargo/registry/src/github.com-1ecc6299db9ec823/swc_ecma_parser-0.104.2/src/parser/expr/ops.rs:96:30
   |
96 |             let start = left.span_lo();
   |                              ^^^^^^^ method not found in `Box<swc_ecma_ast::Expr>`

error[E0599]: no method named `span_lo` found for struct `Box<swc_ecma_ast::Expr>` in the current scope
   --> /home/carllerche/.cargo/registry/src/github.com-1ecc6299db9ec823/swc_ecma_parser-0.104.2/src/parser/expr/ops.rs:215:34
    |
215 |             span: Span::new(left.span_lo(), right.span_hi(), Default::default()),
    |                                  ^^^^^^^ method not found in `Box<swc_ecma_ast::Expr>`

error[E0599]: no method named `span_hi` found for struct `Box<swc_ecma_ast::Expr>` in the current scope
   --> /home/carllerche/.cargo/registry/src/github.com-1ecc6299db9ec823/swc_ecma_parser-0.104.2/src/parser/expr/ops.rs:215:51
    |
215 |             span: Span::new(left.span_lo(), right.span_hi(), Default::default()),
    |                                                   ^^^^^^^ method not found in `Box<swc_ecma_ast::Expr>`

error[E0599]: no method named `span_hi` found for struct `Box<swc_ecma_ast::Expr>` in the current scope
   --> /home/carllerche/.cargo/registry/src/github.com-1ecc6299db9ec823/swc_ecma_parser-0.104.2/src/parser/expr/ops.rs:256:45
    |
256 |             let span = Span::new(start, arg.span_hi(), Default::default());
    |                                             ^^^^^^^ method not found in `Box<swc_ecma_ast::Expr>`

error[E0599]: no method named `span_hi` found for struct `Box<swc_ecma_ast::Expr>` in the current scope
   --> /home/carllerche/.cargo/registry/src/github.com-1ecc6299db9ec823/swc_ecma_parser-0.104.2/src/parser/expr/ops.rs:314:44
    |
314 |                 span: Span::new(start, arg.span_hi(), Default::default()),
    |                                            ^^^^^^^ method not found in `Box<swc_ecma_ast::Expr>`

error[E0599]: no method named `span_lo` found for struct `Box<swc_ecma_ast::Expr>` in the current scope
   --> /home/carllerche/.cargo/registry/src/github.com-1ecc6299db9ec823/swc_ecma_parser-0.104.2/src/parser/expr/ops.rs:347:40
    |
347 |                 span: span!(self, expr.span_lo()),
    |                                        ^^^^^^^ method not found in `Box<swc_ecma_ast::Expr>`

error[E0599]: no method named `span_lo` found for struct `Box<swc_ecma_ast::Expr>` in the current scope
  --> /home/carllerche/.cargo/registry/src/github.com-1ecc6299db9ec823/swc_ecma_parser-0.104.2/src/parser/expr.rs:22:26
   |
22 |         let start = expr.span_lo();
   |                          ^^^^^^^ method not found in `Box<swc_ecma_ast::Expr>`

error[E0599]: no method named `span_hi` found for struct `Box<swc_ecma_ast::Expr>` in the current scope
   --> /home/carllerche/.cargo/registry/src/github.com-1ecc6299db9ec823/swc_ecma_parser-0.104.2/src/parser/expr.rs:220:45
    |
220 |             let span = Span::new(start, alt.span_hi(), Default::default());
    |                                             ^^^^^^^ method not found in `Box<swc_ecma_ast::Expr>`

error[E0599]: no method named `span_lo` found for reference `&Box<swc_ecma_ast::Expr>` in the current scope
   --> /home/carllerche/.cargo/registry/src/github.com-1ecc6299db9ec823/swc_ecma_parser-0.104.2/src/parser/expr.rs:918:44
    |
918 |                     exprs.first().unwrap().span_lo(),
    |                                            ^^^^^^^ method not found in `&Box<swc_ecma_ast::Expr>`

error[E0599]: no method named `span_hi` found for reference `&Box<swc_ecma_ast::Expr>` in the current scope
   --> /home/carllerche/.cargo/registry/src/github.com-1ecc6299db9ec823/swc_ecma_parser-0.104.2/src/parser/expr.rs:919:43
    |
919 |                     exprs.last().unwrap().span_hi(),
    |                                           ^^^^^^^ method not found in `&Box<swc_ecma_ast::Expr>`

error[E0599]: no method named `span_lo` found for struct `Box<swc_ecma_ast::Expr>` in the current scope
   --> /home/carllerche/.cargo/registry/src/github.com-1ecc6299db9ec823/swc_ecma_parser-0.104.2/src/parser/expr.rs:960:36
    |
960 |         let tagged_tpl_start = tag.span_lo();
    |                                    ^^^^^^^ method not found in `Box<swc_ecma_ast::Expr>`

error[E0599]: no method named `span_lo` found for enum `swc_ecma_ast::Callee` in the current scope
    --> /home/carllerche/.cargo/registry/src/github.com-1ecc6299db9ec823/swc_ecma_parser-0.104.2/src/parser/expr.rs:1171:38
     |
1171 |             let span = Span::new(obj.span_lo(), self.input.last_pos(), Default::default());
     |                                      ^^^^^^^ method not found in `swc_ecma_ast::Callee`

error[E0599]: no method named `span_lo` found for enum `swc_ecma_ast::Callee` in the current scope
    --> /home/carllerche/.cargo/registry/src/github.com-1ecc6299db9ec823/swc_ecma_parser-0.104.2/src/parser/expr.rs:1172:34
     |
1172 |             debug_assert_eq!(obj.span_lo(), span.lo());
     |                                  ^^^^^^^ method not found in `swc_ecma_ast::Callee`

error[E0599]: no method named `span_lo` found for enum `swc_ecma_ast::Callee` in the current scope
    --> /home/carllerche/.cargo/registry/src/github.com-1ecc6299db9ec823/swc_ecma_parser-0.104.2/src/parser/expr.rs:1293:40
     |
1293 |             let span = span!(self, obj.span_lo());
     |                                        ^^^^^^^ method not found in `swc_ecma_ast::Callee`

error[E0599]: no method named `span_lo` found for enum `swc_ecma_ast::Callee` in the current scope
    --> /home/carllerche/.cargo/registry/src/github.com-1ecc6299db9ec823/swc_ecma_parser-0.104.2/src/parser/expr.rs:1294:34
     |
1294 |             debug_assert_eq!(obj.span_lo(), span.lo());
     |                                  ^^^^^^^ method not found in `swc_ecma_ast::Callee`

error[E0599]: no method named `span_hi` found for enum `swc_ecma_ast::MemberProp` in the current scope
    --> /home/carllerche/.cargo/registry/src/github.com-1ecc6299db9ec823/swc_ecma_parser-0.104.2/src/parser/expr.rs:1295:35
     |
1295 |             debug_assert_eq!(prop.span_hi(), span.hi());
     |                                   ^^^^^^^ method not found in `swc_ecma_ast::MemberProp`

error[E0599]: no method named `span_hi` found for struct `Box<swc_ecma_ast::Expr>` in the current scope
    --> /home/carllerche/.cargo/registry/src/github.com-1ecc6299db9ec823/swc_ecma_parser-0.104.2/src/parser/expr.rs:1634:60
     |
1634 | ...                   span: Span::new(start, alt.span_hi(), Default::default()),
     |                                                  ^^^^^^^ method not found in `Box<swc_ecma_ast::Expr>`

error[E0599]: no method named `span_lo` found for reference `&swc_ecma_ast::Expr` in the current scope
    --> /home/carllerche/.cargo/registry/src/github.com-1ecc6299db9ec823/swc_ecma_parser-0.104.2/src/parser/expr.rs:1841:58
     |
1841 |         Ok(self.state.potential_arrow_start == Some(expr.span_lo())
     |                                                          ^^^^^^^ method not found in `&swc_ecma_ast::Expr`

error[E0599]: no method named `span_lo` found for struct `Box<swc_ecma_ast::TsType>` in the current scope
    --> /home/carllerche/.cargo/registry/src/github.com-1ecc6299db9ec823/swc_ecma_parser-0.104.2/src/parser/typescript.rs:2152:42
     |
2152 |                     span: span!(self, ty.span_lo()),
     |                                          ^^^^^^^ method not found in `Box<swc_ecma_ast::TsType>`

error[E0599]: no method named `span_lo` found for struct `Box<swc_ecma_ast::TsType>` in the current scope
    --> /home/carllerche/.cargo/registry/src/github.com-1ecc6299db9ec823/swc_ecma_parser-0.104.2/src/parser/typescript.rs:2159:42
     |
2159 |                     span: span!(self, ty.span_lo()),
     |                                          ^^^^^^^ method not found in `Box<swc_ecma_ast::TsType>`

error[E0599]: no method named `span_lo` found for struct `swc_ecma_ast::Ident` in the current scope
    --> /home/carllerche/.cargo/registry/src/github.com-1ecc6299db9ec823/swc_ecma_parser-0.104.2/src/parser/typescript.rs:2262:26
     |
2262 |         let start = expr.span_lo();
     |                          ^^^^^^^ method not found in `swc_ecma_ast::Ident`

For more information about this error, try `rustc --explain E0599`.
error: could not compile `swc_ecma_parser` due to 22 previous errors

Add docs and implement CI

As title.

Move transpile and similar functionality from CLI to deno_ast

https://github.com/denoland/deno/blob/main/cli/ast/mod.rs

Example does not compile

While trying to run the example:

use deno_ast::parse_module;
use deno_ast::MediaType;
use deno_ast::ParseParams;
use deno_ast::SourceTextInfo;
use std::sync::Arc;

let source_text = Arc::new("class MyClass {}");
let text_info = SourceTextInfo::new(source_text);
let parsed_source = parse_module(ParseParams {
  specifier: "file:///my_file.ts".to_string(),
  media_type: MediaType::TypeScript,
  text_info,
  capture_tokens: true,
  maybe_syntax: None,
  scope_analysis: false,
}).expect("should parse");

// returns the comments
parsed_source.comments();
// returns the tokens if captured
parsed_source.tokens();
// returns the module (AST)
parsed_source.module();
// returns the `SourceTextInfo`
parsed_source.source();

I get an error saying:

error[E0308]: mismatched types
  --> src/parsers/javascript/parser.rs:32:45
   |
32 |         let text_info = SourceTextInfo::new(source_text);
   |                                             ^^^^^^^^^^^ expected `str`, found `&str`
   |

I've tried casting the string, but I can only ever get it to be a &str. Any help is appreciated, I'm not sure what I'm going wrong.

Compare cjs exports implementation with one in esm.sh

https://github.com/esm-dev/esm.sh/blob/master/packages/cjs-esm-exports/src/cjs.rs

We should at the minimum add more tests from here: https://github.com/esm-dev/esm.sh/blob/master/packages/cjs-esm-exports/src/test.rs

Transform for stripping comments from JS

          LGTM. Can we get a transform that would drop comments from JS sources but otherwise leave them intact? I think we could save some Kbs if we did that for internal code during snapshotting.

Originally posted by @bartlomieju in #221 (review)

deno_graph tests fail with swc 0.99.9

---- ast::tests::test_analyze_dependencies_import_assertions stdout ----
thread 'ast::tests::test_analyze_dependencies_import_assertions' panicked at 'assertion failed: `(left == right)`
  left: `None`,
 right: `Some("json")`', src\ast.rs:395:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

---- tests::test_create_graph_import_assertions stdout ----
thread 'tests::test_create_graph_import_assertions' panicked at 'assertion failed: `(left == right)`

Diff < left / right > :
 Object({
     "roots": Array([
         String(
             "file:///a/test01.ts",
         ),
     ]),
     "modules": Array([
         Object({
             "size": Number(
                 9,
             ),
             "mediaType": String(
                 "Json",
             ),
             "specifier": String(
                 "file:///a/a.json",
             ),
         }),
         Object({
             "size": Number(
                 7,
             ),
             "mediaType": String(
                 "Json",
             ),
             "specifier": String(
                 "file:///a/b.json",
             ),
         }),
         Object({
             "size": Number(
                 9,
             ),
             "mediaType": String(
                 "Json",
             ),
             "specifier": String(
                 "file:///a/c.json",
             ),
         }),
         Object({
             "size": Number(
                 7,
             ),
             "mediaType": String(
                 "Json",
             ),
             "specifier": String(
                 "file:///a/d.json",
             ),
         }),
         Object({
             "dependencies": Array([
                 Object({
                     "specifier": String(
                         "./a.json",
                     ),
                     "code": Object({
                         "specifier": String(
                             "file:///a/a.json",
                         ),
                         "span": Object({
                             "start": Object({
                                 "line": Number(
                                     1,
                                 ),
                                 "character": Number(
                                     26,
                                 ),
                             }),
                             "end": Object({
                                 "line": Number(
                                     1,
                                 ),
                                 "character": Number(
                                     36,
                                 ),
                             }),
                         }),
                     }),
                     "assertionType": String(
                         "json",
                     ),
                 }),
                 Object({
                     "specifier": String(
                         "./b.json",
                     ),
                     "code": Object({
                         "specifier": String(
                             "file:///a/b.json",
                         ),
                         "span": Object({
                             "start": Object({
                                 "line": Number(
                                     2,
                                 ),
                                 "character": Number(
                                     35,
                                 ),
                             }),
                             "end": Object({
                                 "line": Number(
                                     2,
                                 ),
                                 "character": Number(
                                     45,
                                 ),
                             }),
                         }),
                     }),
                     "isDynamic": Bool(
                         true,
                     ),
>                    "assertionType": String(
>                        "json",
>                    ),
                 }),
                 Object({
                     "specifier": String(
                         "./c.json",
                     ),
                     "code": Object({
                         "specifier": String(
                             "file:///a/c.json",
                         ),
                         "span": Object({
                             "start": Object({
                                 "line": Number(
                                     3,
                                 ),
                                 "character": Number(
                                     31,
                                 ),
                             }),
                             "end": Object({
                                 "line": Number(
                                     3,
                                 ),
                                 "character": Number(
                                     41,
                                 ),
                             }),
                         }),
                     }),
                     "assertionType": String(
                         "json",
                     ),
                 }),
                 Object({
                     "specifier": String(
                         "./d.json",
                     ),
                     "code": Object({
                         "specifier": String(
                             "file:///a/d.json",
                         ),
                         "span": Object({
                             "start": Object({
                                 "line": Number(
                                     5,
                                 ),
                                 "character": Number(
                                     35,
                                 ),
                             }),
                             "end": Object({
                                 "line": Number(
                                     5,
                                 ),
                                 "character": Number(
                                     45,
                                 ),
                             }),
                         }),
                     }),
                     "isDynamic": Bool(
                         true,
                     ),
                 }),
             ]),
             "mediaType": String(
                 "TypeScript",
             ),
             "size": Number(
                 329,
             ),
             "specifier": String(
                 "file:///a/test01.ts",
             ),
         }),
     ]),
     "redirects": Object({}),
 })

', src\lib.rs:1491:5

---- tests::test_parse_module_import_assertions stdout ----
thread 'tests::test_parse_module_import_assertions' panicked at 'assertion failed: `(left == right)`

Diff < left / right > :
 Object({
     "dependencies": Array([
         Object({
             "specifier": String(
                 "./a.json",
             ),
             "code": Object({
                 "specifier": String(
                     "file:///a/a.json",
                 ),
                 "span": Object({
                     "start": Object({
                         "line": Number(
                             1,
                         ),
                         "character": Number(
                             18,
                         ),
                     }),
                     "end": Object({
                         "line": Number(
                             1,
                         ),
                         "character": Number(
                             28,
                         ),
                     }),
                 }),
             }),
             "assertionType": String(
                 "json",
             ),
         }),
         Object({
             "specifier": String(
                 "./b.json",
             ),
             "code": Object({
                 "specifier": String(
                     "file:///a/b.json",
                 ),
                 "span": Object({
                     "start": Object({
                         "line": Number(
                             2,
                         ),
                         "character": Number(
                             17,
                         ),
                     }),
                     "end": Object({
                         "line": Number(
                             2,
                         ),
                         "character": Number(
                             27,
                         ),
                     }),
                 }),
             }),
             "isDynamic": Bool(
                 true,
             ),
>            "assertionType": String(
>                "json",
>            ),
         }),
     ]),
     "mediaType": String(
         "TypeScript",
     ),
     "size": Number(
         119,
     ),
     "specifier": String(
         "file:///a/test01.ts",
     ),
 })

', src\lib.rs:1896:5


failures:
    ast::tests::test_analyze_dependencies_import_assertions
    tests::test_create_graph_import_assertions
    tests::test_parse_module_import_assertions

Using deno_ast in a project that contains swc_common -F concurrent breaks the build

I am trying to use deno_ast in my project which also uses swc_common.

I am using it via swc_common = { version = "=0.33.18", features = ["concurrent"] }

With the concurrent feature enabled, the compiler throws:

error[E0277]: `Rc<RefCell<Vec<swc_common::errors::Diagnostic>>>` cannot be sent between threads safely
   --> /home/dalsh/.local/rust/cargo/registry/src/index.crates.io-6f17d22bba15001f/deno_ast-0.34.2/src/transpiling/mod.rs:280:46
    |
280 | impl crate::swc::common::errors::Emitter for DiagnosticCollector {
    |                                              ^^^^^^^^^^^^^^^^^^^ `Rc<RefCell<Vec<swc_common::errors::Diagnostic>>>` cannot be sent between threads safely
    |
    = help: within `DiagnosticCollector`, the trait `Send` is not implemented for `Rc<RefCell<Vec<swc_common::errors::Diagnostic>>>`
note: required because it appears within the type `DiagnosticCollector`
   --> /home/dalsh/.local/rust/cargo/registry/src/index.crates.io-6f17d22bba15001f/deno_ast-0.34.2/src/transpiling/mod.rs:266:8
    |
266 | struct DiagnosticCollector {
    |        ^^^^^^^^^^^^^^^^^^^
note: required by a bound in `swc_common::errors::Emitter`
   --> /home/dalsh/.local/rust/cargo/registry/src/index.crates.io-6f17d22bba15001f/swc_common-0.33.18/src/errors/emitter.rs:39:20
    |
39  | pub trait Emitter: crate::sync::Send {
    |                    ^^^^^^^^^^^^^^^^^ required by this bound in `Emitter`

Is there any way around this?

Upgrade swc which includes parsing files with missing closing braces

This was added in swc here: swc-project/swc#4187

It will give us much better code analysis abilities in the editor.

suggestion: upgrade to `>[email protected]`

[email protected] introduced support for the using keyword with functions. Once these changes make it to the runtime, we can support the using keyword for stubbed and spied functions.

Reland maybe_base

Re-land maybe_base, #87

Combine EmitOptions - inline_source_map and inline_sources

These options are mutually exclusive, so we should combine them into something like:

source_map: SourceMapOption,

Where SourceMapOption has None, Inline, Separate

Fix and unrevert "fix: allow specifying a base for source maps"

#92 -- Need to fix the issue

cc @kitsonk

Add option to apply swc `ts_resolver` on parse

We should move in this code from deno_lint to this crate so it can be used in the deno lsp. I'm not sure exactly what that entails...

https://github.com/denoland/deno_lint/blob/7825cb95115bfdffc62c076c8f458b3424f7190c/src/ast_parser.rs#L114

Once this is done, we should open up a PR in deno to reuse the ast for linting.

missing crate feature "minifier"

lib.rs has a cfg annotation for the "minifier" feature, although the Cargo.toml doesn't specifiy it

We can no longer start files with `BytePos(0)`

After swc-project/swc#4616 we can no longer start files at BytePos(0) and therefore now have to offset all our node byte positions. This is a very unfortunate change because byte positions of nodes no longer align with byte positions in the text.

I think we should deprecate using swc's BytePos and instead start using an opaque type that you can perform operations on, but need to provide a parsed source to get the index in the file.

Preallocate apply_text_changes result

deno_ast/src/text_changes.rs

Line 34 in ff2b406

let mut final_text = String::new();

We can pre-allocate this based on the current string length and the text changes.

update `swc_ecma_transforms_proposal` to `v0.167.15`

This version fixes swc-project/swc#8020, which currently causes incorrect ReferenceErrors in Deno:

// foo.ts

using foo = null

const bar = 1

console.log(baz()) // should log `1`

function baz() {
  return bar
}

$ deno run foo.ts
error: Uncaught ReferenceError: bar is not defined
  return bar
  ^
    at baz (file://.../foo.ts:8:3)
    at file://.../foo.ts:5:13

Update resolver order on next swc upgrade

swc-project/swc#3400

Add upgrading swc instructions

Make sure to highlight how all swc option structs should be analyzed for changes (because swc doesn't allow using a struct expr for some options--also SyntaxError codes should be updated because it uses non_exhaustive)

Map file extension to content type

denoland/deno#17172 adds --ext to deno, eg.

deno run --ext js mod

--ext needs to map file extensions to mime types

The mapping function in deno is incomplete and temporary.

deno_ast's MediaType already maps module specifiers and mime types to MediaType. I think this is a good place to map

file extensions to MediaType
MediaType to mime types

Automatically tag and publish on merge of Cargo.toml with new version

This would automatically tag and publish to crates.io when a Cargo.toml is merged to main that has a version bump.

I think we could trivially do this via a deno script. I haven't found a GH action that does it.

Add more unit tests for transpile function

We should add more unit tests that run this transpile using input and output files.

https://github.com/denoland/deno/blob/43a63530acb16e57cbb190eacedbd097c536a775/cli/ast/mod.rs#L208

Return structured error from `deno_ast::ParsedSource::transpile`

Currently returns an anyhow::Error. Should really be returning a proper structured enum based error (for ease of use thiserror can be used to build this error struct).

Provide web assembly mod

Could a web assembly mod be provided like deno_doc does?
This would be really useful for tools in deno_std.

Ref: denoland/deno_std#2820 (comment)

MediaType from deno://lib.deno.d.ts is JavaScript instead of Dts

Right now the code to convert a specifier to a media type won't handle deno://lib.deno.d.ts because calling .path() on the specifier won't return lib.deno.d.ts.

Develop script to update all repos locally and run tests

It is very painful to find an swc regression in the CLI after releasing all the crates. It would be good to mitigate this occurrence and automate crate updating. This would be a deno script similar to the release scripts.

Here's my current thoughts...

Assumptions

All repos exist as sibling directories of each other.

Phase 1 - Setup

Ensure all repos are on main or switch to it if no local changes. Error if there are local changes.
Update the local main to the latest upstream remote main.
Bump swc versions to the latest version in both dprint-swc-ecma-ast-view and deno_ast.
Update all repos to refer to local versions of each other.

Phase 2 - Cargo Build

Run cargo build in each repo.

At this point, any breaking changes between repos can be resolved.

Phase 3 - Testing

Prompt to run cargo test in each repo (based on dependency tree) until a confirmation is given.
Once a confirmation is given run cargo test in the remainding repos.

At this point, any testing issues can be manually resolved.

Phase 4 - Accepting Changes—Open PRs

A prompt will be given asking to bump each repo's major, minor, or patch.
Reverts all the local path changes.
Updates all the versions in every repo, creates a branch, commits, and pushes a branch for every repo.

At this point, the developer would have to manually open PRs for each branch one after the other based on dependency order, which is still somewhat painful, but not as bad.

`source_map` option appears to be ignored

Seen in deno: after denoland/deno@8b1f160#diff-38d1afee57b1861bf415ef3a8abae123d790c15842a2fc099eed03a535b7b52eL100 landed, startup time was impacted, likely because source maps are being generated.

Update scope analysis after SWC upgrade

New version of SWC will collect top-level bindings, so we need to double check our scope analysis code and possibly remove that code.

swc-project/swc#4436