tsify + babelify + browserify with threejs modules: SyntaxError: 'import' and 'export' may appear only with 'sourceType: module' (9:0) about babelify HOT 17 CLOSED

Wow, thanks a lot. The require setting indeed solved the problem. Here is the final gulp script:

function es6Bundle() {
    log("✳️  ES6 Bundling!");
    return browserify({
        debug: true
    })
    .add("./src/main.ts")
    .require(require.resolve('three/build/three.module.js'), { expose: 'three' })
    .plugin(tsify, { noImplicitAny: true })
    .transform(
        babelify,
        {
            only: [
                "./node_modules/three/build/three.module.js",
                "./node_modules/three/examples/jsm/*"
              ],
            global: true,
            sourceType: "unambiguous",
            presets: ["@babel/preset-env"],
            plugins: ['@babel/plugin-transform-modules-commonjs']
        }
      )
    .bundle()
    .on('error', function(e) {log.error('Error when updating the Bundle: \n' + e);})
    .on('end', function() {log("➡️  Bundle created, uploading to dist")})
    .pipe(source('bundle.js'))
    .pipe(gulp.dest("dist"))
    .on('end', function() {log("✅  Bundle Updated")});
}

and the following dependencies:

"devDependencies": {
    "@babel/cli": "^7.4.4",
    "@babel/core": "^7.4.5",
    "@babel/preset-env": "^7.4.5",
    "@babel/plugin-transform-modules-commonjs": "^7.4.4",
    "babelify": "^10.0.0",
    "browserify": "^16.2.3",
    "fancy-log": "^1.3.3",
    "gltf-pipeline": "^2.1.3",
    "gulp": "^4.0.2",
    "gulp-cli": "^2.2.0",
    "gulp-sourcemaps": "^2.6.5",
    "gulp-typescript": "^5.0.1",
    "tsify": "^4.0.1",
    "typedoc": "^0.14.2",
    "typescript": "^3.5.2",
    "vinyl-buffer": "^1.0.1",
    "vinyl-source-stream": "^2.0.0",
  },
  "dependencies": {
    "three": "git://github.com/JohannesDeml/three.js.git#a2caed8481085e3fe72142f3708d77a7ed5c09d5"
  }

Sadly the build time now went up from 8 to 15 seconds for me, but at least it works now :)

from babelify.

Oh, I think babel in v7 also changed how its babelrc resolution works, so a local babelrc is no longer used for modules inside node_modules(?). You might try:

b.transform(babelify, { global: true, plugins: ['@babel/plugin-transform-modules-commonjs'] })

that should not be affected by the new babelrc resolution. you can then use .babelrc only for the transforms you need in your own code.

You can also add https://www.npmjs.com/package/tfilter to optimize things a bit and only transform the three.js modules:

b.transform(
  tfilter(babelify, { include: /\/three\/examples\/jsm/ }),
  { global: true, plugins: ['@babel/plugin-transform-modules-commonjs'] }
)

from babelify.

Wow, thanks a lot for the quick response!
Sadly now it seems like there is a syntax problem, not quite sure why though:
TypeError: browserify(...).add(...).plugin(...).transform(...).pipe is not a function

function buildBundle() {
    return browserify({
        debug: true
    })
        .add("./src/main.ts")
        .plugin(tsify, { target: 'es6'})
        .transform(babelify, { global: true, presets: ["@babel/preset-env"], plugins: ['@babel/plugin-transform-modules-commonjs'] })
        .pipe(source('bundle.js'))
        .pipe(buffer())
        .pipe(terser())
        .pipe(gulp.dest("dist"));
}

I tried it with and without the preset setting. Thanks also for the info with optimizing. I will do that once the pipeline works again :)

from babelify.

you can explicitly do import 'three/build/three.module.js' to get the ESM version, or do

b.require(require.resolve('three/build/three.module.js'), { expose: 'three' })

to alias the .module.js version to the bare module name.

from babelify.

In your babel config, I think you're looking for the "modules": "commonjs" option instead of "esmodules": true.

Browserify transforms only run on "local" code (not inside node_modules) by default, to encourage more self-contained modules in the ecosystem. To allow transforms to run on files inside node_modules, use global: true: .transform(babelify, { global: true }).

from babelify.

Hello @goto-bus-stop

Thanks a lot for the reply. Good point, the modules should be commonjs from what I can tell. I changed my babelrc and gulp file to the following:

{
    "presets": [
        ["@babel/preset-env",
            {
                "targets": {
                    "modules": "commonjs"
                }
            }
        ],
    ],
    "plugins": ["@babel/plugin-transform-modules-commonjs"], // Tried it with and without the plugin
    "extensions": [ ".js", ".ts", ".tsx" ]
}

function buildBundle() {
    return browserify({
        debug: true
    })
        .add("./src/main.ts")
        .plugin(tsify, { target: 'es6'})
        .transform(babelify, {"compact": false, "global": "true"}) // compact false since three.js is larger than 500kb
        .bundle()
}

Sadly I still get the same error:

[12:16:46] Starting 'buildBundle'...
[12:16:57] 'buildBundle' errored after 11 s
[12:16:57] SyntaxError: 'import' and 'export' may appear only with 'sourceType: module' (8:0) while parsing C:\project\node_modules\three\examples\jsm\loaders\GLTFLoader.js while parsing file: C:\project\node_modules\three\examples\jsm\loaders\GLTFLoader.js     
    at DestroyableTransform.end [as _flush] (C:\project\node_modules\insert-module-globals\index.js:114:21)
    at DestroyableTransform.prefinish (C:\project\node_modules\through2\node_modules\readable-stream\lib\_stream_transform.js:138:10)
    at emitNone (events.js:106:13)
    at DestroyableTransform.emit (events.js:208:7)
    at prefinish (C:\project\node_modules\through2\node_modules\readable-stream\lib\_stream_writable.js:619:14)
    at finishMaybe (C:\project\node_modules\through2\node_modules\readable-stream\lib\_stream_writable.js:627:5)
    at endWritable (C:\project\node_modules\through2\node_modules\readable-stream\lib\_stream_writable.js:638:3)
    at DestroyableTransform.Writable.end (C:\project\node_modules\through2\node_modules\readable-stream\lib\_stream_writable.js:594:41)
    at BabelifyStream.onend (_stream_readable.js:595:10)
    at Object.onceWrapper (events.js:313:30)

Any idea what else I can try?

from babelify.

you're missing a .bundle() call between transform and pipe to start the bundling:

.transform(babelify, ...)
.bundle()
.pipe(source(...))

from babelify.

Oh, just saw that as well. 😊 It works again, thank you so much for your guidance!

from babelify.

Sorry to post once again, but sadly the new setup created a problem: It seems to me like three.js is now included two times in the bundle. When I'm counting for specific lines of code in the new bundle I always get two matches (for the bundles I created before babel it was only one hit). Also the heap size seems to fit with the theory and the problem described here.

That being said, I guess the problem could be solved with defining tfilter correctly to only run on the jsm folder. I tried the following:

b.transform(
  tfilter(babelify, { include: /\/three\/examples\/jsm/ }),
  { global: true, plugins: ['@babel/plugin-transform-modules-commonjs'] }
)

b.transform(
  tfilter(babelify, { include: "*/three/examples/jsm*" }),
  { global: true, plugins: ['@babel/plugin-transform-modules-commonjs'] }
)

.transform(babelify,
            { compact: false,
              only: [
                  "./src",
                  "./node_modules/three/examples/jsm"
              ],
              plugins: ['@babel/plugin-transform-modules-commonjs'], 
              sourceMaps: false })

For all of them I get the same error again:

SyntaxError: 'import' and 'export' may appear only with 'sourceType: module' (9:0) while parsing C:\project\node_modules\three\examples\jsm\loaders\GLTFLoader.js while parsing file: C:\project\node_modules\three\examples\jsm\loaders\GLTFLoader.js

Is there something other than tfilter you can recommend or what is wrong with the configuration you suggested?

folder structure:

-dist/bundle.js
-src/mySourceFiles
-node_modules/three/...
-gulpfile.js
-tsconfig.js
-.babelrc

from babelify.

Try adding sourceType: "unambiguous" to your babel config, which makes Babel infer the source type (module or script) based on the presence of import/export declarations.

from babelify.

Try adding sourceType: "unambiguous" to your babel config, which makes Babel infer the source type (module or script) based on the presence of import/export declarations.

Thanks for the suggestion, I added it to the config and to the gulpfile, sadly it seems to change nothing.

{
    "sourceType": "unambiguous",
    "presets": [
        ["@babel/preset-env",
            {
                "targets": {
                    "esmodules": true
                }
            }
        ],
    ],
    "extensions": [ ".js", ".ts", ".tsx" ]
}

function buildBundle() {
    return browserify({
        debug: false
    })
        .add("./src/main.ts")
        .plugin(tsify, { noImplicitAny: true })
        .transform(
            tfilter(babelify, { include: /(\\three\\examples\\jsm|\/three\/examples\/jsm|three.module.js)/ }),
            { global: true, sourceType: "unambiguous", presets: ["@babel/preset-env"], plugins: ['@babel/plugin-transform-modules-commonjs'] }
          )
        .bundle()
}

I did solve the tfilter riddle: You need to use backslashes for windows paths, and the jsm are also in the build folder which also needs to be included. That however didn't solve my problem with a lot of things being included twice.

Is this a problem on my pipeline or is this maybe caused by threejs?

from babelify.

three.js includes different bundle versions, so it might be that you're accidentally including the commonjs version somewhere and the ESM version somewhere else.

from babelify.

three.js includes different bundle versions, so it might be that you're accidentally including the commonjs version somewhere and the ESM version somewhere else.

That sounds quite likely. On the other hand I don't know how that would happen... From what I can tell from the Documentation I can import the commonjs version with var THREE = require('three'); and the ESM version with import { Scene } from 'three';

I went through my project and I never use require for threejs

The imports I'm doing for threejs look like that:

import { Scene, Group, PerspectiveCamera, CubeTexture, Object3D, AmbientLight } from 'three';
import { GLTFLoader } from "three/examples/jsm/loaders/GLTFLoader";

Sorry, I'm not doing that much webdev, how can I find out when commonjs is imported?

from babelify.

Babelify transforms imports to CommonJS before browserify looks at the file, so browserify sees require() calls. Those will use the CommonJS build of three.js. (browserify does not support ES modules yet).

The only way to get to the ESM build of three.js is by explicitly require()ing its path. The jsm examples may(?) do that, not sure!

from babelify.

I'm not familiar with browserify, but if it supports es modules you can pass the modules: false option to preset-env to avoid transforming import/export to require()/exports.*.

from babelify.

Babelify transforms imports to CommonJS before browserify looks at the file, so browserify sees require() calls. Those will use the CommonJS build of three.js. (browserify does not support ES modules yet).

The only way to get to the ESM build of three.js is by explicitly require()ing its path. The jsm examples may(?) do that, not sure!

Ah, that makes sense. I took a look in one of the examples, they are also using import from the build/three.module.js:

import {
	BufferGeometry,
	DefaultLoadingManager,
	Euler,
	FileLoader,
	Float32BufferAttribute,
	Group,
	LineBasicMaterial,
	LineSegments
} from "../../../build/three.module.js";

In my code I'm just pointing to three (import { Scene, Group, PerspectiveCamera, CubeTexture, Object3D, } from 'three';), so probably that import is pointing to the normal three version. So I guess in the end I have to find out how to point to build/three.module.js instead of build/three.min.js

I'm not familiar with browserify, but if it supports es modules you can pass the modules: false option to preset-env to avoid transforming import/export to require()/exports.*.

I think that is sadly not an option. I got the error from browserify because of using import and export.

from babelify.

Oh, I think babel in v7 also changed how its babelrc resolution works, so a local babelrc is no longer used for modules inside node_modules(?). You might try:

b.transform(babelify, { global: true, plugins: ['@babel/plugin-transform-modules-commonjs'] })

that should not be affected by the new babelrc resolution. you can then use .babelrc only for the transforms you need in your own code.

You can also add https://www.npmjs.com/package/tfilter to optimize things a bit and only transform the three.js modules:

b.transform(
  tfilter(babelify, { include: /\/three\/examples\/jsm/ }),
  { global: true, plugins: ['@babel/plugin-transform-modules-commonjs'] }
)

I messed with this for days - this resolved all my issues - TY so much! :)

These parts in particular helped me:
yarn add -D @babel/plugin-transform-modules-commonjs

Then in my typescript gulp task:
{ global: true, plugins: ['@babel/plugin-transform-modules-commonjs'] }

from babelify.