Missing Exports for fs in Rollup - A Rollup issue about divine-engine HOT 24 CLOSED

from divine-engine.

Continuing this conversation on the Rollup's Gitter for advice as this is time sensitive.

from divine-engine.

After reading through th Rollup config, the issue might be fixed by moving the commonjs() plugin to the top of the list. Will rest this later on.

Additionally, add to the Babel plugin config ‘exclude: ‘node_modules/**’’

from divine-engine.

@srepollock Hey buddy, yeah this turned out being very finicky, but it does work. I've tested it throughly. You need to make sure your plugins are in the correct order and you are using the preferBuiltins option.

from divine-engine.

@SupremeTechnopriest when you say preferBuiltins you mean the rollup-plugin-node-resolve option correct?

from divine-engine.

Yes, make sure it's only set to true for a browser environment.

from divine-engine.

Now it's giving me the error that readFile is not exported by node_modules/rollup-plugin-node-builtins/src/es6/empty.js

I've modified my Rollup.config.js to be:

import typescript from 'rollup-plugin-typescript2';
import babel from 'rollup-plugin-babel';
import globals from 'rollup-plugin-node-globals';
import builtins from 'rollup-plugin-node-builtins';
import resolve from 'rollup-plugin-node-resolve';
import commonjs from 'rollup-plugin-commonjs';
import pkg from "./package.json";

export default {
    input: './src/index.ts',
    plugins: [
        typescript(),
        babel({
            babelrc: false,
            presets: [["@babel/preset-env", {"modules": false}]],
            runtimeHelpers: true
        }),
        builtins(),
        resolve({
            preferBuiltins: true,
            browser: true
        }),
        commonjs(),
        globals({
            process: true,
            global: true,
            dirname: true,
            filename: true
        }),
    ],
    watch: {
        include: 'src/**/*.ts'
    },
    output: [
        {
            file: pkg.main,
            format: 'umd',
            name: 'Divine'
        },
        {
            file: "./lib/divine.cjs.js",
            format: 'cjs',
            name: 'Divine'
        },
        {
            file: pkg.module,
            format: 'es',
            name: 'Divine'
        },
    ],
}

from divine-engine.

Can you try

import typescript from 'rollup-plugin-typescript2';
import babel from 'rollup-plugin-babel';
import globals from 'rollup-plugin-node-globals';
import builtins from 'rollup-plugin-node-builtins';
import resolve from 'rollup-plugin-node-resolve';
import commonjs from 'rollup-plugin-commonjs';
import pkg from "./package.json";

export default {
    input: './src/index.ts',
    plugins: [
        typescript(),
        babel({
            babelrc: false,
            presets: [["@babel/preset-env", {"modules": false}]],
            runtimeHelpers: true
        }),
        commonjs(),
        resolve({
            preferBuiltins: true,
            browser: true
        }),
        builtins(),
        globals({
            process: true,
            global: true,
            dirname: true,
            filename: true
        }),
    ],
    watch: {
        include: 'src/**/*.ts'
    },
    output: [
        {
            file: pkg.main,
            format: 'umd',
            name: 'Divine'
        },
        {
            file: "./lib/divine.cjs.js",
            format: 'cjs',
            name: 'Divine'
        },
        {
            file: pkg.module,
            format: 'es',
            name: 'Divine'
        },
    ],
}

and tell me if there's an error?

from divine-engine.

I will as soon as I’m off work 😊

from divine-engine.

Okay. I have a little time today, I can help you debug if its time sensitive. For reference... I use a generalized rollup config for all my projects and it looks like this:

import builtinList from 'builtins'
import babel from 'rollup-plugin-babel'
import builtins from 'rollup-plugin-node-builtins'
import globals from 'rollup-plugin-node-globals'
import resolve from 'rollup-plugin-node-resolve'
import commonjs from 'rollup-plugin-commonjs'
import minify from 'rollup-plugin-minify-es'
import progress from 'rollup-plugin-progress'
import size from 'rollup-plugin-bundle-size'
import visualizer from 'rollup-plugin-visualizer'

const FILE = process.env.EMS_FILE
const NAME = process.env.EMS_NAME
const FORMAT = process.env.EMS_FORMAT
const BROWSER = process.env.EMS_BROWSER === 'true'
const BUILTINS = process.env.EMS_BUILTINS === 'true'
const BABEL_EXCLUDE = process.env.EMS_BABEL_EXCLUDE
const EXTERNAL = process.env.EMS_EXTERNAL === 'false' ? false : process.env.EMS_EXTERNAL

const config = {
  input: 'src/index.js',
  output: [{
    name: NAME,
    file: `${FILE}.min.js`,
    sourcemap: true,
    format: FORMAT
  }],
  plugins: [
    babel({
      exclude: BABEL_EXCLUDE.concat([ 'node_modules/**' ]),
      plugins: [ 'external-helpers' ]
    }),
    commonjs(),
    resolve({
      main: true,
      module: true,
      preferBuiltins: BUILTINS,
      browser: BROWSER,
      customResolveOptions: {
        moduleDirectory: [ 'node_modules', 'src' ]
      }
    }),
    BUILTINS && builtins(),
    BUILTINS && globals(),
    minify(),
    progress(),
    size(),
    visualizer({ sourcemap: true })
  ]
}

if (EXTERNAL) {
  const external = Object.keys(require(EXTERNAL).dependencies)
  config.external = builtinList().concat(external)
}

export default config

from divine-engine.

AH! @srepollock I know what this is. So you need to add builtins({ fs: true }) if you want to use the browserify-fs module. See here https://github.com/calvinmetcalf/rollup-plugin-node-builtins/blob/master/src/index.js#L56-L58

You will notice crypto and fs are not included by default. I did this because these libraries are HUGE and will bloat your package size significantly. I would consider changing your application code before enabling fs or crypto. For crypto you have the WebCrypto API and for fs you have features like the Filesystem API with spotty support or features like IndexedDB and LocalStorage with universal support.

If you really want to use the browserified version of fs change your config to this:

import typescript from 'rollup-plugin-typescript2';
import babel from 'rollup-plugin-babel';
import globals from 'rollup-plugin-node-globals';
import builtins from 'rollup-plugin-node-builtins';
import resolve from 'rollup-plugin-node-resolve';
import commonjs from 'rollup-plugin-commonjs';
import pkg from "./package.json";

export default {
    input: './src/index.ts',
    plugins: [
        typescript(),
        babel({
            babelrc: false,
            presets: [["@babel/preset-env", {"modules": false}]],
            runtimeHelpers: true
        }),
        builtins({ 
            fs: true 
        }),
        resolve({
            preferBuiltins: true,
            browser: true
        }),
        commonjs(),
        globals({
            process: true,
            global: true,
            dirname: true,
            filename: true
        }),
    ],
    watch: {
        include: 'src/**/*.ts'
    },
    output: [
        {
            file: pkg.main,
            format: 'umd',
            name: 'Divine'
        },
        {
            file: "./lib/divine.cjs.js",
            format: 'cjs',
            name: 'Divine'
        },
        {
            file: pkg.module,
            format: 'es',
            name: 'Divine'
        },
    ],
}

from divine-engine.

OH nice your making a game engine! ❤️

from divine-engine.

Yes I’m building a game engine in typescript! This is my first time working with typescript and learning all about modules, task runners and transports too. This has been a huge learning curve since it’s inception but been a lot of fun. That Rollup.config file looks awesome too! I’ll for sure be using this project’s documentation and setup for future projects so trying to document and compile everything I can now so I won’t have to research time and time again later.

from divine-engine.

Yeah its a labyrinth... and there be dragons... Reminds me of this post from 2016 https://hackernoon.com/how-it-feels-to-learn-javascript-in-2016-d3a717dd577f

from divine-engine.

Holy crap that describes my last 4 months in a nutshell

from divine-engine.

@SupremeTechnopriest I've just tested with the config file you sent me in the previous comment and it's giving a similar error:

> [email protected] bundle /Users/Spencer/Documents/git/divine-engine
> rollup -c


./src/index.ts → ./lib/divine.umd.js, ./lib/divine.cjs.js, ./lib/divine.es.js...
(!) Missing exports
https://github.com/rollup/rollup/wiki/Troubleshooting#name-is-not-exported-by-module
src/core/helperfunctions.ts
readFile is not exported by node_modules/rollup-plugin-node-builtins/src/es6/empty.js
30:     // return fs.readFileSync(filename, "utf8");
31:     var output;
32:     fs.readFile(filename, (err, data) => {
           ^
33:         if (err) {
34:             LogError(ErrorCode.ReadJSONFile, `Error writing ${filename}`);
src/core/helperfunctions.ts
writeFile is not exported by node_modules/rollup-plugin-node-builtins/src/es6/empty.js
59:  */
60: export function writeJSONFile(filename, data) {
61:     fs.writeFile(filename, data, "utf8", (err) => {
           ^
62:         if (err) {
63:             LogError(ErrorCode.WriteJSONFile, `Error writing ${filename}`);

I'm going to keep troubleshooting and see what I can find

from divine-engine.

Moving around some things I've dug deeper.

It seems that the rollup-plugin-node-builtins needed the option fs set to true.

    ...
   builtins({
        fs: true,
    }),
    ...

However, this now gives the same error, and a HUGE circular-dependency warning to console.

(!) `this` has been rewritten to `undefined`
https://github.com/rollup/rollup/wiki/Troubleshooting#this-is-undefined
node_modules/prr/prr.js
 9: (function (name, context, definition) {
10:   if (typeof module != 'undefined' && module.exports) module.exports = definition();else context[name] = definition();
11: })('prr', this, function () {
              ^
12:   var setProperty = typeof Object.defineProperty == 'function' ? function (obj, key, options) {
13:     Object.defineProperty(obj, key, options);
(!) Missing exports
https://github.com/rollup/rollup/wiki/Troubleshooting#name-is-not-exported-by-module
src/core/helperfunctions.ts
readFile is not exported by node_modules/browserify-fs/index.js
30:     // return fs.readFileSync(filename, "utf8");
31:     var output;
32:     fs.readFile(filename, (err, data) => {
           ^
33:         if (err) {
34:             LogError(ErrorCode.ReadJSONFile, `Error writing ${filename}`);
src/core/helperfunctions.ts
writeFile is not exported by node_modules/browserify-fs/index.js
59:  */
60: export function writeJSONFile(filename, data) {
61:     fs.writeFile(filename, data, "utf8", (err) => {
           ^
62:         if (err) {
63:             LogError(ErrorCode.WriteJSONFile, `Error writing ${filename}`);
(!) Circular dependency: node_modules/rollup-plugin-node-builtins/src/es6/readable-stream/duplex.js -> node_modules/rollup-plugin-node-builtins/src/es6/readable-stream/readable.js -> node_modules/rollup-plugin-node-builtins/src/es6/readable-stream/duplex.js
(!) Circular dependency: node_modules/rollup-plugin-node-builtins/src/es6/readable-stream/duplex.js -> node_modules/rollup-plugin-node-builtins/src/es6/readable-stream/writable.js -> node_modules/rollup-plugin-node-builtins/src/es6/readable-stream/duplex.js
(!) Circular dependency: node_modules/fwd-stream/node_modules/readable-stream/lib/_stream_writable.js -> node_modules/fwd-stream/node_modules/readable-stream/lib/_stream_duplex.js -> node_modules/fwd-stream/node_modules/readable-stream/lib/_stream_writable.js
(!) Circular dependency: node_modules/fwd-stream/node_modules/readable-stream/lib/_stream_writable.js -> node_modules/fwd-stream/node_modules/readable-stream/lib/_stream_duplex.js -> commonjs-proxy:/Users/Spencer/Documents/git/divine-engine/node_modules/fwd-stream/node_modules/readable-stream/lib/_stream_writable.js -> node_modules/fwd-stream/node_modules/readable-stream/lib/_stream_writable.js
(!) Circular dependency: node_modules/level-blobs/node_modules/readable-stream/lib/_stream_duplex.js -> node_modules/level-blobs/node_modules/readable-stream/lib/_stream_readable.js -> node_modules/level-blobs/node_modules/readable-stream/lib/_stream_duplex.js
(!) Circular dependency: node_modules/level-blobs/node_modules/readable-stream/lib/_stream_duplex.js -> node_modules/level-blobs/node_modules/readable-stream/lib/_stream_readable.js -> commonjs-proxy:/Users/Spencer/Documents/git/divine-engine/node_modules/level-blobs/node_modules/readable-stream/lib/_stream_duplex.js -> node_modules/level-blobs/node_modules/readable-stream/lib/_stream_duplex.js
(!) Circular dependency: node_modules/level-blobs/node_modules/readable-stream/lib/_stream_writable.js -> node_modules/level-blobs/node_modules/readable-stream/lib/_stream_duplex.js -> node_modules/level-blobs/node_modules/readable-stream/lib/_stream_writable.js
(!) Circular dependency: node_modules/level-blobs/node_modules/readable-stream/lib/_stream_writable.js -> node_modules/level-blobs/node_modules/readable-stream/lib/_stream_duplex.js -> commonjs-proxy:/Users/Spencer/Documents/git/divine-engine/node_modules/level-blobs/node_modules/readable-stream/lib/_stream_writable.js -> node_modules/level-blobs/node_modules/readable-stream/lib/_stream_writable.js
(!) Circular dependency: node_modules/readable-stream/lib/_stream_readable.js -> node_modules/readable-stream/lib/_stream_duplex.js -> node_modules/readable-stream/lib/_stream_readable.js
(!) Circular dependency: node_modules/readable-stream/lib/_stream_duplex.js -> node_modules/readable-stream/lib/_stream_writable.js -> node_modules/readable-stream/lib/_stream_duplex.js
(!) Circular dependency: node_modules/readable-stream/lib/_stream_duplex.js -> node_modules/readable-stream/lib/_stream_writable.js -> commonjs-proxy:/Users/Spencer/Documents/git/divine-engine/node_modules/readable-stream/lib/_stream_duplex.js -> node_modules/readable-stream/lib/_stream_duplex.js
(!) Circular dependency: node_modules/readable-stream/lib/_stream_readable.js -> node_modules/readable-stream/lib/_stream_duplex.js -> commonjs-proxy:/Users/Spencer/Documents/git/divine-engine/node_modules/readable-stream/lib/_stream_readable.js -> node_modules/readable-stream/lib/_stream_readable.js
(!) Circular dependency: src/core/dobject.ts -> src/core/engine.ts -> src/core/SceneManager.ts-> src/core/dobject.ts
created ./lib/divine.umd.js, ./lib/divine.cjs.js, ./lib/divine.es.js in 8.2s

This is referenced in an issue under builtins but it seems that it is just being ignored as an issue. With this it seems I am back to the issue that browserify-fs is not exporting fs.writeFile and fs.readFile from the level-filesystem (but I know they are).

I am also now getting an error of 'this' has been written to 'undefined' from a Rollup issue

from divine-engine.

Noting this here so I don't forget. I need to do some refactoring to fix circular dependencies. This will modify a majority of the project as I will need to look at the MessageSystem implementation and NodeJS. I believe that I cannot have my own MessageSystem and will have to use the EventEmitter baked into the browser or Electron. RE: Notes: I may have overscoped on this and may just need to build off of Electron...

from divine-engine.

Hey yeah you might have missed my last post from about 7 hours ago... It explains why I set a flag for that.

AH! @srepollock I know what this is. So you need to add builtins({ fs: true }) if you want to use the browserify-fs module. See here https://github.com/calvinmetcalf/rollup-plugin-node-builtins/blob/master/src/index.js#L56-L58

You will notice crypto and fs are not included by default. I did this because these libraries are HUGE and will bloat your package size significantly. I would consider changing your application code before enabling fs or crypto. For crypto you have the WebCrypto API and for fs you have features like the Filesystem API with spotty support or features like IndexedDB and LocalStorage with universal support.

If you really want to use the browserified version of fs change your config to this:

import typescript from 'rollup-plugin-typescript2';
import babel from 'rollup-plugin-babel';
import globals from 'rollup-plugin-node-globals';
import builtins from 'rollup-plugin-node-builtins';
import resolve from 'rollup-plugin-node-resolve';
import commonjs from 'rollup-plugin-commonjs';
import pkg from "./package.json";

export default {
input: './src/index.ts',
plugins: [
typescript(),
babel({
babelrc: false,
presets: [["@babel/preset-env", {"modules": false}]],
runtimeHelpers: true
}),
builtins({
fs: true
}),
resolve({
preferBuiltins: true,
browser: true
}),
commonjs(),
globals({
process: true,
global: true,
dirname: true,
filename: true
}),
],
watch: {
include: 'src/**/*.ts'
},
output: [
{
file: pkg.main,
format: 'umd',
name: 'Divine'
},
{
file: "./lib/divine.cjs.js",
format: 'cjs',
name: 'Divine'
},
{
file: pkg.module,
format: 'es',
name: 'Divine'
},
],
}

from divine-engine.

Oh snap I completely missed that comment. I tried it late last night and it didn’t seem to help so I think my last comment still stands that I just need to look at doing some refactoring in my project to really get it going

from divine-engine.

Best of luck!

from divine-engine.

There could be fixed for now with using prototype functions. The issue with circular dependency was on the DObject. This was because it relied on Engine to use its MessageSystem to send messages. Should be fixed if I add prototype functions for the DObject to reference when sending messages. This requires some testing and I will have it in the DObject.spec.ts script.

Example of implementation:

// Engine.ts
DObject.prototype.sendMessage = (event: string, data: Message) => {
    Engine.instance.messageSystem.emit(event, data);
}

See Engine.ts in this commit de1df54

from divine-engine.

This issue is being closed as I'm going in a different direction for asset loading. I will not be using the 'fs' from node.

from divine-engine.

Appreciate the help with this @SupremeTechnopriest!

from divine-engine.