Code Monkey home page Code Monkey logo

rollup-plugin-web-worker-loader's People

Contributors

adamjmcgrath avatar altinselimi avatar chrstntdd avatar code2933 avatar darionco avatar dependabot[bot] avatar hughrawlinson avatar jeetiss avatar mingweisamuel avatar p0lip avatar schlusslicht avatar thecreazy avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

rollup-plugin-web-worker-loader's Issues

rollup watch broken

When using the plugin the watch mode is broken. My webworker is written with ts.

My config looks like this:

resolve(),
  workerLoader({inline: false, sourcemap: false}),
  typescript({
    sourceMap: true,
    exclude: ['node_modules', '**/*.test.ts'],
  }),

The watch mode executes the build, but the files generated are the same as before.
The workaround currently used is this:

npm run clean && rollup -c && chokidar 'src/**/*.*' -c 'rollup -c'

But the builds are only half as fast as with rollup watchmode enabled.

My repository is this:
https://github.com/Waxolunist/paint2

To reproduce start the watchmode and the devserver:

`npm bin`/rollup -cw
npm run serve

When changing a file (e.g. changing the html in paint-app.ts) the changes are not written to the bundle directory.
Commenting out the workerLoader plugin in rollup.config.js changes are picked up immediately, so I assume the workerLoader plugin is something doing to the cache.

rollup-plugin-babel does not transpile helper functions

as reported in #1 by @PixlRainbow

I have just tried pairing this plugin with rollup-plugin-babel and found that for some reason, the web worker runs after babel and doesn't get its helper functions (e.g. createBase64WorkerFactory) transpiled to ES5 even though I ordered the babel plugin after your plugin.

Cannot import anything from node_modules/ external dependencies into web worker

The Web Worker is failing to load external dependencies.

The web worker works fine for standalone functions that are written inside the Worker.ts file but when I moved actual code that had external dependencies I saw this in the console:

0446a38b-53bf-4461-b2cd-4427cdafe0bc:23408 Uncaught ReferenceError: _ is not defined
    at 0446a38b-53bf-4461-b2cd-4427cdafe0bc:23408:4

Saw this when running the rollup -c:

(!) Missing global variable names
Use output.globals to specify browser global variable names corresponding to external modules
lodash (guessing '_')
moment (guessing 'moment$2')
moment-range (guessing 'momentRange')
uuid (guessing 'uuid')
@tandemdiabetes/taco-js-pump-events-decoder (guessing 'tacoJsPumpEventsDecoder')
lodash/flatMap (guessing 'flatMap')
lru-cache (guessing 'LRU')

This is my rollup.config.js

import typescript from 'rollup-plugin-typescript2';
import json from '@rollup/plugin-json';
import { nodeResolve } from '@rollup/plugin-node-resolve';
import commonjs from '@rollup/plugin-commonjs';
import { terser } from 'rollup-plugin-terser';
import analyze from 'rollup-plugin-analyzer';
import { dependencies, peerDependencies } from './package.json';
import webWorkerLoader from 'rollup-plugin-web-worker-loader';



const devMode = process.env.NODE_ENV === 'development';

const onAnalysis = ({
  bundleSize,
  bundleOrigSize,
  bundleReduction,
  moduleCount,
  modules,
}) => {
  if (!devMode) return;
  console.log({
    bundleSize,
    bundleOrigSize,
    bundleReduction,
    moduleCount,
  });
};

const getDependenciesFromPackageJson = () => {
  return [...Object.keys(dependencies), ...Object.keys(peerDependencies)]
    .map((name) => new RegExp(`^${name}`))
    .concat([/node_modules/]);
};

export default {
  external: [...getDependenciesFromPackageJson()],
  input: 'src/index.ts',
  output: [
    {
      file: 'lib/index.esm.min.js',
      format: 'es',
    },
    {
      file: 'lib/index.min.js',
      format: 'cjs',
    },
  ],
  plugins: [
    commonjs(),
    typescript({ tsconfig: './tsconfig.json' }),
    json(),
    nodeResolve(),

    webWorkerLoader(),
    analyze({
      summaryOnly: false,
      hideDeps: true,
      skipFormatted: true,
      onAnalysis: onAnalysis,
    }),
  ],
};

Should you be able to load external dependencies into the web worker? I am able to import relative files just fine but it breaks anytime I try to use an npm package. Is there some configuration I need to change to make this work?

Linting code

Using the yarn lint command I see the following errors in console:

~/opensource/rollup-plugin-web-worker-loader/src/plugin/load.js
  21:10  error  Expected to return a value at the end of function 'findChunk'  consistent-return

~/opensource/rollup-plugin-web-worker-loader/src/plugin/options.js
   1:33  error  'options' is already declared in the upper scope  no-shadow
  18:38  error  'path' is not defined                             no-undef
  26:43  error  'path' is not defined                             no-undef
  27:60  error  'path' is not defined                             no-undef
  43:43  error  'path' is not defined                             no-undef
  44:60  error  'path' is not defined                             no-undef

✖ 7 problems (7 errors, 0 warnings)

I don't know the code well enough to make you a pull request, sorry

when inline is disabled, an error is thrown

when the inline setting is disabled the following error is throw.

idMap is not defined in generateBundle.js

ReferenceError: idMap is not defined
    at generateBundle (C:\Users\-\code\html\see\node_modules\rollup-plugin-web-worker-loader\src\plugin\generateBundle.js:6:30)
    at Object.generateBundle (C:\Users\-\code\html\see\node_modules\rollup-plugin-web-worker-loader\src\index.js:61:13)
    at C:\Users\-\Code\html\see\node_modules\rollup\dist\shared\rollup.js:18100:25

Reverting back to 1.1.3 fixes this issue for now.

create worker by URL.createObjectURL()

Refer to the ts code example to create webworker() by URL.createObjectURL()

`import { MessageManager } from './MessageManager'
import WebWorker from 'web-worker:./Worker.ts'

function main(): void {
const messageManager = new MessageManager('This instance was created on the main thread')

const workerBlob = new Blob([console.log(123)], {
type: 'application/javascript',
})
const worker = new WebWorker(URL.createObjectURL(workerBlob))
worker.postMessage({ type: 'init', args: 'This instance was created in a worker' })

messageManager.sayHello('main thread')
worker.postMessage({ type: 'exec', func: 'sayHello', args: 'web worker' })

messageManager.printMessage()
worker.postMessage({ type: 'exec', func: 'printMessage', args: null })
}

window.addEventListener('DOMContentLoaded', () => {
main()
})
`

but report an error:
Uncaught TypeError: Failed to construct 'Worker': parameter 2 ('options') is not an object. at new WorkerFactory (sdk.js:67) at main (sdk.js:99) at sdk.js:107

import error when run in rollup -c.

i run with the server with cross-env NODE_ENV=Develop rollup -c build/rollup.config.js -w
when i change my component code where i import the webworker and save.
there appear an error Error: 'default' is not exported by src\worker\PlaneLineMergeWorker.js
but when i restart the server.that error disappear.

11

tsdx support

Hi, I'm trying to use this together with tsdx and I unfortunately can't get it to work.

I created a small example repo: https://github.com/lucasmerlin/tsdx-webworker-test

I looked at your typescript example repo and did the following:

  1. create new tsdx project
  2. add tsdx.config.js and configure the webworker loader plugin
  3. add the webworker typescript definition

It's compiling but unfortunately the worker.ts file is not bundled as it should be. Am I missing something or is this plugin incompatible with tsdx?

import error when run rollup -c -w

when i import the worker in more than one place, there will appear an error Error: 'default' is not exported by src\worker\xxxWorker.js。
if there is only one file import the worker, there is nothing wrong.

Webpack warning when using targetPlatform "auto" and webpack target "web"

Hi, I created a Gist of how to reproduce the webpack warning: https://gist.github.com/adamjmcgrath/fbc6b1a0991f98fa9ca13bef92bc825b

When you run rollup -c && webpack rolled-up.js you get:
Module not found: Error: Can't resolve 'worker_threads' in '/dev/tmp-rollup-webpack'

It's essentially when you target web in webpack.

I'm testing our output against a Gatsby build that will do one webpack build for node (to generate the html) and one for web to generate the JavaScript.

error building webworker with current versions of rollup plugins

I have adapted your sample for typescript for an existing rollup with typescript based application. I was not able to make it work because of the following error message:

[!] (plugin rollup-plugin-web-worker-loader) Error: Could not load rollup-plugin-worker-loader::module:C:\Users\test\source\repos\project1-ui\src\element-cache\element-cache.worker.ts (imported by src\element-cache\element-cache.element.ts): @rollup/plugin-typescript: Rollup 'dir' option must be used when Typescript compiler option 'outDir' is specified.

for the following import in my typescript class:

import CacheWebWorker from 'web-worker:./element-cache.worker.ts';

I am furthermore confused that the above import of the webworker isaccepted by vscode in your sample but not in my project.

Here are my configurations.

rollup.config.js:

import commonjs from '@rollup/plugin-commonjs';
import { terser } from 'rollup-plugin-terser';
import typescript from '@rollup/plugin-typescript';
import multiEntry from '@rollup/plugin-multi-entry';
import replace from '@rollup/plugin-replace';
import workerLoader from 'rollup-plugin-web-worker-loader';

const env = process.env.NODE_ENV;

export default {
    input: [
      'src/**/*.element.ts', 
      'src/environment.ts', 
      'src/dependency-injection.ts'],
    output: {
      dir: './public',
      format: 'iife',
      sourcemap: true
    },
    plugins: [
        replace({
          exclude: 'node_modules/**',
          preventAssignment: true,
          '/environment':  env ? `/environment.${env}` : '/environment'
        }),
        multiEntry({
          exports: false,
          entryFileName: 'bundle.js'
        }),
        workerLoader(),
        typescript(),
        resolve(),
        commonjs(),
        env === 'qa' && terser()
    ]
  };

package.json:

{
    "name": "web-components-playground",
    "devDependencies": {
        "@rollup/plugin-commonjs": "^16.0.0",
        "@rollup/plugin-multi-entry": "^4.0.0",
        "@rollup/plugin-node-resolve": "^10.0.0",
        "@rollup/plugin-replace": "^2.4.1",
        "@rollup/plugin-typescript": "^6.1.0",
        "@types/oboe": "^2.1.0",
        "browser-sync": "^2.26.13",
        "cross-env": "^7.0.3",
        "npm-run-all": "^4.1.5",
        "rollup": "^2.33.1",
        "rollup-plugin-terser": "^7.0.2",
        "rollup-plugin-web-worker-loader": "^1.6.1",
        "serve": "^11.3.2",
        "ts-lit-plugin": "^1.2.1",
        "typescript": "^4.0.5"
    },
    "dependencies": {
        "lit-element": "^2.4.0",
        "oboe": "^2.1.5",
        "rollup-plugin-typescript2": "^0.30.0",
        "rxjs": "^6.6.6",
        "save-dev": "0.0.1-security",
        "tslib": "^2.1.0"
    },
    "scripts": {
        "build": "rollup -c",
        "watch": "rollup -c -w",
        "start": "node browser-sync start --config ./browser-sync.js",
        "dev": "npm-run-all --parallel start watch",
        "build-dev": "cross-env NODE_ENV=dev npm run build",
        "build-qa": "cross-env NODE_ENV=qa npm run build"
    }
}

And my tsconfig.lson:

{
    "compilerOptions": {
        "target": "es2017",
        "module": "es2015",
        "lib": ["es2017", "dom", "DOM.Iterable"],
        "declaration": true,
        "declarationMap": true,
        "sourceMap": true,
        "outDir": "./public",
        "rootDir": "./src",
        "strict": true,
        "noUnusedLocals": true,
        "noUnusedParameters": true,
        "noImplicitReturns": true,
        "noFallthroughCasesInSwitch": true,
        "moduleResolution": "node",
        "allowSyntheticDefaultImports": true,
        "experimentalDecorators": true,
        "forceConsistentCasingInFileNames": true,
        "plugins": [
            {
                "name": "ts-lit-plugin",
                "strict": true,
                "rules": {
                    "no-missing-import": "off"
                }
            }
        ]
    },
    "include": ["src/**/*.ts"],
    "exlcude": []
}

Is it maybe related to the version of the typescript plugin? I have loaded the old typescript plugin which gives me different error messages? I really appreciate some help :)

Cannot read property 'search' of undefined

Hi,

I'm developing a JS library that assumes to be used into higher language projects such as Angular, React, Vue ...
In this project I need to run a Web worker that's why I used your plugin to do so.
I've managed to run it on multiple projects but not all

  • ✅ Sample JS project that imports UMD file
  • ✅ included into Angular Library consumed in an angular project
  • ✅ included into Sample React app runned using react-script start
  • 🔴 included into sample React app in prod mode (build+ serve -s build)
  • 🔴 included into sample Vue app

The error that occured is
Cannot read property 'search' of undefined
Located in

function funcToSource(fn, sourcemapArg) {
    var sourcemap = sourcemapArg === undefined ? null : sourcemapArg;
    var source = fn.toString();
    var lines = source.split('\n');
    lines.pop();
    lines.shift();
    var blankPrefixLength = lines[0].search(/\S/);

Here is what I digged from browser console

var WorkerFactory = createInlineWorkerFactory(/* rollup-plugin-web-worker-loader */function () {
(function () {
  '__worker_loader_strict__';

  self.addEventListener("message",(function(e){/*some_code*/}));

}());
}, 'data:application/json;charset=utf-8;base64,eyJ.....');

When I tried to use the content of createInlineWorkerFactory into the steps of funcToScope I achieved to get lines[0], making the error strange to me

Configuration

Dependencies
  • "rollup": "1.32.1",
  • "rollup-plugin-web-worker-loader": "^1.5.0",
Rollup
webWorkerLoader({
      targetPlatform: 'browser',
      sourceMap: !isProduction,
      preserveSource: !isProduction,
      pattern: /^(?!(?:[a-zA-Z]:)|\/).+\.worker\.js$/,
    }),
Import and creation
import TokenWorker from './token.worker.js'
...
private worker?: Worker
...
if ('serviceWorker' in navigator) {
this.worker = new TokenWorker()
}

I tried a second test with setting inline: false in webWorkerLoader config, the error was gone but I got another one that told me the worker JS file included a < character because the content was my built html file.

I would like to know how to properly set your plugin to be compatible with a react and Vue projects.

Do not hesitate to ask more .

Thanks

list rollup as a peerDependency instead of devDependency

Hello there!

I just spun up a project that uses yarn 2/berry and attempted to use your loader and encountered some troubles. Looks like your project has rollup in the package.json as a devDependency when it should be a peerDependency - that we expect users of this plugin to have rollup installed themselves. This is the error I was encountering. It looks like most of the official plugins do it this way

This should be a straight forward change and I think it would be beneficial. Glad to make these changes if they're welcome.

error when build inline worker

see https://github.com/LoveofRedMoon/bug-vite-worker-20201125.git
vite app use worker with the following config

plugins: [
..........,
require('rollup-plugin-web-worker-loader')({
        targetPlatform: 'browser',
        pattern: /(.+)\?worker$/,
        extensions: resolver_1.supportedExts,
        sourcemap: false // it's inlined so it bloats the bundle
    })
]

when build some workers with dynamicImports, the rollup will erroring:
UMD and IIFE output formats are not supported for code-splitting builds.

I'm not sure if there is a better way to resolve this problem,
one way to resolve is modify load.js
to

bundle.generate({format: 'iife', name: 'worker_code', sourcemap: true, inlineDynamicImports: true}).then(result => {
                    resolve(handleBundleGenerated(state, config, addWatchFile, id, workerID, result));
                }).catch(reject);

use inlineDynamicImports

Incompatible with @rollup/plugin-alias

The webworker plugin works fine without any configuration.

However, upon adding the separate alias plugin to my rollup.config.js, I see this error when executing the command `rollup -c -w"

[!] Error: 'Engine' is not exported by src/script.js, imported by src/ui/store.js
https://rollupjs.org/guide/en/#error-name-is-not-exported-by-module
src/ui/store.js (1:9)
1: import Engine from 'web-worker:../script.js';

My aliases look like:

alias({
			entries: [
				{ find: 'src', replacement: './src/' },
				{ find: 'scripts', replacement: './src/scripts' },
				{ find: 'engine', replacement: './src/engine' },
				{ find: 'ui', replacement: './src/ui' }
			]
		}),

Any idea what could be causing it?

Breaking rollup-plugin-css-only plugin

  • Rollup Plugin Name: rollup-plugin-web-worker-loader
  • Rollup Plugin Version: 1.6.1
  • Rollup Version: 2.62.0
  • Operating System (or Browser): macOS 11.6 BigSur
  • Node Version: v16.13.0
  • Link to reproduction : project.zip

Expected Behavior

rollup-plugin-web-worker-loader shouldn't break the rollup-plugin-css-only plugin.

Actual Behavior

With my config, when I enable the rollup-plugin-web-worker-loader and I have at least one web worker, my rollup-plugin-css-only does not output anything.

How to reproduce

You can run npm install && npm run build, there will be no bundle.css generated in public/build.
If you disable the rollup-plugin-web-worker-loader in rollup.config.js and build again, there will be a bundle.css.

Rollup format ESM not working any more

Since V1.4.0 my worker is not functioning any more when using ESM as format in Rollup.

Seems the change to format IIFE in load.js is causing the problem:
bundle.generate({format: 'iife', name: 'worker_code', sourcemap: true})

Could you maybe make this dynamic based on the format used by Rollup?

Terser / minified worker code?

Sorry if I'm missing something obvious, this is my first attempt at using rollup.

I'm using your plugin successfully to inline my worker. The worker includes a significant amount of code and is about 1.5mb (it includes an entire Emscripten project). This has been incredibly useful because shipping a library with several files and importing workers and wasm puts a lot of burden on the end user to figure a lot of things out even to do "hello world".

When I rollup the project I see the resulting output is 2.2mb, and minified (terser) output is the same size. I realized that what's happening is the worker has about ~500kb of extra un-minified source, which then gets base64 encoded and inlined before terser is invoked. So the final minification step on the does not do anything useful since the majority of the JS has been encoded.

Is there a way to include minification in the worker before it gets base64 encoded? Inline is definitely the end result I want, but ideally in as compact a form as possible.

Does not work with esm.

It is showing that UMD and IIFE are not supported for code splitting, I have inline set as false.

Support for return just the base64 encode

Hey @darionco, how are you?

I'm working in a project where I have to load an AudioWorkletProcessor and is basically the same thing as a Worker but it has a different way to instantiate it.

I like your plugin because is doing what I need, creating a inline version of a worker file in base64 but I want to know if it could be possible to add an option to just return the base64 string instead of the worker so I can do the job of createBase64WorkerFactory by myself

Allow using rollup 3.0.0

peerDeps are:

"peerDependencies": {
"rollup": "^1.9.2 || ^2.0.0"
}

so load is not up to date and this can clash and dedupe an already installed rollup 3.

Would you consider adding ^3.0.0 to it?

The only place I see it is used is here and the rollup API seems to be the same between v2 and v3:

rollup.rollup(inputOptions).then(bundle => {
state.exclude.delete(id);
state.exclude.delete(target);
const bundleOptions = {
format: 'iife',
name: 'worker_code',
sourcemap: true,
inlineDynamicImports: true,
};
bundle.generate(bundleOptions).then(result => {
resolve(handleBundleGenerated(state, config, addWatchFile, id, workerID, result));
}).catch(reject);
}).catch(reason => {
state.exclude.delete(id);
state.exclude.delete(target);
reject(reason);
});

Why not show any example of the correct workerLoader configuration syntax ??

Hi
I'm trying to adapt your rollup-typescript-webworkers project from a web-worker type to a shared-worker WorkerFactory type
My attempt to update the workers.d.ts is apparently incorrect =>

declare module 'shared-worker:*' {
    const WorkerFactory: new () => SharedWorker;
    export default WorkerFactory;
}

OR my attempt to define the 'shared-worker' plugin option in rollup.config.js is wrong =>

const workerLoader = require('rollup-plugin-web-worker-loader');
...
        plugins: [
            ...
            workerLoader({
              'shared-worker?': /\.\/Worker\.ts$/
            }),
            ...
       ]

I also import the shared-worker module into index.ts =>

import SharedWorker from 'shared-worker:./Worker.ts';

When I run yarn build:all I get this error =>

!) Unresolved dependencies
https://rollupjs.org/guide/en/#warning-treating-module-as-external-dependency
shared-worker:./Worker.ts (imported by src/index.ts)

I assume this means the workerLoader plugin is not loading the shared-worker dependency.
Correct?
Your readme content does not actually show an example of a workerLoader configuration.
The related section shows this =>

import webWorkerLoader from 'rollup-plugin-web-worker-loader';

export default {
    entry: 'src/index.js',
    plugins: [
        webWorkerLoader(/* configuration */),
    ],
    format: 'esm',
};

Why not add an example that shows the expected syntax?
Any help with my issue is appreciated

Worklet support

Hi,

I'm writing to ask if you would be open to a PR adding support for other types of worker-like constructs such as Worklets. There are several other constructs that require an almost identical setup, but use a different constructor, such as:

  • navigator.serviceWorker.register(moduleId)
  • CSS.paintWorklet.addModule(moduleId)
  • new AudioContext().audioWorklet.addModule(moduleId)

This could be accomplished in a couple of ways.

If the answer is a clear "no, this project is for Workers only", feel free to skip everything below, I understand completely.

Adding additional factory types

We could implement additional factories like the existing createInlineWorkerFactory method for each of the above Worker/Worklet types. Then, modify the options type to accept a parameter choosing Worker/Worklet type. At the same time, that would require this project to be aware of all the relevant W3C specs, and keep up to date with them.

Custom factory

If the user could specify a custom factory, we don't need to update this project whenever a new Worker-ish API is released. They could also specify parameters to Worker-ish constructor if they needed to.

Provide a way to import the encoded bundle as base64 directly

(I'm new to rollup so there might be an obvious way to do this already that I'm unaware of. If that's the case, sorry in advance!)

If web-worker-loader were configured to return the base64 encoded bundle, or blob, instead of doing the following:

import MyWorker from "web-worker:./my-worker.js";
// Right now this code would call the Worker constructor in one of the Factories
// which is not what we want in the case of the other Worker-like constructs
const w = new MyWorker();

you could have the user actually instantiate the Worker-ish object

import MyAudioWorklet from "audio-worklet:./my-audio-worklet.js";
myAudioContext.audioWorklet.addModule(MyAudioWorklet);

Whether or not you're interested in getting support for these in this project, I would value your feedback on the API options - but no pressure!! Thanks for taking the time to read this.

Incorrect utf8/unicode handling when converting base64-encoded string to source code

Why this happens

We are calling atob to convert base64-encoded string to source code, however atob does not handle unicode correctly (see "The_Unicode_Problem" on MDN )

export function createBase64WorkerFactory(base64, sourcemap = null) {
const source = kIsNodeJS ? Buffer.from(base64, 'base64').toString('ascii') : atob(base64);
const start = source.indexOf('\n', 10) + 1;
const body = source.substring(start) + (sourcemap ? `\/\/# sourceMappingURL=${sourcemap}` : '');

Expected behavior and actual behavior

I use rollup with this rollup plugin to bundle my code, and I have some Chinese comments in my worker source code. Chinese characters will be garbled in the bundle 😥 .

image

How to fix it

The simplest solution is that we call encodeURIComponent(...) before we encode the source into base64 format in buildWorkerCode, so we can ensure there are only ascii characters in the base64-encode input. And in createBase64WorkerFactory we call decodeURIComponent after base64 decode.

Webpack error: require function is used in a way in which dependencies cannot be statically extracted

Hi, we're hitting an issue when webpack builds the output in CI mode auth0/auth0-spa-js#447

When we tested it originally we noticed a warning: require function is used in a way in which dependencies cannot be statically extracted but thought we could ignore it.

But with the CI environment enabled, this turns into an error.

It seems webpack doesn't like the var kRequire = require; kRequire('something'); happening in https://github.com/darionco/rollup-plugin-web-worker-loader/blob/master/src/helper/node/WorkerClass.js#L4

WorkerFactory assignment does not seem to be processed by TypeScript

We've noticed when using this library that this line appears directly in our output file, even though we're transpiling to ES5 using TypeScript:

const WorkerFactory = createInlineWorkerFactory(/* rollup-plugin-web-worker-loader */function () {

Which means that we're unable to use this in IE11, which supports the WebWorker API. Digging through the code, I can see why this is happening as the worker module is signalled as being a virtual module. It is then filtered out before TypeScript has a chance to process it.

I can understand why it's like that, though. Perhaps the answer is to not use ES6 code here for maximum compatibility. Otherwise, we have no way of processing it.

Any other suggestions? Here is our rollup plugin config:

const getPlugins = shouldMinify => {
  return [
    resolve({
      browser: true
    }),
    commonjs(),
    webWorkerLoader({
      sourceMap: !isProduction,
      preserveSource: !isProduction
      // pattern: /^[^\/].+\.worker\.ts$/
    }),
    typescript({
      clean: true,
      useTsconfigDeclarationDir: true,
      include: [
        'src/**/*.ts',
        'src/**/*.js',
        'node_modules/rollup-plugin-web-worker-loader/**/*'
      ],
      tsconfigOverride: {
        noEmit: false,
        sourceMap: true,
        compilerOptions: {
          lib: ['dom', 'es6']
        }
      }
    }),
    shouldMinify && terser(),
    sourcemaps()
  ];
};

tsconfig.json here:

{
  "compilerOptions": {
    "lib": ["dom", "webworker"],
    "target": "es5",
    "allowSyntheticDefaultImports": true,
    "esModuleInterop": true,
    "sourceMap": true,
    "declaration": true,
    "declarationDir": "./dist/typings",
    "moduleResolution": "node",
    "allowJs": true
  },
  "exclude": ["./__tests__", "./dist/typings"]
}

These might be a little bit convoluted - in an attempt to get this to work (originally we couldn't even get our worker module to transpile properly, but that's fixed now).

Live Reload code in web worker

When I am running svelte in dev mode, some livereload code is being added to the Base64 web worker code, even though the setting says to skip livereload plugin:

rollup.config.js

import svelte from 'rollup-plugin-svelte';
import resolve from '@rollup/plugin-node-resolve';
import commonjs from '@rollup/plugin-commonjs';
import livereload from 'rollup-plugin-livereload';
import { terser } from 'rollup-plugin-terser';
import webWorkerLoader from 'rollup-plugin-web-worker-loader';

const production = !process.env.ROLLUP_WATCH;

export default {
	input: 'src/main.js',
	output: {
		sourcemap: true,
		format: 'iife',
		name: 'app',
		file: 'public/build/bundle.js'
	},
	plugins: [
		svelte({
			// enable run-time checks when not in production
			dev: !production,
			// we'll extract any component CSS out into
			// a separate file - better for performance
			css: css => {
				css.write('public/build/bundle.css');
			}
		}),

		// If you have external dependencies installed from
		// npm, you'll most likely need these plugins. In
		// some cases you'll need additional configuration -
		// consult the documentation for details:
		// https://github.com/rollup/plugins/tree/master/packages/commonjs
		resolve({
			browser: true,
			dedupe: ['svelte']
		}),
		commonjs(),

		// In dev mode, call `npm run start` once
		// the bundle has been generated
		!production && serve(),

		// Watch the `public` directory and refresh the
		// browser on changes when not in production
		!production && livereload('public'),

		// If we're building for production (npm run build
		// instead of npm run dev), minify
    production && terser(),
    
    webWorkerLoader({
      targetPlatform: 'browser',
      extensions: ['.worker.js'],
      skipPlugins: [ 'liveServer', 'serve', 'livereload', 'terser' ]
    })
	],
	watch: {
		clearScreen: false
	}
};

function serve() {
	let started = false;

	return {
		writeBundle() {
			if (!started) {
				started = true;

				require('child_process').spawn('npm', ['run', 'start', '--', '--dev'], {
					stdio: ['ignore', 'inherit', 'inherit'],
					shell: true
				});
			}
		}
	};
}

testWorker.worker.js

onmessage = function(e) {
  console.log('message in worker');
}

main.js

import App from './App.svelte';
import TestWorker from 'web-worker:./testWorker.worker.js';

const testWorker = new TestWorker();
testWorker.postMessage('tester');

const app = new App({
	target: document.body
});

export default app;

Base64 worker

(function(l, r) { if (l.getElementById('livereloadscript')) return; r = l.createElement('script'); r.async = 1; r.src = '//' + (window.location.host || 'localhost').split(':')[0] + ':35729/livereload.js?snipver=1'; r.id = 'livereloadscript'; l.getElementsByTagName('head')[0].appendChild(r) })(window.document);
onmessage = function(e) {
  console.log('message in worker');
};

browser console error: Uncaught ReferenceError: window is not defined

Avoiding code duplication pointer.

About code duplication, the readme says "there are better solutions for this purpose" - would it be worth linking to the better solutions?

The "path" argument must be of type string. Received type undefined

i import it like this
import ProfileWorker from 'web-worker:./profile_worker.js'

and returns

(plugin web-worker-loader) TypeError: Could not load /Volumes/TIMELESS/tmp/app/src/views/Conversation/services/profile_worker.js (imported by src/views/Conversation/services/profile.js): The "path" argument must be of type string. Received type undefined

Using WorkerLoaderHelper with SSR frameworks

Hi, we're seeing the error ReferenceError: Blob is not defined when we try and use the output of WorkerLoaderHelper.js code in SSR environments like Gatsby and Next.js (eg vercel/next.js#11967 (comment))

I've created a minimal test case here https://gist.github.com/adamjmcgrath/f697de44cf7b5056ac108edcc9815bff

Essentially, module.require is undefined in https://github.com/darionco/rollup-plugin-web-worker-loader/blob/master/src/WorkerLoaderHelper.js#L2

So kRequire is null https://github.com/darionco/rollup-plugin-web-worker-loader/blob/master/src/WorkerLoaderHelper.js#L64

And this branch of code https://github.com/darionco/rollup-plugin-web-worker-loader/blob/master/src/WorkerLoaderHelper.js#L73-L77 is run in node

Which throws: ReferenceError: Blob is not defined

I've noticed that reverting this 14809d3 fixes the issue. But I'm not sure what the module.require logic was being used for, or why it's not working now

Pattern option doesn't affect on chunk's name

From README:

pattern?: RegEx,  // A RegEx instance describing the pattern that matches the files to import as
                            // web workers. If capturing groups are present, the plugin uses the contents of the
                            // last capturing group as the path to the worker script. Default: /web-worker:(.+)/

Sounds like if you have worker with name my-worker.js and import it with web-worker:./my-worker.js you will get chunk file with name my-worker.js

But I checked the source code:

workerID: `web-worker-${state.idMap.size}.js`,

And it doesn't work like this, it always generates file with web-worker-0.js name

Is this a bug? I would like to set worker's name

Typescript ?

Can this plugin be used with Typescript ? I really could use some help - Migrating from Webpack to Rollup and currently am considering going back to webpack bc the experience with workers on Rollup is so bad. If you could post an example project of how you are using this plugin and the associated folder structure, that would be incredibly helpful.

the synax of loader can not been resolved by jest

 Cannot find module 'web-worker:./worker.ts' from 'src/utils/task.ts'

    Require stack:
      src/utils/task.ts
      src/utils/index.ts
      src/index.ts
      test/get-url.test.ts

       5 | // eslint-disable-next-line @typescript-eslint/ban-ts-comment    
       6 | // @ts-ignore
    >  7 | import DataWorker from 'web-worker:./worker.ts'
         | ^
       8 |
       9 | import { Options } from '../../interfaces'
      10 |

      at Resolver.resolveModule (node_modules/jest-resolve/build/resolver.js:324:11)
      at Object.<anonymous> (src/utils/task.ts:7:1)

ParseError: The "path" argument must be of type string. Received undefined

I keep getting this error when I try to import the web-worker.

[0] ParseError: rollup-plugin-worker-loader::module:\src\react-app\store\worker.js
[0] TypeError [ERR_INVALID_ARG_TYPE]: The "path" argument must be of type string. Received undefined
[0] at validateString (internal/validators.js:117:11)
[0] at Object.dirname (path.js:583:5)'

Is there something wrong with my config file or this module is not supported by nollup.

I'm trying to import it Like this:

Path: src\react-app\store\index.js

import StoreWorker from 'web-worker:./worker';

The worker is in the same directory:

Path: src\react-app\store\worker.js

Rollup config:

import path from 'path';
import alias from '@rollup/plugin-alias'
import babel from '@rollup/plugin-babel';;
import hotcss from 'rollup-plugin-hot-css';
import refresh from 'rollup-plugin-react-refresh';
import resolve from '@rollup/plugin-node-resolve';
import commonjs from '@rollup/plugin-commonjs';
import webWorkerLoader from 'rollup-plugin-web-worker-loader';
import { terser } from 'rollup-plugin-terser';

const root = path.resolve(__dirname);

let env = function () {
    return {
        banner: `
            self.process = {
                env: {
                    NODE_ENV: ${JSON.stringify(process.env.NODE_ENV)}
                }
            };
        `
    }
};

let config = {
    input: './src/react-app/main.js',
    output: {
        dir: 'dist',
        format: 'esm',
        entryFileNames: '[name].[hash].js',
        assetFileNames: '[name].[hash][extname]',
    },
    plugins: [
        env(),
        alias({
            entries: [
                { find: '@app', replacement: path.resolve(root, 'src/react-app') }
            ]
        }),
        babel({
            sourceMaps: false,
            babelHelpers: 'bundled',
            exclude: 'node_modules/**'
        }),
        hotcss({
            hot: process.env.NODE_ENV === 'development',
            filename: 'styles.css'
        }),
        resolve(),
        commonjs(),
        webWorkerLoader(),
        process.env.NODE_ENV === 'development' && refresh()
    ]
}

export default config;

`RangeError: Maximum call stack size exceeded` with `enableUnicode: true`

I'm bundling some code from mathjs which uses unicode variable names 🥲 . In order to use this code for an inline browser target, I have to set enableUnicode: true so it can base64 encode/decode it properly. I understand this makes the size much larger.

It builds ok, but when I run it in a browser I see RangeError: Maximum call stack size exceeded in the base64 decode logic. The stacktrace indicates it occurs at the invocation of String.fromCharCode.apply() here:

return String.fromCharCode.apply(null, new Uint16Array(binaryView.buffer));

It sounds like the modern TextDecoder API is able to perform this decoding without the arguments limit. I will make a PR fix.

`inline: false` and `external: []` causes invalid syntax with multiple outputs

Given this rollup configuration:

import webWorker from "rollup-plugin-web-worker-loader";

/** @type {import('rollup').RollupOptions} */
export default {
    input: "src/index.js",
    output: [
        {
            file: "dist/index.js",
            format: "cjs",
        },
        {
            file: "dist/index.modern.js",
            format: "es",
        },
    ],
    plugins: [
        webWorker({
            targetPlatform: "browser",
            inline: false,
            external: [],
        }),
    ],
};

This plugin tries to write both rollup outputs to the same destination index.modern.js. This results in a file that is a mixture of CJS and ESM, an invalid syntax that causes errors.

It seems this is because the plugin holds a single mutable filename reference in state.configuredFileName

const state = {
idMap: new Map(),
exclude: new Set(),
outFiles: new Map(),
options: null,
basePath: null,
forceInlineCounter: 0,
configuredFileName: null,
};

This mutable value is used as the output destination, which is unstable.

function outputOptions(state, config, options) {
if (!config.inline && options.file && !options.dir) {
state.configuredFileName = path.basename(options.file);
return Object.assign({}, options, {
file: null,
dir: path.dirname(options.file),
});
}
return null;
}

I will try to make a PR fix.

JS Example.

I am having a lot of problems implementing the plugIn, I am sure all of them due to my missunderstandings. Better than asking too many obvious questions to you, would it be possible to get a full working example in vanilla JS?
Thanks in advance.

the worker is located on a path inaccessible to the bundle

I am a bit confused about building the project from ts example https://github.com/darionco/rollup-typescript-webworkers. how do the bundle will link to the worker if it is located on an inaccessible path and only dist/** send into the published package

Screen Shot 2021-05-27 at 21 07 43

could you clarify that and give an example of rollup config

I've expected similar structure after bundle

dist
├── index.js
├── worker
      ├──worker.js

with current plugins section

import { nodeResolve } from "@rollup/plugin-node-resolve";
import ts from "@wessberg/rollup-plugin-ts";
import webWorkerLoader from "rollup-plugin-web-worker-loader";
...
[
  nodeResolve(),
  webWorkerLoader(),
  ts({
    tsconfig: "tsconfig.build.json",
    hook: {
      declarationStats: (declarationStats) =>
        console.warn(JSON.stringify(declarationStats)),
    },
  }),
  ...
]

I get warning

.../rollup-typescript-webworkers/src/index.ts → dist/next/index.js...
(!) Unresolved dependencies
https://rollupjs.org/guide/en/#warning-treating-module-as-external-dependency
web-worker:./Worker.ts (imported by src/index.ts)

Error: Unexpected token

A contributor to our project put together a PR (chartjs/Chart.js#7029) proposing to use this plugin, but it's failing and we're stuck on figuring out why.

I was wondering if you might have any ideas what's going wrong based on the error message or if you spot anything obviously wrong in our code

Error: Could not load rollup-plugin-worker-loader::module:../BasicChartWebWorker (imported by test/specs/platform.basic.tests.js): Unexpected token (Note that you need @rollup/plugin-json to import JSON files)
at error (node_modules/rollup/dist/shared/node-entry.js:5400:30)
at Module.error (node_modules/rollup/dist/shared/node-entry.js:9820:16)
at tryParse (node_modules/rollup/dist/shared/node-entry.js:9713:23)
at Module.setSource (node_modules/rollup/dist/shared/node-entry.js:10076:33)
at node_modules/rollup/dist/shared/node-entry.js:12362:20
at async Promise.all (index 11)
at async Promise.all (index 0)

Here's the file with the import: https://github.com/benmccann/Chart.js/blob/PR7029/test/specs/platform.basic.tests.js
Here's the rollup config: https://github.com/benmccann/Chart.js/blob/PR7029/karma.conf.js

Thank you so much for the plugin and any help you can provide!

Feature Request: Change where to see worker scripts (loadPath) in runtime

I'm authoring a library using this plugin, and trying to load worker scripts in both development environment and deployed environment (which can have a different codebase, the library is an external dependency and unchangeable though).
However it forces me to put the worker scripts onto the very same place of each environment, by using the same loadPath option.
Giving an example, I want to put the workers at the root level of my production environment https://example.com/*.worker.js when I'm at https://example.com/page/A (which is achievable by using loadPath: '/'), while in another project I also want to load https://user.github.io/project/dist/*.worker.js or localhost:8080/dist/*.worker.js (which is achievable by using loadPath: '../dist/').
I also don't want to make it inline: true, since the payload is already massive enough and one of my workers needs a support of multibyte chars!

My current solution is to let the loader able to generate the worker factory code with an additional option loadPath, and make it be able to change where to put workers code by purposes.
This is what I did (a single commit): https://github.com/FMS-Cat/rollup-plugin-web-worker-loader/commit/6c7118a49590dda4d3f25d5109c6c63b05e1bd9f

However, having an additional option to WorkerOptions might not be cool.
I would be happy if you can come up with a better interface and solution.

Thank you for the project, it already helps me so much!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.