gja / cloudflare-worker-local Goto Github PK

I have a CF Worker that also uses the "site" functionality to serve up static assets like images. This works fine in the real wrangler dev environment, but I would like something that works entirely locally so I don't have to jump through hoops to refresh static files like CSS, JS, etc.

When I run it with cloudflare-worker-local, the images seem to transfer across the wire with 200 OK messages, but they are mangled somehow. Text-based files come across just fine, but anything binary doesn't. It must be some sort of content encoding issue, but I haven't yet been able to figure out what's causing it.

Any ideas? I'll keep digging in the meantime...

Hi there,

I was wondering if it is possible to run streaming? readablestream

When attempting this I saw error TypeError: readable.getReader is not a function

if this is something you know about, and perhaps have a solution for which you may not have published/pushed yet.

Many thanks.

Would you accept a PR that adds logic to watch the worker script and restart the cluster when the file changes?

Small change to start.js:

  if (cluster.isMaster) {
+   require('chokidar')
+     .watch(process.argv[2], { ignoreInitial: true })
+     .on('change', cluster.disconnect);
    for (var i = 0; i < 4; i++) {
      cluster.fork();
    }
    cluster.on("exit", function(worker, code, signal) {
      console.log("worker " + worker.process.pid + " died");
      cluster.fork();
    });
  } else {

Could put this behind a --watch flag- although it seems like a good default behavior.

The function I'm trying to use:

function sleep(ms) {
  return new Promise(resolve => {
    setTimeout(resolve, ms)
  })
}

I can see similar method working in tutorials like this: https://scotthelme.co.uk/unleashing-the-power-of-cloudflare-workers/

While testing with this package I get:

[0] ReferenceError: setTimeout is not defined
[0]     at resolve (evalmachine.<anonymous>:741:5)
[0]     at new Promise (<anonymous>)

Hello,

Could you please release 1.9.1 with last two commits? Thank you

Cloudflare worker allows controlling cache via a cf object. Proposing a PR to expose this object into request headers. Unit tests can then check if cache rules are set correctly by checking against these headers.

Reference: https://developers.cloudflare.com/workers/recipes/vcl-conversion/controlling-the-cache/

Hey,
The env var MINIO_PORT doesn't work, because env vars are string, and the package accepts numbers.

Can you please do it and release a new version?

Working through https://liftcodeplay.com/2018/10/01/validating-auth0-jwts-on-the-edge-with-a-cloudflare-worker/

Here's what I had to add to the context:

    evaluateWorkerContents(workerContents) {
+     const atob = require('atob');
+     const btoa = require('btoa');
+     const WebCrypto = require('node-webcrypto-ossl');
+     const crypto = new WebCrypto();  // require('isomorphic-webcrypto');  <-- isomporphic-webcrypto is not a good lib
+     const { TextDecoder, TextEncoder } = require('util');
-     const context = { Request, Response, Headers, URL };
+     const context = { Request, Response, Headers, URL, atob, btoa, crypto, TextDecoder, TextEncoder, console };
      const script = new Script(workerContents);

With these changes I was able to successfully validate JWTs in the context of cloudflare-worker-local.

First of all- thank you for this project! Awesome work 🚀

I think the fetch method exposed in the worker context should be "standard fetch" for compatibility. Couple examples:

• using the fetch(url, init) signature in the worker currently doesn't work because fetchUpstream doesn't implement the overload.
• fetchUpstream overwrites the host on all requests

https://github.com/gja/cloudflare-worker-local/blob/master/app/worker.js#L16

  evaluateWorkerContents(workerContents) {
-   const context = { Request, Response, Headers, URL };
+   const context = { Request, Response, Headers, URL, fetch };
    const script = new Script(workerContents);
    script.runInContext(
      createContext(
        Object.assign(context, {
-         fetch: this.fetchUpstream.bind(this),
          addEventListener: this.addEventListener.bind(this),
          triggerEvent: this.triggerEvent.bind(this)
        })
      )
    );
  }

Would you accept a PR with these changes (and tests)?

Trying to access the on method in this line may throw a TypeError: Cannot read properties of null (reading 'on') since the response body can be null or undefined.

Cloudflare worker now supports FetchEvent.passThroughOnException() to handle worker exceptions.

From official docs:

Often, if your Worker isn’t doing security-critical activities, it makes sense for it to act as if it’s not installed at all if it errors. This allows you to install logging, tracking, or other minor Workers without fearing they may break your underlying site. This behavior can be toggled on by calling event.passThroughOnException()

There should be at least a copy of this interface to cloudflare-worker-local so that valid worker scripts won't fail on local dev.

I tried to log something in worker with console.log but nothing shows in output of cloudflare-worker-local. How can I show logs when testing service locally?

For context our worker has some logic dependent on user country in order to test this logic I want to set this header to value different than default 'DEV' value

I have following test.js:

addEventListener('fetch', event => {
  console.log(Math.random())
  event.respondWith(new Response('Hello world'))
})

Now if I run dev server for it with cloudflare-worker-local test.js localhost:3000 4000 and go to localhost:4000 then following is shown in console:

Example app listening on port 4000!
0.2003707286871348
0.38354288738431386

It looks like script is called two times for each request. What is happening? How to prevent it?

Error:

SyntaxError: Unexpected token 'export'
    at new Script (vm.js:102:7)
    at Worker.evaluateWorkerContents (/home/wighawag/dev/wighawag/etherplay/conquest-agent-service/node_modules/cloudflare-worker-local/app/worker.js:111:20)

Rather than have a horrible mix of environment variables, command line arguments, and worker.toml I propose that it all be consolidated into the new worker.toml file.

The worker.toml file path becomes the only parameter and optionally it can be provided via stdin.

The execution then becomes:
$ cloudflare-worker-local /path/to/worker.toml

or:
$ cloudflare-worker-local < /path/to/worker.toml

The stdin option gives the user the option to make the worker.toml file more ephemeral for cloud environments. It also makes it easier to compile the config on the fly:
$ cat /path/to/worker.toml $(echo '[secret]\nmysecret =' $KUBERNETES_OR_DOCKER_SECRET) | cloudflare-worker-local

If this is acceptable will produce a PR

I`d like to be able to test the redirect functionality locally (as dummy test at least), but the Response.redirect function is not defined.

Do you have any plans on supporting it ?

const handler = async function(request) {
  return Response.redirect('some_url', 200);
};

addEventListener('fetch', async function(event) {
  event.respondWith(handler(event.request));
});

Opening to continue discussion from #13 (comment)

What do you think about splitting start.js into two files:

1. cli.js: argv, SIGHUP and file access related code
2. index.js: exports a start and updateWorkerScript functions, re-exports the worker.js and server.js exports?

import { start, updateWorkerScript } from 'cloudflare-worker-local';

const options = {
  port: 4000,
  workerScript: '... javascript ...',
  upstreamHost: 'localhost:3000',
  kvStores: [],
  workerCount: 1
};

start(options);
...
...
updateWorkerScript('... js ...');

This would make the following scenario cleaner / less hacky:

import chokidar from 'chokidar';
import { access, F_OK } from 'fs';

const filename = 'dist/index.js';

let started = false;

async function handler() {
  const exists = await new Promise(resolve => access(filename, F_OK, e => resolve(!e)));
  if (exists && !started) {
    started = true;
    // TEMP: set args expected by cloudflare-worker-local
    process.argv[2] = filename;
    process.argv[3] = 'localhost:7001';
    process.argv[4] = '7000';
    import('cloudflare-worker-local/start.js');
  } else if (exists && started) {
    process.emit('SIGHUP');
  }
}

let timeout = 0;
function debouncedHandler() {
  clearTimeout(timeout);
  timeout = setTimeout(handler, 300);
}

chokidar.watch(filename)
  .on('add', debouncedHandler)
  .on('change', debouncedHandler);

I'm sending 4.6MB POST request which should be fine size for cloudflare workers, but it results with following error when testing locally:

[0] PayloadTooLargeError: request entity too large
[0]     at readStream (/Users/sheerun/Source/.../api/node_modules/raw-body/index.js:155:17)
[0]     at getRawBody (/Users/sheerun/Source/.../api/node_modules/raw-body/index.js:108:12)
[0]     at read (/Users/sheerun/Source/.../api/node_modules/body-parser/lib/read.js:77:3)
[0]     at rawParser (/Users/sheerun/Source/.../api/node_modules/body-parser/lib/types/raw.js:81:5)
[0]     at Layer.handle [as handle_request] (/Users/sheerun/Source/.../api/node_modules/express/lib/router/layer.js:95:5)
[0]     at trim_prefix (/Users/sheerun/Source/.../api/node_modules/express/lib/router/index.js:317:13)
[0]     at /Users/sheerun/Source/.../api/node_modules/express/lib/router/index.js:284:7
[0]     at Function.process_params (/Users/sheerun/Source/.../api/node_modules/express/lib/router/index.js:335:12)
[0]     at next (/Users/sheerun/Source/.../api/node_modules/express/lib/router/index.js:275:10)
[0]     at expressInit (/Users/sheerun/Source/.../api/node_modules/express/lib/middleware/init.js:40:5)

Could you set better limits for body parser?

Hi,
I defined some environment variables in the toml file vars = {MAX_BOT_SCORE = "1"} and trying to read it using process.env.MAX_BOT_SCORE but I get a "process is not defined" exception.
Any thoughts?
Thanks

Should we add support for cloudflare headers? Particularly CF-Connecting-IP.

https://support.cloudflare.com/hc/en-us/articles/200170986-How-does-Cloudflare-handle-HTTP-Request-headers-

I am creating this issue just as a placeholder for a future version.
It would be good to provide an interface that lends itself to the web stack ecosystem.
Providing a uWSGI request interface allows integration with larger projects like nginx and apache.
This also allows for better process handling by these projects when trying to scale up to many workers.

https://en.wikipedia.org/wiki/JSGI

https://uwsgi-docs.readthedocs.io/en/latest/Emperor.html

https://uwsgi-docs.readthedocs.io/en/latest/V8.html

Getting this error while running google index script TypeError: Cannot read property 'asn' of undefined

https://gitlab.com/GoogleDriveIndex/Google-Drive-Index

the issue is while running the below generated .js file. You can regenerate this issue using below link. kindly fix it thank you

https://bdi-generator.hashhackers.com/

https://github.com/LeeluPradhan/G-Index/blob/master/Search/index.js

This script is not working.This is a gdrive index script.The index page is loading,but the files didnt loading..the page kept showing loading. can you please fix this ?

The example worker modifies the Response object from a fetch call. This is not allowed in Workers. There isn't a clean way that I know of to deeply freeze an object in Javascript, but at a minimum this example is invalid:

addEventListener("fetch", e => {
  e.respondWith(fetchAndAddHeader(e.request));
});

async function fetchAndAddHeader(request) {
  const response = await fetch(request);

  if (response.status === 200) {
    response.headers.set("Foo", "Bar"); // <--  KABOOM
  } else {
    response.headers.set("Foo", "Not Bar"); // <-- KABLAMO
  }

  return response;
}