Code Monkey home page Code Monkey logo

core's Introduction

ZenFS

ZenFS is a file system that emulates the NodeJS filesystem API.

It works using a system of backends, which are used by ZenFS to store and retrieve data. ZenFS can also integrate with other tools.

ZenFS is a fork of BrowserFS.

Backends

ZenFS is modular and extensible. The core includes a few built-in backends:

  • InMemory: Stores files in-memory. This is cleared when the runtime ends (e.g. a user navigating away from a web page or a Node process exiting)
  • Overlay: Use read-only file system as read-write by overlaying a writable file system on top of it.
  • AsyncMirror: Use an asynchronous backend synchronously. This is very helpful for asynchronous backends

Note

When constructed, AsyncMirror loads the entire contents of the async file system into a synchronous backend. It performs operations on the synchronous file system and then queues them to be mirrored onto the asynchronous backend.

ZenFS supports a number of other backends. Many are provided as seperate packages under @zenfs. More backends can be defined by separate libraries by extending the FileSystem class and/or providing a Backend object.

For more information, see the docs.

Installing

npm install @zenfs/core

Usage

Note

The examples are written in ESM.
If you are using CJS, you can require the package.
If using a browser environment without support for type=module in script tags, you can add a script tag to your HTML pointing to the browser.min.js and use ZenFS with the global ZenFS object.

import fs from '@zenfs/core'; // You can also use the named export, `fs`

fs.writeFileSync('/test.txt', 'Cool, I can do this in any JS environment (including browsers)!');

const contents = fs.readFileSync('/test.txt', 'utf-8');
console.log(contents);

Using different and/or multiple backends

A single InMemory backend is created by default, mounted on /.

You can configure ZenFS to use a different backend and mount multiple backends. It is strongly recommended to do so using the configure function.

You can use multiple backends by passing an object to configure which maps paths to file systems.

The following example mounts a zip file to /zip, in-memory storage to /tmp, and IndexedDB to /home. Note that / has the default in-memory backend.

import { configure, InMemory } from '@zenfs/core';
import { IndexedDB } from '@zenfs/dom';
import { Zip } from '@zenfs/zip';

const zipData = await (await fetch('mydata.zip')).arrayBuffer();

await configure({
	'/mnt/zip': { backend: Zip, zipData },
	'/tmp': InMemory,
	'/home': IndexedDB,
};

Tip

When configuring a mount point, you can pass in

  1. A Backend object, if the backend has no required options
  2. An object that has the options accepted by the backend and a backend property which is a Backend object
  3. A FileSystem instance (not recommended)

Here is an example that mounts the Storage backend from @zenfs/dom on /:

import { configure, fs } from '@zenfs/core';
import { Storage } from '@zenfs/dom';

await configure({ backend: Storage });

if (!fs.existsSync('/test.txt')) {
	fs.writeFileSync('/test.txt', 'This will persist across reloads!');
}

const contents = fs.readFileSync('/test.txt', 'utf-8');
console.log(contents);

FS Promises

The FS promises API is exposed as promises.

import { configure, promises } from '@zenfs/core';
import { IndexedDB } from '@zenfs/dom';

await configure({ '/': IndexedDB });

const exists = await promises.exists('/myfile.txt');
if (!exists) {
	await promises.write('/myfile.txt', 'Lots of persistant data');
}

Note

You can import the promises API using promises, or using fs.promises on the exported fs.

Important

ZenFS does not provide a seperate public import for importing promises like fs/promises. If you are using ESM, you can import promises functions like fs/promises from the dist/emulation/promises.ts file, though this may change at any time and is not recommended.

Using asynchronous backends synchronously

You may have noticed that attempting to use a synchronous function on an asynchronous backend (e.g. IndexedDB) results in a "not supplied" error (ENOTSUP). If you would like to use an asynchronous backend synchronously you need to wrap it in an AsyncMirror:

import { configure, fs, AsyncMirror, InMemory } from '@zenfs/core';
import { IndexedDB } from '@zenfs/dom';

await configure({
	'/': {
		backend: AsyncMirror,
		sync: InMemory,
		async: IndexedDB,
	},
});

fs.writeFileSync('/persistant.txt', 'My persistant data');

Mounting and unmounting, creating backends

If you would like to create backends without configure (e.g. to do something dynamic at runtime), you may do so by importing the backend and calling createBackend with it.

You can then mount and unmount the backend instance by using mount and umount.

import { configure, createBackend, InMemory } from '@zenfs/core';
import { IndexedDB  } from '@zenfs/dom';
import { Zip } from '@zenfs/zip';

await configure({
	'/tmp': InMemory,
	'/home': IndexedDB,
};

fs.mkdirSync('/mnt');

const res = await fetch('mydata.zip');
const zipFs = await createBackend(Zip, { zipData: await res.arrayBuffer() });
fs.mount('/mnt/zip', zipFs);

// do stuff with the mounted zip

fs.umount('/mnt/zip'); // finished using the zip

Warning

Instances of backends follow the internal ZenFS API. You should never use a backend's methods unless you are extending a backend.

Using with bundlers

ZenFS exports a drop-in for Node's fs module (up to the version of @types/node in package.json), so you can use it for your bundler of preference using the default export.

Building

  • Make sure you have Node and NPM installed. You must have Node v18 or newer.
  • Install dependencies with npm install
  • Build using npm run build
  • You can find the built code in dist.

Testing

Run unit tests with npm test.

core's People

Contributors

james-pre avatar perimosocordiae avatar lavelle avatar bpowers avatar hrj avatar emeryberger avatar billiegoose avatar corhere avatar dustinbrett avatar jvilk avatar snowyu avatar kkoreilly avatar narazaka avatar timdream avatar db48x avatar 1j01 avatar danielruf avatar dreamlayers avatar ajainvivek avatar sheepmaster avatar codefrau avatar boryagames avatar pkit avatar kevinramharak avatar hexxeh avatar misak113 avatar nexusnull avatar tjkoury avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.