blackboardsh / electrobun Goto Github PK

Goal

• seamless update experience
• all you need is a static file host
• decouple downloading new app code from downloading new bun or webview runtime (when not using native webview) versions
• support multiple channels (dev, canary, stable)

Two paths:

1. bun and webview runtimes are stored in a global location
2. bun and webview runtimes are bundled with your app bundle

Open questions:

1. when supporting multiple OSes later, with either of these paths be easier or not feasible. lean toward cross-platform consistency where possible.
2. what manages the update logic (launcher binary, the bun api)? Most likely in bun so we can leverage native ui and notifications
3. when does auto-downloading and auto-applying of updates happen? app shutdown, app startup, etc.

Build flow

Option 1: Global runtime cache

dev

• use symlinks in bundle to bun/webview runtimes in node_modules
  • custom webview runtime can be configured in electrobun.config to a local path
• create app bundle with app code

distribution

• use symlinks to global cache for bun/webview runtimes
• package and notarize app
• upload with versioned filenames
  • app bundle
  • bun runtime
  • webview runtime
  • new update.json

Option 2: Bundled runtimes

dev

• copy runtimes into app bundle

distribution

• copy runtimes into app bundle
• package and notarize app
• upload with versioned filenames
  • complete app bundle for first time users
  • app code as .zip
  • bun runtime
  • webview runtime
  • new update.json

Update flow

• check current channel local version.json vs. hosted update.json version
• if newer then download in background (allow listening to these events)
• downloads bun, webview runtime, and app resources separately
• downloads to a global path
• automatically applies when the app shuts down or starts up
• checksum of each part we need to update
• if all are valid then copies app bundle to tmp location, copies parts into tmp app bundle
• verifies app bundle checksum (it needs to match what was notarized)
• if valid then replaces itself and relaunches itself
• local version.json is part of resources so gets updated to whatever channel/version you change to

Files

// version.json -- in the app bundle
{
    "version": "1.0.0",
    "channel": "stable",
    "url": "http://example.com/update.json"
}

// update.json -- hosted somewhere
{
  "baseUrl": "http://example.com/updates/",
  "channels": {
    "stable": {
      "1.0.0": {
        "notes": "This is a stable release",
        "targets": {
          "windows-x64": {
            "bundle": {
              "checksum": "1234567890abcdef",
              "filesize": 102900
            },
            "app": {
              "checksum": "1234567890abcdef",
              "filesize": 102400
            },
            "bun": {
              "checksum": "abcdef1234567890",
              "filesize": 20480
            },
            "webview": {
              "checksum": "fedcba0987654321",
              "filesize": 30720
            }
          },
          "linux-arm64": {
            "bundle": {
              "checksum": "1234567890abcdef",
              "filesize": 102900
            },
            "app": {
              "checksum": "0987654321fedcba",
              "filesize": 51200
            },
            "bun": {
              "checksum": "abcdef1234567890",
              "filesize": 15360
            },
            "webview": {
              "checksum": "1234567890abcdef",
              "filesize": 20480
            }
          }
        }
      }
    },
    "canary": {
      "1.1.0": {
        "notes": "This is a beta release",
        "targets": {
          // Similar structure for canary builds
        }
      }
    },
    "dev": {
      "1.2.0": {
        "notes": "This is a dev release",
        "targets": {
          // Similar structure for dev builds
        }
      }
    }
  }
}

Tasks:

• rewrite bundle launcher in zig (step toward cross platform)
• move mac bundle js files into Resources folder
• update electrobun.config to have version
• for dev the version number is the git commit sha
• add command to electrobun cli to build --canary and build --stable
• add version.json
• build command should only build for local architecture
• build --canary and --stable should minify everything and generate arm and intel builds
• configure objc and zig to build arm or intel
• generate update.json and all artifacts with deterministic names ready for upload
• basic file upload and replace for google cloud bucket
• update logic (in electrobun's bun api, and/or zig launcher) to read local.json, update.json, downloading, verifying, and installing updates)
• emit events for the update flow

This is a high level roadmap for how we're thinking about building Electrobun over the next few months

• v0.1.0 Core Architecture
  • Initial bun api
  • Initial webview api
  • Initial zig process
  • Initial native bindings for opening a window and webview via objective c
  • RPC
    • RPC between bun and zig
    • RPC between bun and webviews
    • Define RPC Schema types and handlers in typescript
    • Zig promise-like mechanism for waiting for RPC responses
  • Initial electrobun.config format
    • build main process code
    • define and build multiple webview scripts (preload or script tag scripts)
    • copy static assets into the build folder
    • expose built static files via views:// schema
  • native bindings
    • Create a window with a webview
      • load a remote url
      • load inline html
      • load local html, js, css, and other assets defined in electrobun.config via views:// schema
      • define preload scripts via views:// schema
      • add global or webview specific will-navigate handler with ability to allow/block
  • electrobun event emitter, with ability to set, read, and override return values across listeners
  • electrobun cli
    • should build and run your bun and view script files
    • read electrobun.config files
    • build macos bundle
    • wire up log output to terminal
• v0.2.0 Packaging, signing, and updating
  • Electrobun cli - building locally
    • organize files in the macos app bundle better
    • should build an optimized/minified mac app bundle
    • should install specific versions of bun runtime in a global location and verify checksum when running
    • should handle code signing on osx
    • should create update metadata files
    • should zip and upload built assets to a server (likely gcloud or github)
  • electrobun.config
    • configure code signing
    • configure packaging minification and other settings
    • configure artifact file upload
    • reads .env file or similar for keys
  • bun api
    • update logic for developer apps
    • bun runtime update logic for security updates
• v0.3.0 Optional Bundled Web Runtime (webview)
  • pick one, likely chromium embedded framework or similar
  • match current native webview api with custom webview
    • open a window and create a webview
    • support remote url, inline html
    • preload scripts
    • will-navigate handler and response
    • views:// schema
• v0.4.0 Intel mac builds
  • build objc, zig, etc. for intel mac
  • should be able to build Electrobun apps on intel macs
  • should be able to build Electrobun apps for and distribute to intel macs
• v0.5.0 API Parity
  • Build out Electrobun apis to greater coverage compared to Electron and Tauri
  • Migrate our apps from Electron to Electrobun
• v0.6.0 Windows Support
• v0.7.0 Linux Support
• v0.8.0 Stability
• v1.0.0 First Stable API |

The example app should be interactive with a sidebar that lets users explore all the Electrobun features and then jump into the code to see how it works. It would also function as a smoketest/QA tool when iterating on new features

Originally the thought was to just separate the bun runtime from your application code and download them separately. this made more sense when all your app files were packed into a tar or asar file or embedded into the zig binary.

With app bundles there are also complexities because you need the codesign info and stapled notarization and so on.

But the reality is a given app will have lots of separate files (eg: included binaries and assets that can't be bundled) along with text bundles and/or embedded files.

With a typical .blockmap mechanism you can quickly run into offset issues. ie: if some bytes changed near the start of the chunking changed it could offset the rest causing it to think that the whole rest of the chunks need to be downloaded again.

On the other end of the spectrum are things like text diffing and Courgette (which looks at sort of an assembly ast of an executable binary and determines what's actually changed to eliminate offset misinterpretation within a binary).

I think what's needed for the generic electrobun case is something like .blockmap with content delineated chunks (ie: looking at each file separately to eliminate offset issues between files), and then as a follow up later can add more advanced things for specific file types. Although keeping the complexity for the mechanism low and formats for the metadata files relatively stable is a higher priority than trying to save every possible byte of bandwidth during updates)

This task is to figure out a custom binary file format or pair of files that describes:

• all the distributable files
• their size/checksum
• their position inside the file
• a compact and compressed way to store those files for selective download
• possibly chunk those as well

So something like a tar or zip file, where every file is compressed, some kind of checksum is provided to quickly check if the file has changed, a mapping so you can quickly see if any files have been added or removed and what byte offset the compressed file bytes are at.

you can imagine either a single file where you just over-read the first 100 bytes from s3 and parse out how big the metadata section is and where the starting byte is, then read the metadata, then check those files against the current app bundle's files, fetch all the required file chunks, decompress, then build the new version of the bundle with a mix of local and once the bundle is verified to match codesign/notarization then swap it with the application wherever it was running and restart the app.

as well as a mechanism to do that to the local bundle, to determine exactly what byte ranges you need to download to build the new update bundle.

I think some intricacies of all this depend on how things get packaged on windows and linux, so I'm gonna break diff updates out of #3 into this task and revisit it later.

ideally the mechanism would just work with any object storage that allows downloading byte ranges (cloud storage / s3) so you don't need to run an actual server to provide update bytes.

It's cool to see your progress here! Should I tinker with it or wait a little bit?

oven-sh/bun#790 (reply in thread)

linux builds (if/when people ask)

I am hereby doing so. :)

Thanks !

Enable building for and distributing to linux targets.

Probably requires more than this, but off the top of my head

• zig bindings for webview for https://docs.gtk.org/gtk3/ / https://webkitgtk.org/
• compile zig bindings binary for linux targets
• electrobun cli should build some kind of linux application bundle
• update logic to decouple bun runtime, and app code
• (consider breaking out) compile cef for linux
• (consider breaking out) compile electrobun cli for linux to enable development on linux
• testing on ubuntu (what other distros?)

with an Applications shortcut or whatever to drag your app into