powersync-ja / powersync-js Goto Github PK

112.0 9.0 3.0 7 MB

SDK that enables local-first and real-time reactive apps with embedded SQLite for JavaScript clients, including React Native and Web

Home Page: https://www.powersync.com

License: Apache License 2.0

TypeScript 100.00%

local-first local-storage offline offline-first sql sqlite sync powersync state-management

powersync-js's Introduction

Bad connectivity is everywhere, and we're tired of it. PowerSync is on a mission to help developers write offline-first real-time reactive apps.

PowerSync is a service and set of SDKs that keeps Postgres databases in sync with on-device SQLite databases.

PowerSync JavaScript SDKs

powersync-js is the monorepo for PowerSync JavaScript SDKs.

Monorepo Structure: Packages

packages/react-native
- React Native SDK implementation (extension of packages/common)
packages/web
- JS Web SDK implementation (extension of packages/common)
packages/react
- React integration for PowerSync.
packages/attachments
- Attachments helper package for React Native and JavaScript/TypeScript projects.
packages/kysely-driver
- Kysely integration (ORM) for React Native and JavaScript/TypeScript projects.
packages/common
- Shared package: TypeScript implementation of a PowerSync database connector and streaming sync bucket implementation.

Demo Apps / Example Projects

Demo applications are located in the demos/ directory. Also see our Demo Apps / Example Projects gallery which lists all projects by the backend and client-side framework they use.

React Native

demos/react-native-supabase-todolist: A React Native to-do list example app using a Supabase backend.
demos/django-react-native-todolist A React Native to-do list example app using a Django backend.

Web

demos/react-supabase-todolist: A React to-do list example app using the PowerSync Web SDK and a Supabase backend.
demos/yjs-react-supabase-text-collab: A React real-time text editing collaboration example app powered by Yjs CRDTs and Tiptap, using the PowerSync Web SDK and a Supabase backend.
demos/vue-supabase-todolist: A Vue to-do list example app using the PowerSync Web SDK and a Supabase backend.
demos/angular-supabase-todolist An Angular to-do list example app using the PowerSync Web SDK and a Supabase backend.
demos/example-webpack: A minimal example demonstrating bundling with Webpack.
demos/example-vite: A minimal example demonstrating bundling with Vite.
demos/example-nextjs: An example demonstrating setup with Next.js.

Development

This monorepo uses pnpm.

Install workspace dependencies

pnpm install

Build packages

pnpm build:packages

Versioning

Development Packages

Development packages can be published by manually triggering the dev-packages workflow. Development packages are versioned as 0.0.0-{tag}-DATETIMESTAMP.

Production Packages

Pull requests should contain Changesets for changed packages.

Add changesets with

pnpm changeset add

Merging a PR with Changesets will automatically create a PR with version bumps. That PR will be merged when releasing.

React Native Quick SQLite Development

The PowerSync React Native SDK uses a fork of react-native-quick-sqlite

Testing live development changes to @journeyapps/react-native-quick-sqlite will not work with standard yarn link commands. Metro does not work well with symlinks facebook/metro#286.

The process of releasing development packages for @journeyapps/react-native-quick-sqlite for each change can be tedious and slow. A faster (and hackier) method is to use mtsl which will watch and copy the package into this workspace's node_modules.

npm install -g mtsl

mtsl add -s "[source path to your react-native-quick-sqlite repo folder]" -d "[this workspaces root node_modules folder]"/@journeyapps/react-native-quick-sqlite

mtsl start "[the id returned from step above]"

powersync-js's People

Contributors

Stargazers

Watchers

Forkers

bndkt chungminhtu aguynamedloren

powersync-js's Issues

Vite example shows this error `Could not POST streaming to /sync/stream` in `WebRemote.ts` and `404 Not Found http://localhost:3020/sync/stream`

When running example-vite pnpm dev with vite.config.ts:

server: {
   port: 3020,
},

The web seems working. But my browser shows this error:

Could not POST streaming to /sync/stream - 404 - Not Found: [WebRemote.ts:58:14](http://localhost:3020/@fs/Users/guillem/programacio/codi/cites/databases/powersync-js/packages/powersync-sdk-web/src/db/sync/WebRemote.ts)

And also XHR POST http://localhost:3020/sync/stream [HTTP/1.1 404 Not Found 1ms]

Do you know why?

Could not resolve all files for configuration ':journeyapps_react-native-quick-sqlite:androidJdkImage'

I have created a fresh expo project using npx create-expo-app app -t with Navigation (TypeScript) template and installed and configured powersync as per the instruction at https://github.com/powersync-ja/powersync-js/blob/main/packages/powersync-sdk-react-native/README.md

When I build the project to android, I get the following error

> Task :journeyapps_react-native-quick-sqlite:compileDebugJavaWithJavac FAILED

> Task :expo:compileDebugKotlin
w: file:///data/myapp.me/app/node_modules/expo/android/src/main/java/expo/modules/ReactActivityDelegateWrapper.kt:139:50 No cast needed

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':journeyapps_react-native-quick-sqlite:compileDebugJavaWithJavac'.
> Could not resolve all files for configuration ':journeyapps_react-native-quick-sqlite:androidJdkImage'.
   > Failed to transform core-for-system-modules.jar to match attributes {artifactType=_internal_android_jdk_image, org.gradle.libraryelements=jar, org.gradle.usage=java-runtime}.
      > Execution failed for JdkImageTransform: /home/user/Android/Sdk/platforms/android-34/core-for-system-modules.jar.
         > Error while executing process /usr/lib/jvm/java-21-openjdk/bin/jlink with arguments {--module-path /home/user/.gradle/caches/transforms-3/5fc9c09a9df6137601206ecef23d2934/transformed/output/temp/jmod --add-modules java.base --output /home/user/.gradle/caches/transforms-3/5fc9c09a9df6137601206ecef23d2934/transformed/output/jdkImage --disable-plugin system-modules}

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
> Get more help at https://help.gradle.org.

Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

For more on this, please refer to https://docs.gradle.org/8.3/userguide/command_line_interface.html#sec:command_line_warnings in the Gradle documentation.

BUILD FAILED in 16s
170 actionable tasks: 3 executed, 167 up-to-date
Error: /data/myapp.me/app/android/gradlew exited with non-zero code: 1
Error: /data/myapp.me/app/android/gradlew exited with non-zero code: 1
    at ChildProcess.completionListener (/data/myapp.me/app/node_modules/@expo/spawn-async/build/spawnAsync.js:52:23)
    at Object.onceWrapper (node:events:633:26)
    at ChildProcess.emit (node:events:518:28)
    at maybeClose (node:internal/child_process:1105:16)
    at ChildProcess._handle.onexit (node:internal/child_process:305:5)
    ...
    at Object.spawnAsync [as default] (/data/myapp.me/app/node_modules/@expo/spawn-async/build/spawnAsync.js:17:21)
    at spawnGradleAsync (/data/myapp.me/app/node_modules/@expo/cli/build/src/start/platforms/android/gradle.js:72:46)
    at Object.assembleAsync (/data/myapp.me/app/node_modules/@expo/cli/build/src/start/platforms/android/gradle.js:52:18)
    at runAndroidAsync (/data/myapp.me/app/node_modules/@expo/cli/build/src/run/android/runAndroidAsync.js:36:24)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

package.json

{
  "name": "myapp.me",
  "main": "expo-router/entry",
  "version": "1.0.0",
  "scripts": {
    "start": "expo start",
    "android": "expo run:android",
    "ios": "expo run:ios",
    "web": "expo start --web",
    "test": "jest --watchAll"
  },
  "jest": {
    "preset": "jest-expo"
  },
  "dependencies": {
    "@azure/core-asynciterator-polyfill": "^1.0.2",
    "@expo/vector-icons": "^14.0.0",
    "@journeyapps/powersync-sdk-react-native": "^1.2.0",
    "@journeyapps/react-native-quick-sqlite": "^1.1.1",
    "@react-navigation/native": "^6.0.2",
    "base-64": "^1.0.0",
    "expo": "~50.0.6",
    "expo-font": "~11.10.2",
    "expo-linking": "~6.2.2",
    "expo-router": "~3.4.7",
    "expo-splash-screen": "~0.26.4",
    "expo-status-bar": "~1.11.1",
    "expo-system-ui": "~2.9.3",
    "expo-web-browser": "~12.8.2",
    "react": "18.2.0",
    "react-dom": "18.2.0",
    "react-native": "0.73.4",
    "react-native-fetch-api": "^3.0.0",
    "react-native-polyfill-globals": "^3.1.0",
    "react-native-safe-area-context": "4.8.2",
    "react-native-screens": "~3.29.0",
    "react-native-url-polyfill": "^2.0.0",
    "react-native-web": "~0.19.6",
    "text-encoding": "^0.7.0",
    "web-streams-polyfill": "^3.3.2",
    // "react-native-get-random-values": "^1.10.0" // Already installed as a peer-dependency (currently explicit installing raises an error related to dependency conflict)
  },
  "devDependencies": {
    "@babel/core": "^7.20.0",
    "@babel/plugin-transform-async-generator-functions": "^7.23.9",
    "@types/react": "~18.2.45",
    "jest": "^29.2.1",
    "jest-expo": "~50.0.2",
    "react-test-renderer": "18.2.0",
    "typescript": "^5.1.3"
  },
  "private": true
}

Developement Environment

OS: Manjaro (kernel 6.6)
Android sdk: 34 (emulator)
openjdk 21 2023-09-19
OpenJDK Runtime Environment (build 21+35)
OpenJDK 64-Bit Server VM (build 21+35, mixed mode, sharing)

PS: Before installing / configuring powersync, android build worked fine.

PowerSync.init() doesn't return on Windows

The demo repo is at https://github.com/hut36/tauri-powersync.git. This is a tauri project, which works fine on macOS.

The code in question is:

	async function setupPowerSync() {
		let result

		console.log('before powersync init')

		await PowerSync.init();  // <--- never returns on Windows.

		console.log('after powersync init')

		result = await PowerSync.execute(`insert into lists (id, name) values (uuid(), 'list0')`)
		console.log('insert result: ', result)

		result = await PowerSync.execute('select * from lists')
		console.log('select result: ', result)
	}

Support Kysely driver in React `@journeyapps/powersync-react` and `@journeyapps/powersync-react-native`

Right now hooks like usePowerSyncWatchedQuery only support strings.

export declare const usePowerSyncWatchedQuery: <T = any>(sqlStatement: string, parameters?: any[], options?: Omit<SQLWatchOptions, 'signal'>) => T[];

I expect to pass a Kysely query like db.selectFrom('quotes').selectAll().execute(). I'm new with SQL and Kysely, I'm not sure if passing execute is good.

export const db = wrapPowerSyncWithKysely<Database>(powerSync)
const selectAll = db.selectFrom('quotes').selectAll().execute()
...
usePowerSyncWatchedQuery(selectAll)

Watched `LEFT JOIN` query does not update on `local-only` table

Setup

I am using usePowerSyncWatchedQuery and have the following watched query that performs a LEFT JOIN across three tables:

Two normal PowerSync Table tables -> lists and todos
One LocalOnly table -> attachments

const todos = usePowerSyncWatchedQuery<TodoEntry>(
    `SELECT
            todos.id AS todo_id,
            todos*,
            attachments.id AS attachment_id, 
            attachments.*
        FROM 
            todos
        LEFT JOIN 
            lists ON todos.list_id = lists.id
        LEFT JOIN 
            attachments ON todos.photo_id = attachments.id
        WHERE 
            todos.list_id = ?`,
    [listID],
    { tables: ['todos', 'lists', 'attachments'] }
);

Problem

Without specifying tables in the options payload, the query does not update when making changes to the LocalOnly table attachments.

Solution

As pointed out by @rkistner,

[It] should be fixed in the underlying lib. I assume it's just this bit that doesn't work for local-only tables:
https://github.com/journeyapps/powersync-react-native-sdk/blob/af0031b0eabb4524ad2a507b09487c68d268abb1/packages/powersync-sdk-common/src/client/AbstractPowerSyncDatabase.ts#L404

The solution is to instead use the already defined RegEx, here: https://github.com/journeyapps/powersync-react-native-sdk/blob/af0031b0eabb4524ad2a507b09487c68d268abb1/packages/powersync-sdk-common/src/client/AbstractPowerSyncDatabase.ts#L40

example-vite failed to build on Windows

Windows version: Windows 11 23H2 22631.3296
powersync-js: 68738c5

The same build command (pnpm -F example-vite build) works on macOS.

The error output:

D:\repos\powersync-js> pnpm -F example-vite build

> [email protected] build D:\repos\powersync-js\demos\example-vite
> vite build

vite v5.1.4 building for production...
✓ 3 modules transformed.
x Build failed in 52ms
error during build:
Error: [commonjs--resolver] Failed to resolve entry for package "@journeyapps/powersync-sdk-web". The package may have incorrect main/module/exports specified in its package.json.
    at packageEntryFailure (file:///D:/repos/powersync-js/demos/example-vite/node_modules/vite/dist/node/chunks/dep-jDlpJiMN.js:48038:17)
    at resolvePackageEntry (file:///D:/repos/powersync-js/demos/example-vite/node_modules/vite/dist/node/chunks/dep-jDlpJiMN.js:48035:5)
    at tryNodeResolve (file:///D:/repos/powersync-js/demos/example-vite/node_modules/vite/dist/node/chunks/dep-jDlpJiMN.js:47805:20)
    at Object.resolveId (file:///D:/repos/powersync-js/demos/example-vite/node_modules/vite/dist/node/chunks/dep-jDlpJiMN.js:47568:28)
    at file:///D:/repos/powersync-js/demos/example-vite/node_modules/rollup/dist/es/shared/node-entry.js:18731:40
    at async PluginDriver.hookFirstAndGetPlugin (file:///D:/repos/powersync-js/demos/example-vite/node_modules/rollup/dist/es/shared/node-entry.js:18631:28)
    at async resolveId (file:///D:/repos/powersync-js/demos/example-vite/node_modules/rollup/dist/es/shared/node-entry.js:17298:26)
    at async ModuleLoader.resolveId (file:///D:/repos/powersync-js/demos/example-vite/node_modules/rollup/dist/es/shared/node-entry.js:17712:15)
    at async Object.resolveId (file:///D:/repos/powersync-js/demos/example-vite/node_modules/vite/dist/node/chunks/dep-jDlpJiMN.js:8171:10)
    at async PluginDriver.hookFirstAndGetPlugin (file:///D:/repos/powersync-js/demos/example-vite/node_modules/rollup/dist/es/shared/node-entry.js:18631:28)

`getNextCrudTransaction()` is not properly grouping `writeTransaction()` operations

I'd like to add PowerSync to an existing app and I've setup a playground to test basic PowerSync functionality. What I'm observing is that operations executed in the same writeTransaction() call are not grouped together in the same CrudTransaction returned by getNextCrudTransaction().

Here are some code snippets I'm using to test:

// Execute two inserts into the DB
const runTransactionTest = async () => {
    try {
        await powerSync.writeTransaction(async (tx) => {
            await tx.execute(`INSERT INTO action(id, actionType, actionId) VALUES ('1', 'foo', '123')`)
            await tx.execute(`INSERT INTO action(id, actionType, actionId) VALUES ('2', 'bar', '456')`)
        })
    } catch (ex) {
        Alert.alert('Error', ex.message)
    }
}

// Upload logic found in PowerSyncBackendConnector
async uploadData(database: AbstractPowerSyncDatabase) {
    const transaction = await database.getNextCrudTransaction()

    if (!transaction) {
        return
    }

    try {
        console.log(`Processing transaction ${transaction.transactionId} with ${transaction.crud.length} ops`)
        for (let op of transaction.crud) {
            console.log('Op data', op)
        }

        await transaction.complete()
    } catch (ex) {
        // Ignoring exceptions while testing
        await transaction.complete()
    }
}

When I trigger runTransactionTest I expect to see 3 log messages: 1 logging transaction details and 2 logging operation details.

Instead, here is what I see:

 LOG  Processing transaction undefined with 1 ops
 LOG  Op data {"data": {"actionId": "123", "actionType": "foo", "id": "1"}, "id": "1", "op": "PUT", "op_id": 15, "tx_id": 8, "type": "action"}
 LOG  Processing transaction undefined with 1 ops
 LOG  Op data {"data": {"actionId": "456", "actionType": "bar", "id": "2"}, "id": "2", "op": "PUT", "op_id": 16, "tx_id": 8, "type": "action"}

The one writeTransaction call is being processed as two separate transactions in the connector. Note also that transaction.transactionId is undefined yet the two operations have the same tx_id value.

I'm using iOS Simulator to test on an iPhone 15. Additionally, here are dependencies present in my package.json

"dependencies": {
  "@azure/core-asynciterator-polyfill": "^1.0.2",
  "@expo-google-fonts/inter": "^0.2.3",
  "@gorhom/bottom-sheet": "^4.5.1",
  "@journeyapps/powersync-sdk-react-native": "1.0.1",
  "@journeyapps/react-native-quick-sqlite": "1.0.0",
  "@legendapp/motion": "^2.2.1",
  "@react-native-async-storage/async-storage": "1.18.2",
  "@react-navigation/drawer": "^6.6.6",
  "@rneui/base": "4.0.0-rc.8",
  "@sentry/react-native": "5.10.0",
  "@tanstack/react-query": "^5.0.5",
  "@testing-library/jest-native": "^5.4.3",
  "@testing-library/react-native": "^12.4.2",
  "@uidotdev/usehooks": "^2.4.1",
  "base-64": "^1.0.0",
  "expo": "~49.0.21",
  "expo-application": "~5.3.0",
  "expo-asset": "~8.10.1",
  "expo-background-fetch": "~11.3.0",
  "expo-clipboard": "~4.3.1",
  "expo-constants": "~14.4.2",
  "expo-dev-client": "~2.4.12",
  "expo-device": "~5.4.0",
  "expo-file-system": "~15.4.5",
  "expo-font": "~11.4.0",
  "expo-image-picker": "~14.3.2",
  "expo-linking": "~5.0.2",
  "expo-localization": "~14.3.0",
  "expo-router": "^2.0.14",
  "expo-splash-screen": "~0.20.5",
  "expo-status-bar": "~1.6.0",
  "firebase": "^10.5.2",
  "i18next": "^23.6.0",
  "intl-pluralrules": "^2.0.1",
  "jest": "^29.7.0",
  "lodash": "^4.17.21",
  "mathjs": "^12.2.0",
  "react": "18.2.0",
  "react-dom": "18.2.0",
  "react-hook-form": "^7.47.0",
  "react-i18next": "^13.3.1",
  "react-modal-sheet": "^2.2.0",
  "react-native": "0.72.6",
  "react-native-drawer-layout": "^3.2.2",
  "react-native-element-dropdown": "^2.10.1",
  "react-native-fetch-api": "^3.0.0",
  "react-native-gesture-handler": "~2.12.0",
  "react-native-get-random-values": "~1.9.0",
  "react-native-loading-spinner-overlay": "^3.0.1",
  "react-native-modal": "^13.0.1",
  "react-native-polyfill-globals": "^3.1.0",
  "react-native-reanimated": "~3.3.0",
  "react-native-round-flags": "^1.0.4",
  "react-native-safe-area-context": "4.6.3",
  "react-native-screens": "~3.22.0",
  "react-native-toast-message": "^2.2.0",
  "react-native-url-polyfill": "^2.0.0",
  "react-native-web": "0.19.9",
  "reflect-metadata": "^0.1.13",
  "sentry-expo": "~7.1.0",
  "styled-components": "^6.1.0",
  "text-encoding": "^0.7.0",
  "ts-jest": "^29.1.1",
  "typeorm": "^0.3.17",
  "uuid": "^9.0.1",
  "web-streams-polyfill": "^3.3.2"
},
"devDependencies": {
  "@babel/core": "7.23.5",
  "@babel/plugin-proposal-export-namespace-from": "^7.18.9",
  "@types/jest": "^29.5.11",
  "@types/lodash": "^4.14.202",
  "@types/react": "18.2.42",
  "@types/styled-components": "^5.1.29",
  "@types/uuid": "^9.0.7",
  "@typescript-eslint/eslint-plugin": "^6.9.0",
  "@typescript-eslint/parser": "^6.9.0",
  "ajv": "^8.12.0",
  "axios": "^1.6.2",
  "babel-plugin-transform-typescript-metadata": "^0.3.2",
  "eslint": "8.55.0",
  "eslint-config-universe": "12.0.0",
  "eslint-plugin-import": "^2.29.0",
  "eslint-plugin-json": "^3.1.0",
  "eslint-plugin-jsx-a11y": "^6.7.1",
  "eslint-plugin-prettier": "^5.0.1",
  "eslint-plugin-promise": "^6.1.1",
  "eslint-plugin-react": "^7.33.2",
  "eslint-plugin-react-hooks": "^4.6.0",
  "prettier": "^3.1.0",
  "ts-node": "^10.9.2",
  "typescript": "5.3.2"
}