nponiros / sync_server Goto Github PK
View Code? Open in Web Editor NEWA simple server which can be used to synchronize data from multiple devices
License: MIT License
A simple server which can be used to synchronize data from multiple devices
License: MIT License
The current solution is insufficient and does not work with web sockets. It works with ajax assuming that during the next sync the client revision is incremented. If the client does not receive our partials and calls the server again with the old revision it will not work correctly.
I'm using the sync server in web socket mode with the sample WebSocketSyncProtocol.js from Dexie (slightly modified to wait on clientIdentity
response before subscribing).
I had a situation where, when doing an update of a record in one client, the update was not getting replicated to the other client. I poked around and found that it was because, unless the second client did updates that caused the server to send the first client updates, the first client did not have it's baseRevision
updated. Therefore, any further updates (to the same record) made in client 1 were sent as changes referencing its initial baseRevision
and thus filtered by the server as a conflict.
Digging deeper, I believe this is because the server, when notifying clients of changes, skips the client initiating the changes. I think the initiating client still needs the changes
message to get the correct revision
of the server. It's ok to send it because the changeset to the initiating client will be an empty array. The message to that client then just becomes a notification of the new server revision.
I made the change to not filter out the initiating client and things seem to be working correctly in my case. I will include a PR with this issue to see if you agree.
Thanks for the consideration and for creating this server!
There are cases where Dexie.Syncable might send again data which were already saved by the server. For example if the connection times out and Syncable does not get a change to call onChangesAccepted()
, it would resend the data on the next sync.
In this case the server should either ignore the CREATE change for which we already saved data or overwrite the data.
The read.me says:
cleanup changes table -> Can only do that after Dexie.Syncable supports the clear flag
Can you provide any more info on this. Is there a Dexie issue on the clear
flag etc.
The ability to cleanup the changes
table along with any other tables will be important in production with larger databases.
@nponiros I'm using sync-server with a change to use Websockets instead of http. This is based on the Dexie.js Websocket samples.
So far I've only had to replace your server.js with mine and much to my amazement it worked first time. You've done a great job of restructuring the Dexie sample code.
I am having a problem with specific changes not syncing when there are multiple clients and don't know yet whether this is a problem in my websocket server implementation or elsewhere. Have you tried sync-server with multiple clients (Browsers). ie. Add data in one and check it syncs correctly in the other one?
Also could you look at sendAnyChanges()
in WebSocketSyncServer.js and let me know if your getServerChanges()
should match the behaviour of sendAnyChanges()
. ref this comment:
Get all changes after syncedRevision that was not performed by the client we're talkin' to.
The XHR implementation does work a bit differently to the Websocket one which starts with a "subscribe" action to get any changes from the server that aren't in the client. This may be where I'm going wrong.
I'm calling syncHandler()
when the subscribe
message is received with syncData
including the syncedRevision
.
I have cloned the repo, run npm install, run the init command in an empty directory, and attempted to start the server with --path to the empty directory
node is displaying an error that says Action not defined not supported
any suggestions ?
@nponiros
So the server seems to boot and work well with the samples included. The only change I had to make so far was in the changes_table.js. I kept getting an error when trying to boot the server telling me that
this.store.find(query).sort({ rev: 1 }).exec()
is not a function. so i swapped the function with
this.store.find(query).sort({ rev: 1 }).toArray()
seems to work just fine.
Does your syn-client package have a web small website example to test ?
Due to a limitation in NeDB an attribute with a dot is not supported. Need to replace the dot with some other string during addition into the changes table and replace again the string with the dot during reading.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.