yorkie-team / yorkie-js-sdk Goto Github PK
View Code? Open in Web Editor NEWYorkie JavaScript SDK
Home Page: https://yorkie.dev/docs/js-sdk
License: Apache License 2.0
Yorkie JavaScript SDK
Home Page: https://yorkie.dev/docs/js-sdk
License: Apache License 2.0
While running the toy example (docker-compose up
=> npm start
) I noticed that if I restart the frontend server, synchronization happens only after a client makes some local changes to the document. So for example if I make changes on a client, the changes are not reflected in any other clients until they create some events on their part (e.g., moving the cursor around, writing to the document et cetera)
What you expected to happen: Seamless synchronization
How to reproduce it (as minimally and precisely as possible):
docker-compose up
in docker/
npm start
Ctrl + C
)npm start
Anything else we need to know?:
When I quit the frontend server, it does show an error: Assertion failed: (0), function uv_close, file ../deps/uv/src/unix/core.c, line 178.
Then after restarting the server, it produces this error whenever I close a client browser tab: [HPM] Error occurred while trying to proxy request /api.Yorkie/WatchDocuments from localhost:9000 to http://localhost:8080 (ECONNRESET) (https://nodejs.org/api/errors.html#errors_common_system_errors)
Environment:
yorkie version
): Yorkie: 0.0.10 (Commit: 165cdbf / Go: go1.14.3)We need to add reference docs so that users can use the JS SDK in more detail. It would be nice to take a look at TypeDoc, which has been introduced by other TypeScript based libraries. And we'll also need to add TSDoc comments as well.
Also, it would be nice to think about how to integrate it naturally in yorkie.dev.
A script error occurs when a local update comes in while applying remote changes. JavaScript has no multi-routines or mutex, so We should think about a solution that is different from the traditional approach.
What happened:
The result of the two replicas is not the same.
What you expected to happen:
The result of the two replicas should be the same
How to reproduce it (as minimally and precisely as possible):
Yorkie.dev
Anything else we need to know?:
Environment:
yorkie version
): public yorkie dev server versionAfter several typing, it looks like data is twisted at some point.
What happened:
After twisting, Quill does not work when changing the style of the entire text. ex) H1, H2, list order
What you expected to happen:
In the picture, Fail in quill lib,
The data looks like remote 35-35 {"list":"ordered"}
but It should come remote 35-36 {"list":"ordered"}
. retain: 1 is missing.
How to reproduce it (as minimally and precisely as possible):
This occurs intermittently when constantly changing style and text.
Anything else we need to know?:
Environment:
Now yorkie.dev environment
During the code development process, I checked the following errors.
It occurred because of the code below.
catch(err){
logger.error(`[AC] c:"${this.getKey()}" err :"${err}"`);
}
I am wondering whether it is better to be able to receive everything even an object, or whether it is better to pass an only string as the argument.
If the logger receives all values, it will proceed as follows.
info: (...messages: unknown[]): void => {
if (level > LogLevel.Info) {
return;
}
console.log('YORKIE I: ', ...messages);
},
Currently, several tests rely on setTimeout
to wait for changes to be applied.
However, this is not a good practice as it is nondeterministic and could lead to different results (for example, poor network condition can lead to sync occurring later than the time set by the setTimeout
).
Therefore, I suggest that we remove setTimeout
and make it event driven.
We currently only support Quill editor fonts and paragraph styles, not media.
To support images and videos, we need to handle JSON as well as text in the insert command.
// Insert a bolded "Text"
{ insert: "Text", attributes: { bold: true } }
// Insert a link
{ insert: "Google", attributes: { link: 'https://www.google.com' } }
// Insert an embed
{
insert: { image: 'https://octodex.github.com/images/labtocat.png' },
attributes: { alt: "Lab Octocat" }
}
// Insert another embed
{
insert: { video: 'https://www.youtube.com/watch?v=dMH0bHeiRNg' },
attributes: {
width: 420,
height: 315
}
}
Perhaps if we don't support partial modification of the insert's value, we can serialize the JSON and treat it as a string.
When we receive changes from remote, the changes is applied to the document in Yorkie. However, when we use an external model like CodeMirror example, we need to also apply the changes to the external model.
To handle this, we need an interface that can notify the outside of the SDK.
I am currently implementing increase operation
. And there was a problem during implementation.
This problem occurred because the ObjectProxy
get the value property of JSONPrimitive
.
https://github.com/yorkie-team/yorkie-js-sdk/blob/master/src/document/proxy/proxy.ts#L39
In my opinion, like the go version, ObjectProxy
needs to get a JSONPrimitive
instance to use a function like increase
.
And this seems to be consistent with the go version.
We implemented GC of the text type datatype in yorkie-team/yorkie#58. We also need to implement GC in Text and RichText implemented in JS SDK.
Implementing GC on the data type used by Agent, we can reduce the size of the snapshot stored in the DB. However, if we do not implement it in the SDK, the garbages of the replica document are not collected in the Client.
updatedAt
is used to indicate that the relative position of the element has changed by move operation in the array, not the value change. Therefore, updatedAt
has changed to movedAt
in Yorkie and should be reflected in yorkie-js-sdk as well.
yorkie-js-sdk/src/core/client.ts
Line 291 in d69214b
This code is an infinite loop.
So PushPull do action continuesly.
What happened:
The attribute option is not applied for richText
it('should handle edit operations', function () {
const doc = Document.create('test-col', 'test-doc');
assert.equal('{}', doc.toSortedJSON());
// -- ins links ---
// | |
// [init] - [ABC] - [\n] - [D]
doc.update((root) => {
const text = root.createRichText('k1');
text.edit(0, 0, 'ABCD', { b: '1' }); // <-- bold attr
text.edit(3, 3, '\n');
}, 'set {"k1":"ABC\nD"}');
doc.update((root) => {
assert.equal(
'[0:00:0:0 ][1:00:2:0 ABC][1:00:3:0 \n][1:00:2:3 D][1:00:1:0 \n]',
root['k1'].getAnnotatedString(),
);
});
assert.equal(
'{"k1":[{"attrs":{},"content":ABC},{"attrs":{},"content":\n},{"attrs":{},"content":D},{"attrs":{},"content":\n}]}',
doc.toSortedJSON(),
); // This test should not passe, but it is pass now
});
What you expected to happen:
The attribute should be applid for richText
How to reproduce it (as minimally and precisely as possible):
Anything else we need to know?:
Environment:
yorkie version
):I found the modified code without prettier
applied in some files during code modification.
And it can include changes that are independent of the commit.
I sent a pull request for prevent this. (#52)
If the contributor modifies the code without using prettier
, this problem will occur again.
So how about recommending the use of 'prettier' in contribution guide?
Yorkie supports seven primitive types: boolean, integer, long, double, string, byte array, and date. For now, We implemented only three types: boolean, integer and string in JS SDK.
When implementing the rest of the types, it would be nice to refer to Yorkie's primitive.
relevant test:
https://github.com/yorkie-team/yorkie-js-sdk/blob/master/test/yorkie_test.ts#L159-L175
value to bytes:
https://github.com/yorkie-team/yorkie/blob/master/pkg/document/json/primitive.go#L128-L158
bytes to value:
https://github.com/yorkie-team/yorkie/blob/master/pkg/document/json/primitive.go#L42-L66
Description:
#133 (comment)
From the above PR comment, we found out that depending on the version of protobuf
, the behavior may not match our intention.
So we think we need to figure out the proper protobuf
version and write it to README.md.
Why:
It can reduce unnecessary confusion for developers.
We separated the tests according to their scope of responsibility through the yorkie-team/yorkie#118 issue. I think yorkie-js-sdk
also needs to improve the test structure according to yorkie
.
Description:
If you check yorkie_pb.d.ts
after building protobuf, the editor warns you.
Array can be declared generic in actual typescript,
In yorkie_pb.d.ts
, Array is a phenomenon that occurs because there is a conflict because Array class exists after protobuf build.
Is it safe to use message names that can conflict with the data types provided by the language?
Why:
Assuming you add yorkie-java-sdk
, I wonder if it will be safe after the build result. This is because Object and Object messages are expected to collide in Java. I think this is worth testing.
Currently, In Yorkie project, we enforce comments for important functions and added comments. yorkie-team/yorkie#114
In my view, it would be great to also apply TSLint completed-docs rule for writing comments in yorkie-js-sdk
How do you think about creating the setter in ArrayProxy?
It will be good to use or not?
Support network auto recovery
We introduce JavaScript Proxy to allow users to use yorkie-js-sdk's documents like plain objects. At that time, I didn't know the proxy and Vue.js works together.
Recently @lqez added a kanban board example using Vue.js and yorkie-js-sdk. And It is really amazing that yorkie-js-sdk's document and vue.js work together in a few lines.
https://github.com/yorkie-team/yorkie-team.github.io/blob/master/static/js/demo-kanban.js#L101-L106
https://yorkie.dev/demo
We need to know how to pass the remote changes to Vue.js efficiently when some of the fields changed in the document.
What happened:
Cannot pass test code
Yorkie Can handle concurrent moveBefore operations FAILED
AssertionError: expected '{"k1":[1,2,0]}' to equal '{"k1":[2,1,0]}'
What you expected to happen:
PASSED
How to reproduce it (as minimally and precisely as possible):
When I edit test code like below
d1.update((root) => {
const next = root['k1'].getElementByIndex(0);
const item = root['k1'].getElementByIndex(2);
root['k1'].moveBefore(next.getID(), item.getID());
assert.equal('{"k1":[2,0,1]}', root.toJSON());
});
await c1.sync();
d1.update((root) => {
const next = root['k1'].getElementByIndex(0);
const item = root['k1'].getElementByIndex(2);
root['k1'].moveBefore(next.getID(), item.getID());
assert.equal('{"k1":[1,2,0]}', root.toJSON());
});
from
yorkie-js-sdk/test/yorkie_test.ts
Lines 331 to 357 in 07f71fe
Anything else we need to know?:
I am researching moveAfter functions and I cannot found the purpose of some codes.
yorkie-js-sdk/src/document/json/rga_tree_list.ts
Lines 148 to 149 in 07f71fe
When I was debugging some codes,
prevNode
from const prevNode = this.findByCreatedAt(prevCreatedAt, value.getCreatedAt());
was not same with as I expected.
I cannot catch the purpose of this code.
yorkie-js-sdk/src/document/json/rga_tree_list.ts
Lines 120 to 134 in 07f71fe
Environment:
yorkie version
): 0.0.12yorkie-js-sdk/src/core/client.ts
Lines 126 to 131 in ef354f3
Currently, if there's a network error, client.activate()
will simply reject and finish. This means that even when the network comes back up, it's not going to attempt to activate the client anymore. Therefore, the user would have to do the following in order for everything to work.
But in case of 1., the network failure might be on our part (e.g., envoy went down) which the user wouldn't know of. In that case, the client would have to blindly refresh the page until it works.
Therefore I'm wondering if we should attempt to activate the client in a loop as in runSyncLoop
and runWatchLoop
, and never reject the promise.
I think this question applies to all methods that send a gRPC request (which include .activate()
, .deactivate()
, .attach()
and .detach()
)
Is there any way to use RichText or Markdown?
If not, how about supporting RichText or Markdown?
What happened:
As the title suggests, I've tried to attach to a document that has been previously detached.
This emits the following error message: "document already attached"
What you expected to happen:
I expected the client to reattach to the document with no problem.
How to reproduce it (as minimally and precisely as possible):
To reproduce it with minimal efforts, go to test/yorkie_test.ts
, then insert the following lines after withTwoClientsAndDocuments:L57 (after detaching from the document)
await client1.attach(doc1, true);
await client2.attach(doc2, true);
await client1.detach(doc1);
await client2.detach(doc2);
I accidentally added npm-check-updates
and committed. It must be removed.
https://github.com/yorkie-team/yorkie-js-sdk/blob/master/package.json#L55
If the application has its own model, Yorkie will provide a change handler that can synchronize with the application's model in the document.
On the other hand, we decided to find a way to use Yorkie directly as a model for the application.
First, Yorkie provides a native interface using Javascript Proxy like Immer, so we will find a way how to interact with the React framework in a similar way to Immer.
Thanks for the amazing works.
I've just started experimenting with yorkie and stumbled upon an issue. I noticed that we can display peers accessing a document in realtime (as found in this example). However, it is limited to get the peer ids (the id which is returned by the agent on client activation).
Is there a way to get more details (perhaps the key used on activation)?
My use case is that I'm trying to build a real time text editor, similar with Google Docs. I would like to display all user names accessing the document (not just the ids). I'm thinking of putting the client's key with the user id, so that I could retrieve more details (e.g. username) later in the other connected peer sessions. However, I could not find out how to do that. Maybe I'm missing something here.
I found insertAfter
and moveBefore
at https://github.com/yorkie-team/yorkie-js-sdk/blob/master/src/document/proxy/array_proxy.ts
But I can't figure out how to put an item in front of an array. Do I have to get the first item and use it as an argument of moveBofore
?
Google Docs shows that the document is either 'Saved to Drive' or is 'Working offline'. Latter is shown above.
Likewise, it would be useful for our users as well to know whether the document is being synced in real time, or is experiencing some sort of errors.
Thus @hackerwins suggested that we add a status icon in our examples.
Description:
Not long ago, when we introduced strict mode, we found out that we use null
and undefined
together. We need to use undefined
, not null
.
// tsconfig.json
...
"compilerOptions": {
...
"strictNullChecks": false, // We need to remove this line
}
...
https://github.com/yorkie-team/yorkie-js-sdk/blob/main/tsconfig.json#L10
Why:
Since grpc-web uses undefined
, it is better for us to use undefined
rather than null
to keep the code consistent. And projects are written in TypeScript also tend to use undefined.
client, document provided by yorkie currently showing d.ts information.
yorkie.d.ts
// client
import { Client, ClientOptions } from './core/client';
// document
import { Document } from './document/document';
During the PR, I felt that the values below would be needed.
// client.d.ts
export declare enum ClientStatus {
Deactivated = "deactivated",
Activated = "activated"
}
export declare enum StreamConnectionStatus {
Connected = "connected",
Disconnected = "disconnected"
}
export declare enum DocumentSyncResultType {
Synced = "synced",
SyncFailed = "sync-failed"
}
export declare enum ClientEventType {
StatusChanged = "status-changed",
DocumentsChanged = "documents-changed",
DocumentsWatchingPeerChanged = "documents-watching-peer-changed",
StreamConnectionStatusChanged = "stream-connection-status-changed",
DocumentSyncResult = "document-sync-result"
}
// document.d.ts
export declare enum DocEventType {
Snapshot = "snapshot",
LocalChange = "local-change",
RemoteChange = "remote-change"
}
When I import Yorkie JS SDK from CodePair created with create-react-app with TypeScript enabled, I confirmed that it does not work properly.
./node_modules/yorkie-js-sdk/src/yorkie.ts 24:33
Module parse failed: Unexpected token (24:33)
You may need an appropriate loader to handle this file type, currently no loaders are configured to process this file. See https://webpack.js.org/concepts#loaders
| // e.g) yorkie.createClient(...)
| export default {
> createClient: function (rpcAddr: string, opts?: ClientOptions): Client {
| return new Client(rpcAddr, opts);
| },
We need to modify the deployment script of JS SDK by following TypeScript's publishing guide.
We need to check the logic for drawing the selection.
yorkie-js-sdk/src/document/proxy/object_proxy.ts
Lines 66 to 73 in a9d0f41
While working on the CodePair example, I noticed that many errors occur in Typescript strict
mode. We can change it to the strict mode by adding a property to tsconfig.json.
// tsconfig.json
{
"compilerOptions": {
"sourceMap": true,
"target": "esnext",
"removeComments": false,
"allowSyntheticDefaultImports": true,
"strict": true // <-- this line to check strict at compile time
},
"include": ["src/**/*"],
"exclude": ["node_modules"],
"typeRoots": ["node_modules/@types"]
}
I think that checking in strict mode will further reduce the possibility of bugs.
What happened:
If there are more than 6 peers, there is no response from yorkie
What you expected to happen:
If there are more than 6 people, it should be possible to connect.
How to reproduce it (as minimally and precisely as possible):
If more than 6 people are connected, there is no response afterward, and if the connected peer goes out, the connection is possible from the next time.
Anything else we need to know?:
The request was made by yorkie-sdk-js, but the request was not received by yorkie.
It may be an envoy configuration issue
Environment:
yorkie version
): branch master commit f9b2bb6658e70f7ba52844e26c6e2c074a172bc2When developing new applications, users can use Yorkie's document directly as a model. However, applications that have already been developed have their own models. So We need to find a way to work with the model of an application that has already been developed.
Currently, Text provides a change handler for integration with CodeMirror.
https://github.com/yorkie-team/yorkie-js-sdk/blob/master/dist/index.html#L155-L176
And Document provides a change event stream.
https://github.com/yorkie-team/yorkie-js-sdk/blob/master/dist/drawing.html#L66
What happened:
I am working on issue #109. I worked so that ClientEventType
can be exported from yorkie.ts
.
And I tested at codepair.
The result was that the imported ClientEventType
was undefined.
I felt weird and also checked the RichText class that was originally exporting. This is also undefined.
// CodeEditor.ts of codepair
...
import { ClientEventType, RichText } from 'yorkie-js-sdk';
console.log(RichText);
console.log(ClientEventType);
...
What you expected to happen:
I expected the information of ClientEventType
to be displayed on the console.
How to reproduce it (as minimally and precisely as possible):
I pushed to the export-event-type-undefined branch so I could run this test right away.
ClientEventType
from yorkie.ts.// yorkie.ts
...
import { Client, ClientOptions, ClientEventType } from './core/client';
import { Document, DocEventType } from './document/document';
export { Client, Document, ClientEventType, DocEventType };
...
// package.json of codepair
"dependencies": {
...
"yorkie-js-sdk": "yorkie-team/yorkie-js-sdk#export-event-type-undefined",
...
}
ClientEventType
from CodeEditor.tsx
of codepair. And run.// CodeEditor.tsx of codepair
...
import { ClientEventType } from 'yorkie-js-sdk';
console.log(ClientEventType);
...
Anything else we need to know?:
I did some tests to see how to solve this problem. And I actually solved it.
You can see what the export-event-type branch did.
The solution is not to build using webpack, but build using tsc. When building with tsc, js files for each package are created instead of a single js file.
And this really worked.
I don't know exactly why this result was. We should consider using tsc if the library built with webpack doesn't improve to import other objects.
Environment:
yorkie version
): 0.1.1What happened:
Fail 'Can watch documents' test.
HeadlessChrome 84.0.4147 (Mac OS X 10.15.6) Yorkie Can watch documents FAILED
AssertionError: expected '{}' to equal '{"k1":"v1"}'
at Context.<anonymous> (test/webpack:/test/yorkie_test.ts:163:12)
What you expected to happen:
How to reproduce it (as minimally and precisely as possible):
Run script npm run test
or npm run test:watch
Anything else we need to know?:
Environment:
yorkie version
): 0.0.10Not long ago we introduced the Garbage Collection feature in Yorkie.
And we implemented Document.GarbageCollect in Go.
We also need to implement Document.GarbageCollect
function in JS SDK. Then we need to add logic to execute GarbageCollect by receiving the MinSyncedTicket in PushPull's response.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.