stefanw / channels-yroom Goto Github PK

View Code? Open in Web Editor NEW

12.0 2.0 5.0 137 KB

Django Channels WebSocket consumer and worker for synchronizing Yjs clients

Home Page: https://channels-yroom.readthedocs.io/en/latest/

License: MIT License

Python 84.04% Dockerfile 0.65% JavaScript 12.93% CSS 0.40% HTML 1.67% Shell 0.13% Procfile 0.17%

channels django websocket yjs yjs-provider

channels-yroom's People

Contributors

Stargazers

Watchers

Forkers

linspw jannkleen its-dart atheler sorokinvld

channels-yroom's Issues

Defect: parent points to a block which is not a shared type

Thank you for this package, definitely helped me get TipTap/YJS working with my Django app.

I seemed to have come across an error where it seems as though a corrupted document state is being persisted causing both the loading of the document to not work as well as the yroom worker to crash.

Below is the relevant stacktrace:

 app[rtc.1]:  [INFO] yroom.roomsync: Creating new YRoom 'sequence.2868' with data and settings YRoomSettings { protocol_version: V1, name_prefix: true, server_start_sync: false, disable_pipelining: true }
 app[rtc.1]: thread '<unnamed>' panicked at 'Defect: parent points to a block which is not a shared type', /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/yrs-0.16.10/src/block.rs:1492:30
 app[rtc.1]: note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
 app[rtc.1]:   File "/app/.heroku/python/lib/python3.10/site-packages/channels_yroom/worker.py", line 80, in run_consumer
 app[rtc.1]:     await self.consumer.dispatch(message)
 app[rtc.1]:   File "/app/.heroku/python/lib/python3.10/site-packages/channels/consumer.py", line 73, in dispatch
 app[rtc.1]:     await handler(message)
 app[rtc.1]:   File "/app/.heroku/python/lib/python3.10/site-packages/channels_yroom/channel.py", line 71, in message
 app[rtc.1]:     await self.create_room_from_snapshot(room_name, conn_id)
 app[rtc.1]:   File "/app/.heroku/python/lib/python3.10/site-packages/channels_yroom/channel.py", line 61, in create_room_from_snapshot
 app[rtc.1]:     return self.room_manager.connect_with_data(room_name, conn_id, snapshot)
 app[rtc.1]: 2024-03-08 14:52:08,208 [ERROR] channels_yroom.worker: Caught exception: Defect: parent points to a block which is not a shared type
 app[rtc.1]: 2024-03-08 14:52:08,409 [INFO] channels_yroom.worker: Shutting down...

I don't believe there is a way to recover corrupted documents from the saved bytes data?

The only relevant link I found was: y-crdt/ypy#101

Which seems to be referencing a much older version of packages and was from a few years back?

Any ideas that you may have would be appreciated!

Thank you

For some reason, after a while the worker crashes and/or it crashes after some error.

I noticed that sometimes this happens, because there was an error, and the worker stopped responding after this error.

For example:

class YDocUpdateManager(models.Manager):
    def get_snapshot(self, name):
        try:
            data = self.get(name=name).data
            return bytes(None) # <<<<<<<
        except YDocUpdate.DoesNotExist:
            return None

    def save_snapshot(self, name, data):
        return self.update_or_create(name=name, defaults={"data": data})

If the return of the snapshot has something wrong (example: byte(None)) it breaks silently.
And the application stops responding to other WS requests.

This also happens when the active worker spends some time, I imagine that it is an error that happened and it is holding the response due to asyncronism.

Note: I already adjusted for the new update

I've been following each Update and I'm finding your project amazing. Thank you for your attention and initiative!

For some reason the YDoc sometimes remains unsynchronized when loading the page

Gravacao.de.tela.de.04-05-2023.20.14.14.webm

This happens on half the tries. It does not generate errors in the console.
Maybe it could be a concurrency issue. I've looked at the snapshot data and it's correct.

My Workaround is force sync again (works with it) 🤔

    const ydoc = new Y.Doc()
    this.provider = new HocuspocusProvider({
      url: `ws://localhost:8000/workspace/v1/documents/${this.document.id}`,
      name: String(this.document.id),
      document: ydoc,
      onSynced(data) {
        console.log(data)
      },
    })
    this.provider.on('connect', () => {
      this.provider?.forceSync()
    })

    this.editor = new Editor({
      extensions: [
        StarterKit.configure({
          history: false,
        }),
        Collaboration.configure({
          document: ydoc,
          field: 'content',
        }),
        CollaborationCursor.configure({
          provider: this.provider,
          user: { name: this.getRandomName(), color: this.getRandomColor() },
        }),
      ],
    }

Thanks in advance for your attention, Stefan!

Your project is amazing!

The default name room is not being used

Seems like a pretty simple setup to me. 😄
This error appears in the new version, and I confirmed it with commits and debbug

Error:

root@947fc0db06e9:/application# python manage.py yroom
--- Logging error ---
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/logging/__init__.py", line 1110, in emit
    msg = self.format(record)
          ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/logging/__init__.py", line 953, in format
    return fmt.format(record)
           ^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/logging/__init__.py", line 687, in format
    record.message = record.getMessage()
                     ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/logging/__init__.py", line 377, in getMessage
    msg = msg % self.args
          ~~~~^~~~~~~~~~~
TypeError: not all arguments converted during string formatting
Call stack:
  File "/application/manage.py", line 22, in <module>
    main()
  File "/application/manage.py", line 18, in main
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python3.11/site-packages/django/core/management/__init__.py", line 419, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python3.11/site-packages/django/core/management/__init__.py", line 413, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python3.11/site-packages/django/core/management/base.py", line 354, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/local/lib/python3.11/site-packages/django/core/management/base.py", line 398, in execute
    output = self.handle(*args, **options)
  File "/usr/local/lib/python3.11/site-packages/channels_yroom/management/commands/yroom.py", line 50, in handle
    logger.info("Running worker for channel '{}'", channel)
Message: "Running worker for channel '{}'"
Arguments: (None,)
Task exception was never retrieved
future: <Task finished name='Task-1' coro=<YroomWorker.run_worker() done, defined at /usr/local/lib/python3.11/site-packages/channels_yroom/worker.py:44> exception=TypeError('Channel name must be a valid unicode string with length < 100 containing only ASCII alphanumerics, hyphens, underscores, or periods, not None')>
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/channels_yroom/worker.py", line 58, in run_worker
    message = await self.channel_layer.receive(self.channel)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/channels_redis/core.py", line 266, in receive
    assert self.valid_channel_name(channel)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/channels/layers.py", line 160, in valid_channel_name
    raise TypeError(self.invalid_name_error.format("Channel", name))
TypeError: Channel name must be a valid unicode string with length < 100 containing only ASCII alphanumerics, hyphens, underscores, or periods, not None

Debug:

Gravacao.de.tela.de.04-05-2023.09.15.55.webm

OBS:
I tried to use the command with the --channel configuration but it gave parsing error

Improve persistence with incremental updates

Currently, YDocs are in-memory and persisted only when the room is empty or the worker shuts down.

Here are some other Yjs persistence implementations:

A common theme is writing incremental updates and consolidating later.

Maybe creating an implementation for yrs-persistence in yroom might be the way to go.

Is there any way to edit YDocs directly instead of using YRoomDocument?

Hello Stefan, how are you?

Is there any way to edit YDocs directly instead of using YRoomDocument export?

The purpose of this is to be able to handle YDocs directly the way they are on the server side.

Error when taking snapshot

This erros occurs in first room created

stefanw / channels-yroom Goto Github PK

channels-yroom's People

Contributors

Stargazers

Watchers

Forkers

channels-yroom's Issues

Defect: parent points to a block which is not a shared type

For some reason, after a while the worker crashes and/or it crashes after some error.

For some reason the YDoc sometimes remains unsynchronized when loading the page

The default name room is not being used

Improve persistence with incremental updates

Is there any way to edit YDocs directly instead of using YRoomDocument?

Error when taking snapshot

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent