reactive-python / reactpy-django Goto Github PK

Current Situation

Currently commented out component tags will attempt to be parsed by our component regex.

For example, this should not be parsed <!-- {% component "my.component" %} -->

Proposed Actions

Modify the regex to avoid any tags within comments.

As a quick test, I attempted to see what would happen if I prefixed the component regex with a negative lookbehind (?<!(<!--))\s*, however, it did not work for any tags containing a space between the comment start <!-- and tag start {%

Additionally, I'm not entirely how to write regex for situations like <!-- {% component "my.component" %} --> while not hitting false negatives for things such as <!-- a comment --> {% component "my.component" %} <!-- another comment -->

If a regex solution can't be made, a quick and dirty solution could be running some kind of HTML minifier to strip out comment tags prior to reading the template HTML.

Or a running a simple regex replace on all comments prior to parsing.

IDOM_WEB_MODULES_DIR contains JS modules that are dynamically added at runtime. Since JS files are typically loaded statically, it would be nice if we could improve how quickly these files can be accessed. There are three ways to do that:

• Implement a management command for collecting these static files ahead of time.
• Make the view accessing the files async to avoid blocking while waiting on the file system
• Cache this view using Django's cache framework.

Notes

To implement the management command we could trace over all the templates and look for idom_component template tag nodes in order to figure out what modules to import so that we can trigger the collection of the JS files into IDOM_WEB_MODULES_DIR. Then once they've been collected those files can be copied into a permanent static location.

To do this we can take inspiration from the django compressor library which takes a similar approach. See the COMPRESS_OFFLINE setting to figure out how they're doing it. Also the compressor.offiline module contains uniform interfaces for parsing template syntax for Django and Jinja that we might want to copy (with the latest LICENSE, what to do about AUTHORS file?).

Conversation originated from: this comment

Old Behavior

State is generated on a per-render basis and has no method of being permanently stored.

New Behavior

Allow for persistent state that is databased backed, linked to a User.

Implementation Details

Persistently store/restore state within a database model, linked to a User as a foreign key.

Store context data within a BinaryField.

This will require only serializable data will be allowed within this. Consider using dill.pickle to serialize data more robustly than standard library.

The state should only be fetched from the database once, upon first initialization of the hook's state value. On every set_state call, the value should be synchronized to the database.

We may also want to implement an expires_at parameter to decide when the value gets evicted from the database.

Example Database Model

class IdomUserState(models.Model):
    # One state is stored per user
    # When the user is deleted, the state is also deleted from the database
    user = models.ForeignKey(User, on_delete=models.CASCADE)
    state = models.BinaryField(blank=True, null=True)

The interface may look like this...

# Note: The default state is only used in situations where there is no value in the database
# The websocket is needed in order to grab the user from the `scope`
state, set_state = hooks.use_user_state({"foo":"bar"})
state = {"something":"else"}

# This saves the values to the database
set_state(state)

In the background, the set state is doing the following...

def set_state(state, websocket):
    query = IdomUserState.objects.get_or_create(user=websocket.scope.user)
    query.state = dill.pickle(state)
    query.save()
    ...

Code of Conduct

• I agree to follow the Code of Conduct.

Current Situation

django-idom has been renamed to reactpy-django. We need to warn existing users about this change.

Proposed Actions

Add a print/log statement to src/django_idom/__init__.py to tell users about the rename.

We need to create a new branch based off of commit b1ff783c887bbd7d93fdf586d21555692b502dcb in order create new django-idom releases.

Since one can use the same component more than once we should avoid calling _register_component more than once if it's already been done. A simple containment check in IDOM_REGISTERED_COMPONENTS should work here.

Current Situation

A file was accidentally committed to the repo.

Proposed Actions

Remove it.

Current Situation

Currently, use_query will generate an exception of a Model or QuerySet isn't returned. This makes it incompatible with external ORMs such as SQLAlchemy or encode/orm

Proposed Actions

Create some kind of interface to remove these restrictions. Needs to be able to handle our current stuff as well, so maybe add two args to the hook:

fetch_recursive_classes=[QuerySet]
fetch_classes=[Model]

Argument variable names might need some more thought.

We're nearly ready for a 0.0.2 release. We need a changelog though to document the work that's been done since 0.0.1.

Old Behavior

Currently, we output our JS to static/js/django-idom-client.js, but there is a standardized pattern for pip installable Django apps to store their static files within a folder named after the package.

New Behavior

Output JS to static/django-idom/client.js

Implementation Details

See above.

Code of Conduct

• I agree to follow the Code of Conduct.

Old Behavior

Currently no workflow exists

New Behavior

Create a new workflow.

Implementation Details

https://github.com/idom-team/idom/blob/main/.github/workflows/publish-js.yml
https://github.com/idom-team/idom/blob/main/.github/workflows/publish-py.yml

Code of Conduct

• I agree to follow the Code of Conduct.

Current Situation

Currently, the Django ORM will shoot out a SynchronousOnlyOperation if used within a IDOM component. This is caused by Django's current limitation surrounding mixed sync-async contexts.

Proposed Actions

Choose one of the following

1. Create a React-style hooks to perform ORM CRUD
2. Set DJANGO_ALLOW_ASYNC_UNSAFE when within a component rendering context.
3. Create a hook that utilizes database_sync_to_async that can perform ORM queries

There's a high chance we will use our cache backend for more than just web modules in the future. It's best to rename the backend now while there still aren't many django-idom users yet.

Current Situation

There currently isn't a native way to cache or defer execution of a use_query call.

Proposed Actions

Mimic behavior of apollo's useQuery fetch policies within our use_query hook.

The only parameters that make sense to carry over are:

• cache-first
• cache-only
• no-cache

Current Situation

Right now, we don't do any type checking with MyPy. We also do not distribute a py.typed marker file which would allow MyPy to work with django_idom when it's installed in other projects.

Proposed Actions

• Add type checking with mypy. Preferably we'd eventually check in --strict mode, but we don't have to start there.
• Distribute a py.typed marker file.

Old Behavior

State is generated on a per-render basis and has no method of being permanently stored.

Additionally, there is no convenient way to perform DB queries within components (as of Django 4.1)

New Behavior

Allow for persistent state that is databased backed.

Implementation Details

Persistently store/restore context based on a database model.

The interface should take in a single database object...

from my_django_app.models import GeneralSettings

state, set_state = hooks.use_database_state(GeneralSettings.objects.get(id=20))
state.my_database_field = True

# This saves the values to the database and issues a re-render.
set_state(state)

In the background, the set state is doing the following...

def set_state(state):
    state.save()
    ...

Code of Conduct

• I agree to follow the Code of Conduct.

Old Behavior

======================================================================
ERROR: test_component_from_web_module (test_app.tests.TestIdomCapabilities)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\username\Documents\Repositories\django-idom\.venv\lib\site-packages\django\test\testcases.py", line 272, in _setup_and_call
    self._pre_setup()
  File "C:\Users\username\Documents\Repositories\django-idom\.venv\lib\site-packages\channels\testing\live.py", line 52, in _pre_setup
    self._server_process.start()
  File "C:\Users\username\AppData\Local\Programs\Python\Python39\lib\multiprocessing\process.py", line 121, in start
    self._popen = self._Popen(self)
  File "C:\Users\username\AppData\Local\Programs\Python\Python39\lib\multiprocessing\context.py", line 224, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "C:\Users\username\AppData\Local\Programs\Python\Python39\lib\multiprocessing\context.py", line 327, in _Popen
    return Popen(process_obj)
  File "C:\Users\username\AppData\Local\Programs\Python\Python39\lib\multiprocessing\popen_spawn_win32.py", line 93, in __init__
    reduction.dump(process_obj, to_child)
  File "C:\Users\username\AppData\Local\Programs\Python\Python39\lib\multiprocessing\reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)
AttributeError: Can't pickle local object 'convert_exception_to_response.<locals>.inner'    

======================================================================
ERROR: test_counter (test_app.tests.TestIdomCapabilities)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\username\Documents\Repositories\django-idom\.venv\lib\site-packages\django\test\testcases.py", line 272, in _setup_and_call
    self._pre_setup()
  File "C:\Users\username\Documents\Repositories\django-idom\.venv\lib\site-packages\channels\testing\live.py", line 52, in _pre_setup
    self._server_process.start()
  File "C:\Users\username\AppData\Local\Programs\Python\Python39\lib\multiprocessing\process.py", line 121, in start
    self._popen = self._Popen(self)
  File "C:\Users\username\AppData\Local\Programs\Python\Python39\lib\multiprocessing\context.py", line 224, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "C:\Users\username\AppData\Local\Programs\Python\Python39\lib\multiprocessing\context.py", line 327, in _Popen
    return Popen(process_obj)
  File "C:\Users\username\AppData\Local\Programs\Python\Python39\lib\multiprocessing\popen_spawn_win32.py", line 93, in __init__
    reduction.dump(process_obj, to_child)
  File "C:\Users\username\AppData\Local\Programs\Python\Python39\lib\multiprocessing\reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)
AttributeError: Can't pickle local object 'convert_exception_to_response.<locals>.inner'    

======================================================================
ERROR: test_hello_world (test_app.tests.TestIdomCapabilities)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\username\Documents\Repositories\django-idom\.venv\lib\site-packages\django\test\testcases.py", line 272, in _setup_and_call
    self._pre_setup()
  File "C:\Users\username\Documents\Repositories\django-idom\.venv\lib\site-packages\channels\testing\live.py", line 52, in _pre_setup
    self._server_process.start()
  File "C:\Users\username\AppData\Local\Programs\Python\Python39\lib\multiprocessing\process.py", line 121, in start
    self._popen = self._Popen(self)
  File "C:\Users\username\AppData\Local\Programs\Python\Python39\lib\multiprocessing\context.py", line 224, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "C:\Users\username\AppData\Local\Programs\Python\Python39\lib\multiprocessing\context.py", line 327, in _Popen
    return Popen(process_obj)
  File "C:\Users\username\AppData\Local\Programs\Python\Python39\lib\multiprocessing\popen_spawn_win32.py", line 93, in __init__
    reduction.dump(process_obj, to_child)
  File "C:\Users\username\AppData\Local\Programs\Python\Python39\lib\multiprocessing\reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)
AttributeError: Can't pickle local object 'convert_exception_to_response.<locals>.inner'    

======================================================================
ERROR: test_parametrized_component (test_app.tests.TestIdomCapabilities)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\username\Documents\Repositories\django-idom\.venv\lib\site-packages\django\test\testcases.py", line 272, in _setup_and_call
    self._pre_setup()
  File "C:\Users\username\Documents\Repositories\django-idom\.venv\lib\site-packages\channels\testing\live.py", line 52, in _pre_setup
    self._server_process.start()
  File "C:\Users\username\AppData\Local\Programs\Python\Python39\lib\multiprocessing\process.py", line 121, in start
    self._popen = self._Popen(self)
  File "C:\Users\username\AppData\Local\Programs\Python\Python39\lib\multiprocessing\context.py", line 224, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "C:\Users\username\AppData\Local\Programs\Python\Python39\lib\multiprocessing\context.py", line 327, in _Popen
    return Popen(process_obj)
  File "C:\Users\username\AppData\Local\Programs\Python\Python39\lib\multiprocessing\popen_spawn_win32.py", line 93, in __init__
    reduction.dump(process_obj, to_child)
  File "C:\Users\username\AppData\Local\Programs\Python\Python39\lib\multiprocessing\reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)
AttributeError: Can't pickle local object 'convert_exception_to_response.<locals>.inner'    

----------------------------------------------------------------------
Ran 0 tests in 0.027s

FAILED (errors=4)
Destroying test database for alias 'default'...
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Users\username\AppData\Local\Programs\Python\Python39\lib\multiprocessing\spawn.py", line 109, in spawn_main
Traceback (most recent call last):
Traceback (most recent call last):
  File "<string>", line 1, in <module>
    fd = msvcrt.open_osfhandle(new_handle, os.O_RDONLY)
  File "<string>", line 1, in <module>
OSError: [Errno 9] Bad file descriptor
  File "C:\Users\username\AppData\Local\Programs\Python\Python39\lib\multiprocessing\spawn.py", line 109, in spawn_main
  File "C:\Users\username\AppData\Local\Programs\Python\Python39\lib\multiprocessing\spawn.py", line 109, in spawn_main
    fd = msvcrt.open_osfhandle(new_handle, os.O_RDONLY)
OSError: [Errno 9] Bad file descriptor
    fd = msvcrt.open_osfhandle(new_handle, os.O_RDONLY)
OSError: [Errno 9] Bad file descriptor
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Users\username\AppData\Local\Programs\Python\Python39\lib\multiprocessing\spawn.py", line 107, in spawn_main
    new_handle = reduction.duplicate(pipe_handle,
  File "C:\Users\username\AppData\Local\Programs\Python\Python39\lib\multiprocessing\reduction.py", line 79, in duplicate
    return _winapi.DuplicateHandle(
OSError: [WinError 6] The handle is invalid

New Behavior

Tests should properly execute without pickling errors on Windows.

Implementation Details

Should be resolved within django-channels.

Issue is being tracked on django/channels#1207

Tested versions:

Django == 3.2.4 - 3.2.7
Channels == 3.0.0 - 3.0.4
asgiref == 3.3.0 - 3.4.1
idom == any
django-idom == any

Code of Conduct

• I agree to follow the Code of Conduct.

Old Behavior

When executing the standard build procedure, the following error is given in console

Uncaught TypeError: Failed to resolve module specifier "react". Relative references must start with either "/", "./", or "../".

New Behavior

Unbork django-idom builds that are created on Windows machines.

Implementation Details

No response

Code of Conduct

• I agree to follow the Code of Conduct.

Configuration file should be set up to automatically enable some settings, such as EOF newlines and formatting on save.

Additionally, we should add a list of suggested extensions to improve development workflow.

Current Situation

There are a number of usability problems with use_query:

1. Prefetch logic is not perfect and can have significant performance costs: #110
2. The interface does not lend itself to future extensions: #104, #103
3. And that, when a user accesses a field that has not been pre-fetched in the scope of a render function, the SynchronousOnlyOperation error they get from Django does not communicate this fact and how to resolve it.

Proposed Actions

Discuss and arrive at a solution to each of the above problems. I'll update this section as we reach consensus on how to proceed.

This already exists within Conreq, but needs some work to be ported over to Django-IDOM

Old Behavior

State is generated on a per-render basis and has no method of being permanently stored.

New Behavior

Allow for persistent state that is databased backed, linked to a user's Session.

This will only function when django.contrib.sessions is in INSTALLED_APPS

Implementation Details

Persistently store/restore state within a database model, linked to a Session as a foreign key.

Store context data within a BinaryField.

This will require only serializable data to be used within this. Consider using dill.pickle to serialize data more robustly than standard library.

The state should only be fetched from the database once, upon first initialization of the hook's state value. On every set_state call, the value should be synchronized to the database.

We may want to document that the user should regularly purge old sessions.

This implementation will require the use of database-backed sessions or django.contrib.sessions.backends.cached_db.

Example Database Model

class IdomSessionState(models.Model):
    # One state is stored per user browser session
    # When the browser session expires, the state is deleted from the database
    session = models.ForeignKey(Session, on_delete=models.CASCADE)
    state = models.BinaryField(blank=True, null=True)

The interface may look like this...

# Note: The default state is only used in situations where there is no value in the database
# The websocket is needed in order to grab the session from the `scope`
state, set_state = hooks.use_session_state({"foo":"bar"}, websocket=websocket)
print(state.data)

# This saves the values to the database
set_state({"something":"else"})

In the background, the set state is doing the following...

def set_state(state, websocket):
    query = IdomSessionState.objects.get_or_create(session=websocket.scope.user.session)
    query.state = dill.pickle(state)
    query.save()
    ...

Code of Conduct

• I agree to follow the Code of Conduct.

Current Situation

Currently, we serve our JavaScript web modules (JavaScript Components) via views. This is due to the fact that our current architecture does not let us know what web modules exist until runtime.

However, Django views are less efficient at serving files than a webserver such as Nginx or Apache.

Proposed Actions

Develop a way of integrating with the Django Static File system.

Work Items

We have a couple of paths we could take

1. Create a static file finder that points to ReactPy's web modules directory
   • This is the preferred implementation if feasible
2. Develop a collect_modules command, similar to django-compressor's collect_static command
   • This option requires changes in ReactPy to deterministically collect all JavaScript web modules during startup.
   • Also, what should we do if we accidentally did not obtain a static file during startup? Should we fallback to using our Django view, or just throw a 500: Internal Server Error?
3. Make users put JavaScript modules in dedicated JAVASCRIPT_MODULES = [ ... ] folder(s)
   • User's will need to statically define their ReactPy JavaScript modules with this method.

See #4 for the original issue.

Current Situation

ManyToMany fields are not properly prefetched, thus causing SynchronousOnlyOperation exceptions.

For example, in this case.

Proposed Actions

Determine why the current prefetch logic doesn't work on this use case.

Discussed in #121

parent = models.ForeignKey(Module, ..., related_name="compiled")

prefetch_fields.append(f"{field.name}_set")

prefetch_fields.append(field.related_name or f"{field.name}_set")

Old Behavior

Currently, IDOM relies on synchronous code to properly queue renders. This code blocks the asyncio event loop, causing massive concurrency issues.

See section "Two Worlds of Python" for why this is a problem:
https://arunrocks.com/a-guide-to-asgi-in-django-30-and-its-performance/

New Behavior

Rewrite parts of IDOM core to do everything using asyncio.

Implementation Details

Anything that relies on order of operations should be executed in a FIFO queue.

If synchronous code is required, it should be run in a thread in order to avoid blocking asyncio event loops.

All synchronous functions should be converted to async within IDOM core.

I had previously attempted threading the django-idom dispatcher as a quick fix, but that did not yield worthwhile performance benefits, likely due to context switching.

Code of Conduct

• I agree to follow the Code of Conduct.

Current Situation

In the "Get Started" section, Django applications must go before "reactpy_django" and the ellipsis (...) in "INSTALLED_APPS" are after "reactpy_django" which causes problems when following that installation step.

Since then running the django manager does not work.

Proposed Actions

Change the order in which ellipses are displayed in the "INSTALLED_APPS" section of "Get Started"

Current Situation

Currently we use Selenium to run our tests, but the core project has switched to using Playwright. The setup for Playwright is much more straightforward because it includes an automatic installer for the webdriver (just one playwright install command). For Selenium you have to manually install it yourself.

Proposed Actions

Rewrite the tests using Playwrite based on this example.

Work Items

Rewrite the tests.

Current Situation

Currently, args/kwargs are serialized and stored in client-sided HTML. However, this is susceptible to spoofing, and makes it impossible to pass in non-serializable values such as Python objects.

Proposed Actions

During a HTTP request...

1. Check if the component's signature contains any args/kwargs
   • Exit here if no args/kwargs exist
2. Store the args/kwargs into a container (dataclass)
3. Serialize this container into a byte string using dill.dumps.
4. Store this byte string in a uniquely identifiable way.
   • This will be locked down to a specific component uuid.

During a WS component load...

1. Check if the component's signature contains any args/kwargs
   • Render the component without args/kwargs if none exist
2. Retrieve the bytes string from the database using uuid and/or session identifier.
3. Deserialize the data back into an object using dill.loads
4. Render the component using the deserialized args/kwargs

Now this brings up a question about data retention. Ensuring automatic deletion of data is going to be fairly important, and this is a common issue with things such as Django's database-backed sessions model.

Given that, we will need to store a expiration_date value within the ComponentParams database model. In that scenario, we have three options we can pick-and-match from.

Automatic deletion of expired sessions on...

1. Django Start-up
   • I recommend we implement this regardless
2. Periodic Schedule (background task)
   • If technologically feasible without external dependencies, we should do this.
   • Django Channels supports background worker processes, but I'll need to dig into whether we can programmatically start it.
   • My guess is this won't be technologically feasible without mandating users to run a separate python manage.py runworker command alongside their webserver, which doesn't feel clean/simplistic.
   • Maybe we can implement this as an "alternative option" for users willing to go through the hassle of manually running a background worker?
3. WS connection
   • Doing a DB operation on every WS connection would be cumbersome and overkill
   • However, it's possible for us to rate-limit this by storing last-run timestamps within cache, and only re-running if we hit some configured time threshold.

Current Situation

Currently, our Python APIs are manually added to the docs, which can potentially result in API <-> Docs mismatch.

Proposed Actions

Use mkdocstrings within the docs to automatically generate markdown based on Python code, type hints, and docstrings.

Implementing this is currently blocked by these issues:

• mkdocstrings/mkdocstrings#450
• mkdocstrings/mkdocstrings#466

Current Situation

There is currently has no support for conveniently rendering tables from Django querysets.

Proposed Actions

Create a table rendering component. The interface can be developed a few ways:

1. Inspired by Material UI Table
2. Inspired by Data Grid
3. Inspired by TanStack Table
4. Inspired by some other ReactJS table library
5. Compatible with Django-Tables2 interface.
   • This option is iffy, since it would be difficult to attach ReactPy event handlers to things.

Draft Interface

# https://mui.com/x/api/data-grid/data-grid-pro/
django_table(
    {"example-prop-value": 123},
    query_set=ExampleQuerySet(),
    columns=[
        {
            "name": "ID",
            "header": "ID",
            "cell": lambda x: None,
        },
        {
            "name": "Name",
            "header": "Name",
            "cell": lambda x: None,
        },
        {
            "name": "Age",
            "header": "Age",
            "cell": lambda x: None,
        },
    ],
    checkbox_selection=True,
    checkbox_selection_visible_only=False,
    classes="example",
    edit_mode="cell" or "row",
    filter_debounce_ms=150,
    # get_cell_class_name=lambda x: None,
    # get_detail_panel_content=lambda x: None,
    # get_row_class_name=lambda x: None,
    # get_row_id=lambda x: None,
    hide_footer=True,
    initial_state=None,
    # is_cell_editable=lambda x: None,
    # is_group_expanded_by_default=lambda x: None,
    # is_row_selectable=lambda x: None,
    loading=True,
    page_size_options=[10, 25, 50, 100],
    row_selection=True,
    slots_props={},
    slots={},
    sorting_order="asc" or "desc",
    # pinned_rows={"left": [], "right": []},
    # pinned_columns={"top": [], "bottom": []},
    # row_reordering=False,
    on_cell_click=lambda x: None,
    on_cell_double_click=lambda x: None,
    on_cell_edit_start=lambda x: None,
    on_cell_edit_stop=lambda x: None,
    on_cell_key_down=lambda x: None,
    on_column_header_click=lambda x: None,
    on_column_header_double_click=lambda x: None,
    on_column_header_enter=lambda x: None,
    on_column_header_leave=lambda x: None,
    on_column_header_out=lambda x: None,
    on_column_header_over=lambda x: None,
    on_column_order_change=lambda x: None,
    on_filter_model_change=lambda x: None,
    on_menu_close=lambda x: None,
    on_menu_open=lambda x: None,
    on_preference_panel_close=lambda x: None,
    on_preference_panel_open=lambda x: None,
    on_process_row_update_error=lambda x: None,
    on_row_click=lambda x: None,
    on_row_double_click=lambda x: None,
    on_row_edit_commit=lambda x: None,
    on_row_edit_start=lambda x: None,
    on_row_edit_stop=lambda x: None,
    on_cell_modes_model_change=lambda x: None,
    on_pagination_model_change=lambda x: None,
    on_row_modes_model_change=lambda x: None,
    on_row_selection_model_change=lambda x: None,
    on_sort_model_change=lambda x: None,
    pagination_model=None,
    row_modes_model=None,
    row_selection_model=None,
    sort_model=None,
    # disable_children_filter=False,
    # disable_children_sorting=False,
    # disable_column_filter=False,
    # disable_column_menu=False,
    # disable_column_pinning=False,
    # disable_column_reorder=False,
    # disable_column_resize=False,
    # disable_column_selector=False,
    # disable_density_selector=False,
    # disable_multiple_columns_filtering=False,
    # disable_multiple_columns_sorting=False,
    # disable_multiple_row_selection=False,
    # disable_row_selection_on_click=False,
)

This is just a reminder for myself to do it.

Old Behavior

Currently there is no way of sharing component states across multiple clients.

Proposed Actions

Add in the ability for arbitrary data to be synchronized cross-client.

We will use Django Channels layers to communicate a state across multiple clients. In order to use this feature, the user will need to first configure a Django Channels Layer with a backend such as redis.

The user interface might end up looking like this:

@component
def example():
    sender = use_channel_layers(receiver, channel_name="example", group_name="example")

    return ...

async def receiver(event: dict):
    ...

Code of Conduct

• I agree to follow the Code of Conduct.

Old Behavior

Currently, Django IDOM has no official documentation outside of the readme. As we continue to add features to Django IDOM, we'll need more than just a readme to encompass all our features.

New Behavior

Create new documentation.

Implementation Details

We should use the development of any one of our custom Django integration features as an excuse to create new docs.

• #41
• #40
• #39
• #38
• #33
• #32

Also, these docs should contain tidbits such as

• Name parameter on idom_component templatetags
• Explicitly state that we require a websocket parameter, which is different than IDOM core
• Installation Instructions
• Developer's Guide

Code of Conduct

• I agree to follow the Code of Conduct.

Current Situation

There is no easy way to prevent components from rendering based on Django authentication criteria

Proposed Actions

Copy over the authenticated decorator from Conreq.

Work Items

No response

Old Behavior

WebSocket HANDSHAKING /_idom/websocket/test_app.components.ParametrizedComponent/ [127.0.0.1:61944]
WebSocket CONNECT /_idom/websocket/test_app.components.ParametrizedComponent/ [127.0.0.1:61944]
WebSocket DISCONNECT /_idom/websocket/test_app.components.ParametrizedComponent/ [127.0.0.1:61944]
WebSocket HANDSHAKING /_idom/websocket/test_app.components.ParametrizedComponent/ [127.0.0.1:61945]
WebSocket CONNECT /_idom/websocket/test_app.components.ParametrizedComponent/ [127.0.0.1:61945]
WebSocket DISCONNECT /_idom/websocket/test_app.components.ParametrizedComponent/ [127.0.0.1:61945]
WebSocket HANDSHAKING /_idom/websocket/test_app.components.SimpleBarChart/ [127.0.0.1:61953]
WebSocket CONNECT /_idom/websocket/test_app.components.SimpleBarChart/ [127.0.0.1:61953]
WebSocket DISCONNECT /_idom/websocket/test_app.components.SimpleBarChart/ [127.0.0.1:61953]
WebSocket HANDSHAKING /_idom/websocket/test_app.components.Button/ [127.0.0.1:61955]
WebSocket CONNECT /_idom/websocket/test_app.components.Button/ [127.0.0.1:61955]
WebSocket DISCONNECT /_idom/websocket/test_app.components.Button/ [127.0.0.1:61955]
WebSocket HANDSHAKING /_idom/websocket/test_app.components.HelloWorld/ [127.0.0.1:61956]
WebSocket CONNECT /_idom/websocket/test_app.components.HelloWorld/ [127.0.0.1:61956]
WebSocket DISCONNECT /_idom/websocket/test_app.components.HelloWorld/ [127.0.0.1:61956]
WebSocket HANDSHAKING /_idom/websocket/test_app.components.ParametrizedComponent/ [127.0.0.1:61957]
WebSocket CONNECT /_idom/websocket/test_app.components.ParametrizedComponent/ [127.0.0.1:61957]
WebSocket DISCONNECT /_idom/websocket/test_app.components.ParametrizedComponent/ [127.0.0.1:61957]
HTTP GET /_idom/web_module/from-template/victory-bar.js 200 [0.01, 127.0.0.1:64500]

I'm really not sure how to trigger this. Seems to happen sometimes while using the development webserver with autoreload (settings.py:DEBUG=True)

When this occurs, the websocket will be stuck in a perpetual handshake/connect loop for a minute or two. Happens the same within Conreq, and I'm not using any external web modules over there yet.

Somehow, using a keyboard interrupt (CTRL+C) to attempt to escape the situation does not resolve the bug. Will still be stuck in this infinite connection loop upon restarting the development webserver (manage.py runserver)

New Behavior

WS connection and render should be performed smoothly

Implementation Details

Need to debug

Code of Conduct

• I agree to follow the Code of Conduct.

Current Situation

Currently, sites built in IDOM are not SEO compatible. This is a fairly common issue with JavaScript frameworks such as ReactJS.

This might ultimately relate to persistent components (#34).

Proposed Actions

To resolve this, there needs to be an initial HTTP render, followed by a ReactPy re-render. The best way of doing this requires some form of persistent storage of all hook states, due to the fact that ASGI websockets are a completely different stack than HTTP rendering. Hook states will need to be stored server side in order to prevent spoofing. We likely have to use a database.

1. Use the template tag to render the initial component as raw HTML
2. Serialize all the component's hook values (probably through dill.pickle) and
   • This might need exposing some new hook APIs in core that provides returns hook values for a given component instance.
3. Store serialized hook values within the database
   • Use the component's UUID as the database ID.
4. When the JavaScript client requests it, rehydrate the components hook values
5. Execute the ReactJS rendering
   • reactive-python/reactpy#760 will need to be resolve prior to this
6. Delete the component hook values from the database, as they are no longer needed.

The database model might look like this:

class IdomHookState(Model):
    uuid = models.UUIDField(
        primary_key=True, default=uuid.uuid4, editable=False, unique=True
    )
    hook_attributes = models.TextField()

This design brings up a challenge of determining when to evict old hook states (for example, if a user did the initial render but for some reason never performed a websocket connection). We don't want to store everything forever, so some configurable age-based eviction strategy might be needed. Expired entries should be culled before each fetch of IdomHookState.hook_attributes. Expiration should be based on last access time, which is fairly simple to do in Django like such: last_accessed = models.DateTimeField(auto_now=True).

Current Situation

Currently, the docs only show the latest version.

Proposed Actions

Enable mkdocs versioning so that we have revision controlled docs.

See this issue on how we can create a github workflow for this.

Current Situation

This ticket is a duplicate of reactive-python/reactpy#786 to increase transparency that he issue affects both repos.

Proposed Actions

Implement changes suggested in reactive-python/reactpy#786

Old Behavior

IDOM_DEBUG_MODE needed to be manually set

New Behavior

Automatically set it based on settings.py:DEBUG

Implementation Details

IDOM_DEBUG_MODE = django.conf.settings.DEBUG

Code of Conduct

• I agree to follow the Code of Conduct.

Make a utility that returns a component given a certain path, if it has been registered.
This already exists to some degree within IDOM (for the templatetag), but needs some massaging to be a public tool.

Current Situation

Fetching subcategories in Conreq does not automatically occur with use_query

Proposed Actions

Determine why and push a fix.

Old Behavior

Currently, there is no existing way to easily utilize Django Forms (and django-crispy-forms) within ReactPy.

A lot of Django users have the expectation that they can use battle-tested form validation, so this is an important feature.

Implementation Details

Create a reactpy_django.components.django_form that calls Form.render(), then use html_to_vdom to convert into a ReactPy component.

By default, django_form should intercept the on_submit event for html.form to perform Django validation.

@component
def my_component():
    return html.div(
        django_form(MyDjangoForm),
    )

We might need to be extra attentive towards some custom form fields, such as django-colorfield and django-ace, which utilize script tags directly within the form body.

Additionally, some anchor link buttons, such as those within django-crispy-forms, may require special attention as well. For example, should we automatically use reactpy_router to intercept these links? This automatic behavior might need to be a configurable setting within the component.

Current Situation

Due to Django ORM's poor async support, use_mutation and use_query rely on database_sync_to_async with thread_sensitive=True.

This unfortunately means that only one use_query call can execute at a time. We should write the use_query hook to be natively async and support async function calls.

Proposed Actions

For async mutations and query functions:

• Have the developer handle async safety where needed

For sync mutations and query functions:

• Use thread_senstive=True by default, since sync code has a reasonable expectation of thread blocking.
• Create a QueryOptions parameter to customize the value of thread_senstive.

Discussed in #132

from typing import Callable

from django.utils.timezone import now

from django_idom.hooks import use_mutation
from idom import html, use_state, component


def work(t0, trg: Callable):
    import time
    time.sleep(10)
    trg(now() - t0)


@component
def Main():
    sleeper = use_mutation(work)
    elapsed = use_state(None)

    return html.div(
        html.button(
            {
                'on_click': (lambda e: sleeper.execute(t0=now(), trg=elapsed.set_value)),
                'class_name': "btn btn-primary",
                'disabled': sleeper.loading,
            },
            f"Elapsed: {elapsed.value}" if elapsed.value else "use_mutation"
        ),
    )

Old Behavior

Currently, the react client is set to development by default.

New Behavior

Build the react client in production mode by default, or intelligently select a client based on IDOM_DEBUG_MODE

Implementation Details

Change the const { PRODUCTION } = process.env; to default to True

Consider a method where this can be done at run-time.

See related issue reactive-python/reactpy#346

Code of Conduct

• I agree to follow the Code of Conduct.

Old Behavior

Currently, we expect the user to add IDOM_WEB_MODULE_PATH within the urlpattern, however, this is an antipattern within the Django framework

New Behavior

Follow Django conventions and utilize standard path variables.

Implementation Details

Replace instructions to utilize IDOM_WEB_MODULE_PATH with path("idom/", include("django_idom.http.urls"))

Create django_idom.http.urls that contains a urlpatterns = [...]

Anywhere within django-idom where we need to determine the URL, we need to fetch it using django reverse("idom:web_modules")

This implementation also requires that we remove IDOM_BASE_URL as a variable.

Code of Conduct

• I agree to follow the Code of Conduct.

Old Behavior

If the webpage is closed or refreshed prior to component load, an exception will occur.

2021-10-16T22:57:18-0700 | ERROR | Failed to render Button(2992041646976, websocket=WebsocketConnection(scope={'type': 'websocket', 'path': '/_idom/websocket/test_app.components.Button/', 'raw_path': b'/_idom/websocket/test_app.components.Button/', 'headers': [(b'host', b'127.0.0.1:8000'), (b'connection', b'Upgrade'), (b'pragma', b'no-cache'), (b'cache-control', b'no-cache'), (b'upgrade', b'websocket'), (b'origin', b'http://127.0.0.1:8000'), (b'sec-websocket-version', b'13'), (b'user-agent', b'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like 
Gecko) Chrome/94.0.4606.81 Safari/537.36'), (b'accept-encoding', b'gzip, deflate, br'), (b'accept-language', b'en'), (b'cookie', b'csrftoken=jzHH9Te6cml6zXxOwjSkdsMupUjoIPPLGkNGMvCsyUsdidzNmYJNBFWeflJI59Qe; sessionid=1t95jxh44o9a1zfzcgsamlhau1mxjilj'), (b'sec-gpc', b'1'), (b'sec-websocket-key', b'd5LydM5OiDLcAgSqam7GcQ=='), (b'sec-websocket-extensions', b'permessage-deflate; client_max_window_bits')], 'query_string': b'kwargs=%7B%7D', 'client': ['127.0.0.1', 52403], 'server': ['127.0.0.1', 8000], 'subprotocols': [], 'asgi': {'version': '3.0'}, 'cookies': {'csrftoken': 'jzHH9Te6cml6zXxOwjSkdsMupUjoIPPLGkNGMvCsyUsdidzNmYJNBFWeflJI59Qe', 'sessionid': '1t95jxh44o9a1zfzcgsamlhau1mxjilj'}, 'session': <django.utils.functional.LazyObject object at 0x000002B8A39433A0>, 'user': <channels.auth.UserLazyObject object at 0x000002B8A3943CA0>, 'path_remaining': '', 'url_route': {'args': (), 'kwargs': {'view_id': 'test_app.components.Button'}}}, close=<bound method AsyncWebsocketConsumer.close of <django_idom.websocket_consumer.IdomAsyncWebsocketConsumer object at 0x000002B8A3943E50>>, disconnect=<bound method IdomAsyncWebsocketConsumer.disconnect of <django_idom.websocket_consumer.IdomAsyncWebsocketConsumer object at 0x000002B8A3943E50>>, view_id='test_app.components.Button'))
Traceback (most recent call last):
  File "C:\Users\username\Documents\Repositories\django-idom\.venv\lib\site-packages\idom\core\layout.py", line 207, in _render_component
    self._render_model(old_state, new_state, raw_model)
  File "C:\Users\username\Documents\Repositories\django-idom\.venv\lib\site-packages\idom\core\layout.py", line 245, in _render_model
    self._render_model_children(old_state, new_state, raw_model.get("children", []))
  File "C:\Users\username\Documents\Repositories\django-idom\.venv\lib\site-packages\idom\core\layout.py", line 364, in _render_model_children
    self._render_model(old_child_state, new_child_state, child)
  File "C:\Users\username\Documents\Repositories\django-idom\.venv\lib\site-packages\idom\core\layout.py", line 244, in _render_model
    self._render_model_attributes(old_state, new_state, raw_model)
  File "C:\Users\username\Documents\Repositories\django-idom\.venv\lib\site-packages\idom\core\layout.py", line 266, in _render_model_attributes
    self._render_model_event_handlers_without_old_state(
  File "C:\Users\username\Documents\Repositories\django-idom\.venv\lib\site-packages\idom\core\layout.py", line 306, in _render_model_event_handlers_without_old_state
    self._event_handlers[target] = handler
AttributeError: _event_handlers

New Behavior

Need to gracefully handle WS close within _render_model

Implementation Details

No response

Code of Conduct

• I agree to follow the Code of Conduct.

Current Situation

Currently, component state is lost upon websocket reconnect. This forces a re-render from the base state.

Proposed Changes

Configure some method to save off the current state of the component

Implementation Details

Implement as an opt-in feature. Might be implemented as an alternative to use_state (for example, use_persistent_state) or as a decorator @idom.persistent_component.

Storing the state

• Use dill to pickle whatever is currently within state upon each set_state mutation.
• State can be stored client side within sessionStorage.

Communicating the state back to the server

• Need to develop some method of preventing the first render if the browser currently has a stored state.
• When the pickled data is transported back as json, it will need to somehow be automatically put back into the state

Avoid state spoofing

• If a component state contains something like "is_admin", it would be pretty trivial to spoof this.
• Need to ensure the browser owns the state it says it does (if state is stored client sided)
• Need to ensure the state has been unmodified since last connection (if state is stored client sided)

Current Situation

ReactPy router exists now, but there is currently no Django-specific version of this.

Proposed Actions

Create a django_router that can handle django-style URL patterns.

Current Situation

Currently, a query refetch cannot be triggered unless through a use_mutation call. Apollo supports refetching via Result.refetch(), so we should too.

Proposed Actions

Develop a Query.refetch() function.

Current Situation

We currently don't have a method to remind people to update the changelog, despite this being a required part of most PRs.

Proposed Actions

Create a "Welcome Message" CI workflow that comments on each new PR, reminding the user to perform steps such as updating the changelog if needed.