svix / python-ksuid Goto Github PK

Specifications
Python: 3.10
OS: Ubuntu 20.04.5 LTS

Description:
This is an expected issue or maybe a limitation as when generating a new ksuid based on a specific datetime it will only work for datetimes from 2015. Please check the case scenarios below:

Positive case scenario (from 2015):

ksuid.Ksuid(datetime=datetime(2015, 2, 9, 10, 42, 28))
# result:  0CM1cVjc6DnPqpKaeGHOlWt0U9n```

Negative case scenario (before 2014):

ksuid.Ksuid(datetime=datetime(2014, 2, 9, 10, 42, 28))

File "/.../.venv/lib/python3.10/site-packages/ksuid/ksuid.py", line 92, in _inner_init
    return int.to_bytes(timestamp, self.TIMESTAMP_LENGTH_IN_BYTES, "big") + payload
OverflowError: can't convert negative int to unsigned

This is because:

datetime.datetime(2014, 2, 9, 10, 42, 28).timestamp() - EPOCH_STAMP
-8104252.0

as per source code

# (ksuid.py)
# KSUID's epoch starts more recently so that the 32-bit number space gives a
# significantly higher useful lifetime of around 136 years from March 2017.
# This number (14e8) was picked to be easy to remember.

Is this a real limitation or is there any way we can overcome this? without git fork, I mean.

We can use one byte (4ms accuracy), or two bytes (much more) from the payload for a fixed-point decimal portion for the timestamp. This will increase accuracy and will keep it fully compatible with ksuids.

I think 4ms is sufficient, as the accuracy of time-sync and timestamps at a higher resolution is probably questionable. We can always extend it later on if needed, though for our use-cases it's probably more than enough.

Two different ksuids with the same timestamp (but different payloads) will not be equal, but will also not be different when doing comparisons (lt/gt). This is due to the __lt__ function not comparing the whole byte array, but rather just the datetime. It should compare the whole byte array.

In the timestamp method, the type annotation is float but an int is actually returned. The timestamp method on the KsuidMS also has float as the return type.

Just causing my linter to nag me.

Happy to put up a PR if you'd like.

Hi 👋

First thanks for the great package. Adding the KsuidMs variant (40 timestamp + 120 random bits) is great as it kind of gives the best of the two UUIDv4 (122 random bits) and ULID (48 timestamp + 80 random bits) worlds!

Since you also maintain https://github.com/svix/rust-ksuid, have you considered using the Rust implementation with Python to further improve the performance of the Python package and even reduce the amount of code to maintain? (Never did this before, but looks like PyO3 or rust-cpython are among the preferred tools for the task)

I resolved this, though sharing in case anyone runs into a similar issue. The error could absolutely be due to an unconventional configuration of pip on my system or old pip installation:

To recreate

pip3 install svix-ksuid
python3

Throws ModuleNotFoundError

Python 3.7.0 (v3.7.0:1bf9cc5093, Jun 26 2018, 23:26:24) 
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from ksuid import Ksuid
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'ksuid'

Resolution: Use python3 -m pip to install

pip3 uninstall svix-ksuid -y
python3 -m pip install svix-ksuid 
python3

Now works as intended

>>> from ksuid import Ksuid
>>> 

Update

I later ran into a second issue where, upon attempting to call a python script, the module could not be found:

<name_of_python_module> <command>
....
....
....
    from ksuid import Ksuid
ModuleNotFoundError: No module named 'ksuid'

Turns out that I needed to run the installation into the Python installation that my code was referencing using the --target argument. The script ran fine after this

which python3
# /usr/local/bin/python3
pip3 install --target /usr/local/lib/python3.7/site-packages/ --upgrade svix-ksuid

Hello,
How exactly do i store these ksuid(s) in db as str? and are these still sortable if they are of type str?

Also kindly suggest what is the correct way to generate these ids in a web app.