Bug Report:

How to reproduce: Get any tokens returned with docstrings outside of text range
What should happen instead: Should keep the tokens in the places necessary but remove ones outside of view

Feature report:

Salve v0.8.0 should feature a major refactor of the highlight file to simplify its structure and make it easier to maintain. May become another submodule, not sure how I'm going to do this yet but the current state of the file is atrocious across the different branches.

Feature report:

Base the backend off of parsimonious and make it easy to make language parsers. Make some pre made parsers and add to the salve repository hub.

Would make it easier to read

Bug Report:

How to reproduce: run the test_ipc.py
What should happen instead: raise AssertionError
Minimum reproducible example:
the test

Traceback:

Traceback (most recent call last):
  File "c:\...\salve-master\tests\test_ipc.py", line 64, in <module>
    test_IPC()
  File "c:\...\salve-master\tests\test_ipc.py", line 32, in test_IPC
    assert autocomplete_output == {
AssertionError

Feature report:

Explain your idea, how it could be implemented, and why it should be added. The more detail you give, the more likely it is to be added. Code snippets that could help add it are highly recommend.

Feature report:

They're one liner functions: unnecessary and useless

Bug Report:

Bug explanation: If you don't have any docstrings there is no output because the overwrite function has no new tokens to compare against
How to reproduce: Run MRE
What should happen instead: Gives old_tokens
Minimum reproducible example:

from salve_ipc import HIGHLIGHT, IPC
if __name__ == "__main__":
    code_snippet: str = """class foo:
        def bar() -> None:
            if baz:
                qux()
    """
    context = IPC()
    context.update_file("foo", code_snippet)
    context.request(HIGHLIGHT, file="foo", language="python")
    while not (result := context.get_response(HIGHLIGHT)):
        pass
    print(result)

Traceback: N/A

Feature report:

Many of the functions that the server uses could be far more optimized but I don't want to mess with the main module because in the future I may rewrite it in Mojo (for fun and because I like the language, not for the speed claims) which is similar to Python and keeping a Python implementation wouldn't hurt. More than just that though, I really like the state of it and I'm proud of the work I've done and the progress I've made making the tools all on my own. That being said, speed is an option I want available to users. I therefore think that #16, #28 and this can all be pushed into one overarching plan to have a submodule that utilizes the proper tools for the job.

Bug Report:

How to reproduce: Try doing any command, whether that be highlighting, autocomplete, replacements and time it. It takes about .1s to .2s on my machine to get a response from the server. Thats unreasonably slow for a tool thats supposed to help speed up code editors and development time.
What should happen instead: Should almost always complete within .05s with files about 14,000 lines long (when using highlight should use about a 30 line snippet for these results)
Minimum reproducible example:

from time import sleep, time

from salve_ipc import IPC

times = []
for i in range(50):
    context = IPC()

    context.update_file(
        "test",
        open(__file__, "r+").read() * 200,
    )

    start_time = time()
    context.request(
        "autocomplete",
        file="test",
        expected_keywords=[],
        current_word="t",
    )

    while not context.get_response("autocomplete"):
        continue
    end_time = time() - start_time

    context.kill_IPC()

    times.append(end_time)

for i in range(50):
    context = IPC()

    context.update_file(
        "test",
        open(__file__, "r+").read() * 200,
    )

    start_time = time()
    context.request("highlight", file="test", language="python", text_range=(1, 21))

    while not context.get_response("highlight"):
        continue
    end_time = time() - start_time

    context.kill_IPC()

    times.append(end_time)

for i in range(50):
    context = IPC()

    context.update_file(
        "test",
        open(__file__, "r+").read() * 200,
    )

    start_time = time()
    context.request("replacements", file="test", expected_keywords=[], current_word="contest")

    while not context.get_response("replacements"):
        continue
    end_time = time() - start_time

    context.kill_IPC()

    times.append(end_time)

print(sum(times) / len(times))

Feature report:

In #37 (0.7.0) it was decided to mark the package as only available for 3.11 and after as 3.10 was crucial in many places and 3.11 brought NotRequired to typing. I believe there may be a workaround NotRequired and would like to make the package work for more versions then just 3.11 and 3.12 with a goal of constantly keeping the three most recent versions supported (not including alpha versions (looking at you 3.13)). Additionally, when a new release is created, the oldest of the three is dropped even if it's still getting security releases as I deem it better to use the benefits modern versions provide over long term backwards compatibility. What this means is once 3.13 is out of alpha, bye bye 3.10 and back to 3.11 :)

Feature report:

Allow logging (I'm realizing they're actually very useful) but don't require it.

Waiting on Tree Sitter branch.

Feature report:

Albero is going to eventually have an IPC/server backend so it would only make sense to make this a standalone project that both use

Feature report:
Would significantly improve efficiency of files

Use pyeditorconfig

Bug Report:

Python 3.9 doesn't have match statements and they were added in 3.10 so the setup.py needs to be updated. I could remove them but they make the code significantly easier to read and really don't want to because of that. NotRequired debuted in 3.11 so thats the minimum needed version

How to reproduce: use any python version lower than 3.11
Minimum reproducible example:

from salve_ipc import IPC

or

match "test":
    case "test": pass

Traceback:

    match "test":
          ^
SyntaxError: invalid syntax

Feature report:

Plum dispatch could help clean up the request code significantly and is maintained by beartype :)

Feature report:

Tests currently run in ubuntu and I test locally on Mac, should make a windows runner

Yay or nay?

https://github.com/astral-sh/ruff/blob/main/docs/formatter/black.md

Feature report:

I think it would be nice to keep the current setup where the user can use language-agnostic functions or methods that don't require a bunch of executables that help in most scenarios but it would be nice to provide a nice interface for real LSP's that maps the output to the current form wherever possible. This might warrant a separate LSP package like sansio-lsp-client as a base and then the interface is extrapolated. How would this look? What would the API become? How does LSP actually work? How should we make the interactions work? What should be sent to the server? Should this lazily be done like the rest of the project where we just take advantage of the ease of use parts and hope the implementation is fast enough and provide optional optimizations the user can choose to use?

Working on ways to fix this

https://invisible-characters.com

Feature report:

Using caches could increase speeds significantly in certain places. Most notably:

• is_unicode_letter
• get_new_token_type
• get_pygments_comment_regexes
• get_lexer_by_name

Highlighting could be multithreaded by splitting up the line logic into one function that returns all tokens for a line into one function that returns all tokens for a line and these return tokens are just sorted.

It would make much more sense to do this separate from the normal highlighting

Feature report:

Salve IPC will use the three most recent versions (full releases) going forward (once 3.13 has made its full release (currently requires 3.11)) and will drop any older versions whether they are in security fix stage or not, as I hope to keep this package up to date with modern python versions.

Should be added to the readme.

Feature Report:

We should keep a two line buffer (as default (should be changeable)) of tokens being returned so that there is no laginess when scrolling and requesting new tokens. Additionally, any tokens not in the text range (text range includes buffer) should be removed from the final tokens.

Will wait on PR #34 and should be slated for the v0.8.0 release

In accordance with salve-org #3 there should be a To do section in the PR template

Feature report:

Instead of just putting overwriting tokens at the end of the list when highlighting actually update the tokens and push in the new tokens with a overwrite_old_tokens_with_new function that takes the list of old tokens and new ones to add and overwrites them precisely. This should also allow some tokens to coexist like hidden chars and string token in the same ares

Feature report:

Use Token Tools throughout the project

Maybe? Lots to think on regarding this.

May reduce minutes used and significantly reduce e-mail volume.

In tests and examples, it sets a good example for users and lets them know it exists for them

Goes with #3. Using multiprocessing with daemen=True removes the need.

Need to be updated and improved

Bug Report:

How to reproduce: create 2k line long python fuel and try to get highlights for it keeping track of time (even bypass the IPC!). You will notice it can take up to .5 seconds for this. That's not gonna fly. The highlighting code needs to be optimized significantly to be usable
What should happen instead: completes in reasonable amount of time