idea-research / deepdataspace Goto Github PK

#0 396.9 Progress: resolved 2025, reused 0, downloaded 1862, added 1895
#0 397.9 Progress: resolved 2025, reused 0, downloaded 1912, added 1945
#0 398.9 Progress: resolved 2025, reused 0, downloaded 1944, added 1976
#0 399.9 Progress: resolved 2025, reused 0, downloaded 1946, added 1977
#0 400.9 Progress: resolved 2025, reused 0, downloaded 1990, added 2024
#0 401.9 Progress: resolved 2025, reused 0, downloaded 1990, added 2025, done
#0 401.9 .../node_modules/core-js-pure postinstall$ node -e "try{require('./postinstall')}catch(e){}"
#0 401.9 .../[email protected]/node_modules/core-js postinstall$ node -e "try{require('./postinstall')}catch(e){}"
#0 401.9 .../[email protected]/node_modules/es5-ext postinstall$ node -e "try{require('./_postinstall')}catch(e){}" || exit 0
#0 401.9 .../node_modules/@sentry/cli install$ node ./scripts/install.js
#0 402.0 .../node_modules/core-js-pure postinstall: Thank you for using core-js ( https://github.com/zloirock/core-js ) for polyfilling JavaScript standard library!
#0 402.0 .../node_modules/core-js-pure postinstall: The project needs your help! Please consider supporting core-js:
#0 402.0 .../node_modules/core-js-pure postinstall: > https://opencollective.com/core-js
#0 402.0 .../node_modules/core-js-pure postinstall: > https://patreon.com/zloirock
#0 402.0 .../node_modules/core-js-pure postinstall: > https://boosty.to/zloirock
#0 402.0 .../node_modules/core-js-pure postinstall: > bitcoin: bc1qlea7544qtsmj2rayg0lthvza9fau63ux0fstcz
#0 402.0 .../node_modules/core-js-pure postinstall: I highly recommend reading this: https://github.com/zloirock/core-js/blob/master/docs/2023-02-14-so-whats-next.md
#0 402.0 .../[email protected]/node_modules/core-js postinstall: Thank you for using core-js ( https://github.com/zloirock/core-js ) for polyfilling JavaScript standard library!
#0 402.0 .../[email protected]/node_modules/core-js postinstall: The project needs your help! Please consider supporting of core-js:
#0 402.0 .../[email protected]/node_modules/core-js postinstall: > https://opencollective.com/core-js
#0 402.0 .../[email protected]/node_modules/core-js postinstall: > https://patreon.com/zloirock
#0 402.0 .../[email protected]/node_modules/core-js postinstall: > bitcoin: bc1qlea7544qtsmj2rayg0lthvza9fau63ux0fstcz
#0 402.0 .../[email protected]/node_modules/core-js postinstall: I highly recommend reading this: https://github.com/zloirock/core-js/blob/master/docs/2023-02-14-so-whats-next.md
#0 402.0 .../node_modules/core-js-pure postinstall: Done
#0 402.0 .../[email protected]/node_modules/core-js postinstall: Done
#0 402.1 .../[email protected]/node_modules/es5-ext postinstall: Done
#0 402.2 .../node_modules/@sentry/cli install: [sentry-cli] Downloading from https://downloads.sentry-cdn.com/sentry-cli/1.75.2/sentry-cli-Linux-x86_64
#0 1371.5 .../node_modules/@sentry/cli install: Error: Unable to download sentry-cli binary from https://downloads.sentry-cdn.com/sentry-cli/1.75.2/sentry-cli-Linux-x86_64.
#0 1371.5 .../node_modules/@sentry/cli install: Error code: ETIMEDOUT
#0 1371.5 .../node_modules/@sentry/cli install: Failed
#0 1371.5 ?ELIFECYCLE? Command failed with exit code 1.

failed to solve: process "/bin/sh -c npm install -g [email protected] && pnpm install --frozen-lockfile && pnpm run build:app" did not complete successfully: exit code: 1

Hello, I already update the newest version 0.9.1
when I use the collaborate labeling function. With the keypoints labeling tool, everything is fine when labeling and saving success notify. I want to look back the editor keypoint labeling and view the list images labeled.

only the bounding box available but not the key-point saving. I also want to check extract ground truth with script bellow.

dataset = DataSet.find_one({"id": dataset_id})
    # create the image model dynamically
    ImageModel = Image(dataset_id)
    num_images = ImageModel.count_num({})
    print(f"Exporting dataset of {num_images} images: {dataset.name}")
    output_images = []
    # find all images in the dataset sorted by idx and id, and exporting them
    images = ImageModel.find_many({}, sort=[("idx", 1), ("id", 1)])
    for idx, image in enumerate(images):
        # export all fields of the image
        output = {
            "id": image.id,
            "idx": image.idx,
            "url": image.url_full_res,
            "thumbnail": image.url,
            "width": image.width,
            "height": image.height,
            "metadata": json.loads(image.metadata),
            "objects": []
        }
        for anno_obj in image.objects:
            # print(anno_obj)
            obj_data = anno_obj.to_dict(exclude=["compare_result", "matched_det_idx", "confirm_type"]) # exclude some fields
            print(obj_data["points"])
            if len(obj_data["points"]):
                output["objects"].append(obj_data)
        output_images.append(output)
        if len(output["objects"]):
            print("---", output)
        # print(f"{idx + 1}/{num_images}: exported image {image.id} of {len(output['objects'])} objects")


    output = {
        "dataset_id": dataset_id,
        "dataset_name": dataset.name,
        "images": output_images
    }

    os.makedirs(output_dir, exist_ok=True)
    with open(f"{output_dir}/{dataset.name}_{dataset_id}.json", "w", encoding="utf8") as fp:
        json.dump(output, fp, ensure_ascii=False)

but there is no object had labeled too.

Background

A small in-houses annotation team wants to use DDS with a small number of internal users, e.g. 1-3 user(s). The team does not need the full collaboration features, however persistance of the annotation is mandatory.

Issues

There are several areas that could benefit from improvements to enhance usability and comprehension, especially for users who may not consult documents before proceeding:

• In the current setup, the "quick mode" does not persist annotations in the database. This fact, although mentioned in the tutorials and documents, is not explicitly stated within the user interface. This oversight could affect those who lean towards "quick mode" usage without prior reading of the accompanying documentation.

• The existing process for setting up a team in the collaborative mode appears excessive for use cases that primarily require simple shared annotation usage. This complexity could deter users looking for a more straightforward annotation solution.

• Certain compulsory steps for using the collaborative mode, such as user creation, dataset creation, etc., may not be intuitively understandable from the user interface alone. This lack of clarity could be a hurdle for users who do not delve into the documents before initiating usage.

Proposals

• Clarify in the "quick mode" that annotations are not saved automatically, and that users need to manually export for the preservation of annotated results.

• Introduce a set of "collaborative annotation recipes" for a simplified, one-click setup in the collaborative mode. These should include automatic setup of initial users, datasets, permissions, and so on.

• Ensure the user interface explicitly communicates that certain steps need to be executed via the command line interface (CLI). This should help guide users who may not be aware of this necessity before reading the tutorials.

DDS now only support Ubuntu18.04? I just install the DDS in Ubuntu20.04, using conda , python3.8. It seems DDS could start, but there is error in the web ui when it is loading default dataset. Also, I can create an account with ddsop; but cannot login with the created account.

Thank you for the great work!

I had face this bug when try to restart machine. It seem the dds could not load previous dataset had saved before.

2023-11-09 09:07:56,428 INFO [django.server] "GET /api/v1/datasets?page_num=1&page_size=50 HTTP/1.1" 200 433
2023-11-09 09:07:56,441 ERROR [django.request] Internal Server Error: /files/local_files/b/1/-1_-1/image_png//dds/datasets/test_data_ann/sub1/2.png
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/redis/connection.py", line 624, in connect
    sock = self.retry.call_with_retry(
  File "/usr/local/lib/python3.10/site-packages/redis/retry.py", line 46, in call_with_retry
    return do()
  File "/usr/local/lib/python3.10/site-packages/redis/connection.py", line 625, in <lambda>
    lambda: self._connect(), lambda error: self.disconnect(error)
  File "/usr/local/lib/python3.10/site-packages/redis/connection.py", line 658, in _connect
    for res in socket.getaddrinfo(
  File "/usr/local/lib/python3.10/socket.py", line 955, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -3] Temporary failure in name resolution

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/django/core/handlers/exception.py", line 56, in inner
    response = get_response(request)
  File "/usr/local/lib/python3.10/site-packages/django/core/handlers/base.py", line 197, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/dds/source/./deepdataspace/server/resources/files/local_file.py", line 65, in read_file
    if not Redis.sismember(RedisKey.DatasetImageDirs, file_dir):
  File "/usr/local/lib/python3.10/site-packages/redis/commands/core.py", line 3340, in sismember
    return self.execute_command("SISMEMBER", name, value)
  File "/usr/local/lib/python3.10/site-packages/redis/client.py", line 1255, in execute_command
    conn = self.connection or pool.get_connection(command_name, **options)
  File "/usr/local/lib/python3.10/site-packages/redis/connection.py", line 1427, in get_connection
    connection.connect()
  File "/usr/local/lib/python3.10/site-packages/redis/connection.py", line 630, in connect
    raise ConnectionError(self._error_message(e))
redis.exceptions.ConnectionError: Error -3 connecting to redis:6379. Temporary failure in name resolution.
2023-11-09 09:07:56,441 ERROR [django.request] Internal Server Error: /files/local_files/b/1/-1_-1/image_png//dds/datasets/test_data_ann/sub1/2.png
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/redis/connection.py", line 624, in connect
    sock = self.retry.call_with_retry(
  File "/usr/local/lib/python3.10/site-packages/redis/retry.py", line 46, in call_with_retry
    return do()
  File "/usr/local/lib/python3.10/site-packages/redis/connection.py", line 625, in <lambda>
    lambda: self._connect(), lambda error: self.disconnect(error)
  File "/usr/local/lib/python3.10/site-packages/redis/connection.py", line 658, in _connect
    for res in socket.getaddrinfo(
  File "/usr/local/lib/python3.10/socket.py", line 955, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -3] Temporary failure in name resolution

During handling of the above exception, another exception occurred:

how to find the user's password since it is created randomly when add user? or is there a way to change or set the password

Hello!
I got this error when starting DDS docker from docker compose with following configuration:

  deep_data_spaces:
    image: deepdataspace/dds
    ports:
      - 8765:8765
    command: "dds --quickstart -V"
    volumes:
      - ./dds:/dds/runtime

Maybe I need to configure some environment variable related to mongo?

Hello as the topic.
I could not find how to disable the keypoint invisible in when do pose annotation?
please help! thank you

for the Collaborative Mode of annotation, is there a simple way to create a unlabeled dataset from a local directory and then export the annotations, not use the python code in the terminal ?

Screenshots of installation startup and results are as follows

Great work!
I wonder if there is a way of consuming the API via CLI without touching the frontend?
Let's say I have 100 images and I want to do the same procedure with those and not one by one.
Thanks in advance

Greak work！Can CentOS be supported?