Code Monkey home page Code Monkey logo

docker-storj-extension's Introduction

This repository contains a docker desktop extension. In the current form this is nothing more just a UI for starting a Docker Registry <-> Storj gateway.

The started container includes a modified version of distribution/distribution. It's a Docker registry which stores all the data and metadata on Storj Decentralized cloud with implementing a new Storj driver.

This extension just starts the required gateway (and explain how it can be used), but later it can be improved to use full OAuth workflow to get STORJ access grant instead of require it in the form.

The source code of the gateway can be found here and can be tested without this extension with:

docker run -p 9999:5000 -e REGISTRY_STORAGE_STORJ_BUCKET=docker -e REGISTRY_STORAGE_STORJ_ACCESSGRANT=$(cat /tmp/grant) ghcr.io/elek/distribution:618d19fb

Try it out

As the extension is not in the marketplace, you need:

  • Docker Desktop: see the official documentation to install it,
  • Docker Extension CLI: Download it from here and copy it to /usr/lib/docker/cli-plugins/ (or similar folder)

Extension can be installed with the command:

docker extension install ghcr.io/elek/docker-storj-extension:20220728-2

At first time, you need to create a new bucket and generate the access grant (you can use Storj web interface or uplink).

For example on EU1 gateway you can do it at https://eu1.storj.io/access-grants

After starting the container, you can use the local registry backed by Storj:

Pull an image from Docker Hub:

docker pull elek/herbstag

Using default tag: latest
latest: Pulling from elek/herbstag
a0d0a0d46f8b: Pull complete
Digest: sha256:0d86302fb863deb9a1e7371a947e7d3c029c1d55a28bb5f20f7e09317254e562
Status: Downloaded newer image for elek/herbstag:latest

Tag the image with the new location:

docker tag elek/herbstag localhost:9999/elek/herbstag

And push it:

docker push localhost:9999/elek/herbstag

Using default tag: latest
The push refers to repository [localhost:9999/elek/herbstag]
e2eb06d8af82: Layer already exists
latest: digest: sha256:0d86302fb863deb9a1e7371a947e7d3c029c1d55a28bb5f20f7e09317254e562 size: 528

You can double check the content of the Storj bucket:

export UPLINK_ACCESS=$(cat /tmp/grant)    

uplink ls -r sj://docker/  | grep herbstag
OBJ     2022-07-28 11:36:51    71         docker/registry/v2/repositories/elek/herbstag/_manifests/revisions/sha256/0d86302fb863deb9a1e7371a947e7d3c029c1d55a28bb5f20f7e09317254e562/link
OBJ     2022-07-28 11:36:51    71         docker/registry/v2/repositories/elek/herbstag/_manifests/tags/latest/index/sha256/0d86302fb863deb9a1e7371a947e7d3c029c1d55a28bb5f20f7e09317254e562/link
OBJ     2022-07-28 11:36:52    71         docker/registry/v2/repositories/elek/herbstag/_manifests/tags/latest/current/link
OBJ     2022-06-30 11:17:15    71         docker/registry/v2/repositories/elek/herbstag/_layers/sha256/a0d0a0d46f8b52473982a3c466318f479767577551a53ffc9074c9fa7035982e/link
OBJ     2022-06-30 11:17:21    71         docker/registry/v2/repositories/elek/herbstag/_layers/sha256/833c7a986ed965eec8fe864223920c366fb0a25dd23edd0bdd2a4428fd0ce1e2/link

docker-storj-extension's People

Contributors

elek avatar gagandeepp avatar r4topunk avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

Forkers

r4topunk

docker-storj-extension's Issues

Missing Arm platform support?

Look like the extension is missing support for Arm platform.

 docker run --rm mplatform/mquery ghcr.io/elek/docker-storj-extension:20220728-2
Image: ghcr.io/elek/docker-storj-extension:20220728-2 (digest: sha256:d7e0edc4d1f90f36540028789a98b70b6c15e954f38cc7f0329d0a981ce44a57)
 * Manifest List: No (Image type: application/vnd.docker.distribution.manifest.v2+json)
 * Supports: linux/amd64
docker extension install ghcr.io/elek/docker-storj-extension:20220728-2
Extensions can install binaries, invoke commands and access files on your machine. 
Are you sure you want to continue? [y/N] y
Image not available locally, pulling ghcr.io/elek/docker-storj-extension:20220728-2...
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
installation could not be completed due to: executing 'docker --context default cp bed34b8b339e8218045e834489d25060422b02751c2da70d80566740bea70853:/metadata.json /var/folders/1v/ybkpsn791wbg5v5vgxlxky840000gn/T/-ext-install2256936921/metadata.json' : exit status 1: 
Error: No such container:path: bed34b8b339e8218045e834489d25060422b02751c2da70d80566740bea70853:/metadata.json

Streaming console output to the frontend

Backend calls good old CLI commands under the hood (see backend/service.go). If it fails, it returns with an error (including stdout).

But some of the commands (like docker push) are quite slow.

It would be better to start the command and continuously streaming the output to the frontend which would show new lines immediately.

In the current, very simple prototype, we just wait until the response is received.

Wrong Hacktoberfest topic

The Hacktoberfest topic is written wrong on repository topics

Now:
hactoberfest

How should be:
hacktoberfest

image

Update CONTRIBUTING.md

  • Add standard Github workflow commands in the CONTRIBUTING.md (opening issues, get assigned, fork, create clone, add, commit, creating PRs).
  • It provides new potential project contributors with a short guide to how they can help with your project.

Generic error handling in frontend

Some frontend pages already have a lightweight error handling. Like in Push.vue:

var push = function(name, tag) {
  error.value = ""
  message.value = ""
  Api.push(name, tag).then(function () {
    message.value = "Container has been pushed successfully"
  }, function (e) {
    error.value = e.response.data.error
  })
}

And the render:

  <p class="alert alert-danger" role="alert" v-if="error">{{ error }}</p>
  <p class="alert alert-success" role="alert" v-if="message" v-html="message"></p>

Would be nice to make it more generic: in case of any HTTP request the failure should be written out to warn the user, that sg. is wrong.

Make registry accesible for the docker daemon

This is the biggest blocking problem with this extension, right now.

The MVP goal is to push/pull images with the help of a local registry (which is backed by StorjDCS).

This is equivalent with:

docker run -p 9999:5000 -e REGISTRY_STORAGE_STORJ_BUCKET=docker -e REGISTRY_STORAGE_STORJ_ACCESSGRANT=$(cat /tmp/grant) ghcr.io/elek/distribution:618d19fb
docker tag elek/herbstag localhost:9999/elek/herbstag
docker push localhost:9999/elek/herbstag

Unfortunately it's not so easy to start AND access the port of the container.
The port 9999 should be accessible:

  • by the docker daemon itself (to push images)
  • by the extension backend (to check the already uploaded images)

The approaches what we tried out until now:

  • We started new docker instance from backend (docker run...) with forwarded port

    • This approach requires a free port on host
    • The port is accessible by the docker daemon but not by the extension (!!!)
  • We started the registry inside the backend container as a new unix process

    • This makes it available for the backend (to check the already uploaded images)
    • But the port is not available for the docker daemon

There are two possible solution

  1. We can try to expose the port of the backend container to the host. (maybe a change in the docker-compose?)
  2. We can change the implementation of (Service).RemoteImages() in backend/service.go to directly check the Storj DCS bucket instead of checking the port of the docker registry. This would make it possible to use approach 1.
 
 
   

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.