Add kubernetes backend about flame HOT 16 CLOSED

btw spawn looks very neat!

from flame.

I have replaced the k8s library with a lightweight Kubernetes client implementation. Technically we would be down to 0 additional dependencies. ;)

  defp deps do
    [
      {:flame, "~> 0.1.5"},
      {:req, "~> 0.4.5"},
      {:ex_doc, ">= 0.0.0", only: :dev, runtime: false}
    ]
  end

from flame.

... Using separated backend and controller would allow RBAC and Kubernetes dependencies to be apart of the actual application. The backend would then talk to the controller using HTTP.

Hi @chrismccord and others. I started developing the variant based on a Kubernetes controller, with the advantages that @mruoss mentioned and some others aimed at operations teams, such as the possibility of listing all instances of runners via commands such as, for example, kubectl get runners and thus obtain insights into the execution of runners inside Kubernetes. I'll leave the link here for anyone who wants to follow the work or contribute.

Link to repository https://github.com/eigr-labs/flame-k8s

from flame.

There are now two options available, awesome job everyone!

from flame.

@sleipnir looks fantastic!

from flame.

Also note that a bar for us on built-in vs external lib adapter is it doesn't introduce more deps into the main package.

from flame.

Thanks for the insights! I think first steps as an external lib make sense, and nothing in particular requires it to be in core. Folks can explore what it looks like and packup up a k8s backend as flame_k8s and that would be great!

from flame.

Oh yeah I thought about Eigr when I heard about FLAME. I also had the idea of a (plain) k8s backend. @sleipnir I think we can get it done without bonny. But we need k8s. I actually have a running POC locally. My only problem at this moment: I don't receive the {remote_ref, {:remote_shutdown, :idle}} message in my backend when the runner is shut down. This way I can't delete those runner pods that are in state complete. Is this a known issue? Or am I doing something wrong? Because I got everything else to work like a charm...

from flame.

My only problem at this moment: I don't receive the {remote_ref, {:remote_shutdown, :idle}} message in my backend when the runner is shut down. This way I can't delete those runner pods that are in state complete. Is this a known issue? Or am I doing something wrong? Because I got everything else to work like a charm...

#16

from flame.

Alright, @sleipnir and I have decided to publish a simple "all-included" backend for Kubernetes now and follow up with a second one where the backend and controllers are separated. Using separated backend and controller would allow RBAC and Kubernetes dependencies to be apart of the actual application. The backend would then talk to the controller using HTTP.

You can find a first POC version of the "integrated" variant here: https://github.com/mruoss/flame_k8s_backend

However, it does not (yet) delete runner pods until #16 is merged.

from flame.

We will gladly accept pull requests.

from flame.

At the very least, you would have to add the Bonny dependency and that would bring some dependencies with it. You need to follow a series of steps and create a series of resources to be able to interact with the k8s api.

Alternatively you could write a controller, with bonny, or something else and this lib here would interact with the controller api via rest. This would not require bringing the dependencies directly to this project. Something similar to the Fly backend that only interacts with its REST api.
So in my opinion maybe it's better not to add anything here and work in a separate repository to support k8s.

By the way, I am co-creator of the Spawn project, another BEAM related initiative aimed at the Serverless ecosystem, in our case not oriented to FaaS. I want to congratulate you, on behalf of our project, for Flame and what you've been doing, our initiatives are very different, but they have some points in common, and I'm sure we can learn from each other.

from flame.

I started a draft with no expected end date at:

https://github.com/eigr-labs/flame-k8s

I will inform you about the progress.

from flame.

https://github.com/eigr-labs/flame-k8s-controller

Hi @mruoss, happy to see you here.
I would be happy to join forces with you on a common backend. I thought about splitting the work into two parts, a flame-k8s backend https://github.com/eigr-labs/flame-k8s that could interact with a controller installed in kubernetes https://github.com/eigr-lab/flame-k8s-controller.
I preferred to add bonny so he can configure rbacs more easily for me.
Are you creating pods directly or are you using a deployment? I'm unsure whether we create a Deployment with an HPA or whether we create pods directly. I'm biased towards creating Deployments because we could make HPA work in our favor regarding scalability.
We can talk better via Discord. If you are interested in a collaboration, of course.

from flame.

@mruoss just to make you aware, the reference to DRAGONFLY_PARENT has been changed in this commit: 9c2e65c

https://github.com/mruoss/flame_k8s_backend/blob/290be0da4df1628e152a40e0c990a15ba8f6fe10/lib/flame_k8s_backend.ex#L173

from flame.

@mruoss just to make you aware, the reference to DRAGONFLY_PARENT has been changed in this commit: 9c2e65c

THANKS! I was wondering about that name... ;)

from flame.