RFC: most charts should be StatefullSet, but are Deployments about charts HOT 14 CLOSED

Just to udpate you @ishioni i have implemented a configurable deployment/statefulset choice and it is already in the bitwardenrs chart. This can be applied to any/all charts that statefulsets apply to in scope.

from charts.

According to the following issue this can be solved with configuration options. I'm new to this type of issue so maybe I'm still wrong.

kubernetes/kubernetes#74947 (comment)

from charts.

I'm for it, @ishioni. Back when some of these charts were hosted in the helm/charts repo, some of us raised this question and there was a lot of debate. It's worth referencing that conversation here as well:

• helm/charts#1863
• helm/charts#12047 (read the comments in this one)

from charts.

If we choose to pursue this, obviously it would only be necessary to alter charts that basically have a requirement to use persistent storage in order to function properly (e.g. radarr/sonarr/plex/etc). A change from deployment to statefulset will also be a 'breaking' change and will require a major version change as well as documented upgrade path.

from charts.

After reading both of those threads i still think we should proceed forward with going to STS when PVCs are involved. I totally agree with @unguiculus that STS is not a magic fix for the types of applications which don't scale at all, however @davidkarlsen rightly points out that it has some perks over normal DPs.

I think the best thing to do is to point out somewhere in new PRs that stateful applications should now uses STS with claimTemplates instead of DPs with PVCs. This repo has relatively less charts than stable, so I think it'd be a good idea not to exacerbate the issue going forward.

as well as documented upgrade path

This has me stumped most of all. When i converted my charts, most of the time it was best to either migrate the data out-of-bound or use a transporter pod with both PVCs mounted. I think the best way would be to point users to use existingClaim? But then we're still up against helm managing the lifecycle of the PVC, and that may very well lead to PVC deletion

from charts.

@ishioni and @billimek it might be worth using our k8s-at-home group to host charts, maybe we can carry ones over there as STS and slowly deprecate this repo?

https://github.com/k8s-at-home

I've also seen patterns of using git submodules to reference charts, that way each chart has it's own repo, does that seem like a good idea?

from charts.

Why would i want a statefullset, if i use kubernetes native replicated storage such as longhorn?
Statefulsets are in my opinion just more of a hassle in regards of high availability (not failing over automatically)

from charts.

Longhorn complicates restoring backups when using stateful sets?

from charts.

Longhorn complicates restoring backups when using stateful sets?

I dont know but i just dont see the purpose of running hass in a statefullset. Taking into account that you cripple auto failover functionality which is imo a key benefit of running hass in kubernetes to begin with.

from charts.

Statefulsets can failover too. I do not understand your concerns on this or longhorn when dealing with statefulsets. Care to explain more?

from charts.

Yes it can failover but needs manual intervention on node failure.
kubernetes/kubernetes#74689

Well, longhorn is not directly related. What it does offer is super flexible distributed storage with easy automatic failovering over nodes. That contradicts for me with the HA characteristics of a statefulset

from charts.

Looks like it'd be a kube api arg: kubernetes/kubernetes#74651

from charts.

@dirtycajunrice

Not sure if people would be on board with this, as it duplicates a lot of lines. But it is one way of tackling the problem

Edit: I see your sts uses a pvc instead of a volumeclaimtemplate. Is the app "clusterable" in this way? If indeed more than one instance can work on the same set of files, then it doesn't need to be a statefullset at all, and fits the deployment model perfectly

from charts.

I am going to close this issue, with the common library @bjw-s / @dirtycajunrice put together solved this issue for the charts that use it. Meaning to say the user has an option to use a controller type of sts or deployment. Moving forward in new charts, it would be nice if more ppl got onboard using this library. Besides a few outliers most charts here follow the same type of pattern and can easily be made using that library.

from charts.