PVCs names should be defined as variables for NEO4J chart about neo4j-helm HOT 7 CLOSED

@nosnow111 can you be a bit more specific about what names you'd be using. Reason I'm asking is that right now, the name is just "datadir" which I think is going to get other prefixes attached to it. Because the prefixes get used to sort out which one is for core0, core1, etc -- I need a more specific picture of what you're trying to do to think about how to implement this in a way that makes sense.

On way is to set a different prefix/suffix, in which case you're still going to have to pre-create PVCs by particular names. If you wanted something like:

• PVC foo mapped to core0
• PVC bar mapped to core1
• PVC baz mapped to core2

That's quite a bit different in terms of complexity and what you'd have to specify in values to make it work.

from neo4j-helm.

Just leaving a note for later -- there's a second use case for this.

• Customer starts with a read replica, running GDS on some production dataset
• Wants to split off a read replica into a standalone instance so that GDS can run writes on the datset
• Possibly the best way to implement this "split off to a single instance" is to allow a launch to bring its own named PVC, and then launch a single instance against that PVC.

Things still to consider:

• Really launch against an existing PVC? Maybe better to clone it first, since the single instance will change stuff on disk and later render the read replica inoperable (wired to the same PVC)

from neo4j-helm.

Ultimately PV_PATH: /neo4j_pv on the NFS

than create perm storage structure like

/pvc_node0/data-node0
/pvc_node0/logs-node0
...

Below it the snippet of the deployment
...
volumeMounts:
- mountPath: /pvc_node0
name: cluster-storage-node0
- mountPath: /data
name: cluster-storage-node0
subPath: data-node0
- mountPath: /logs
name: cluster-storage-node0
subPath: logs-node0
- mountPath: /conf
name: cluster-storage-node0
subPath: conf-node0
...
volumes:
- name: cluster-storage-node0
persistentVolumeClaim:
claimName: neo4j-pvc
...

from neo4j-helm.

@nosnow111 I think I'm sold on the basic need -- however I need to ask questions to make sure this gets done in a way that's reasonable. What you're suggesting is one single PVC with different sub-folders for the different cores.

Are you sure this is a good idea in your deployment? The reason I ask is 2 fold:

• If you put all of your stable state on one logical disk location, and that fails, you lose the entire cluster all at once (single point of failure)
• If you put it all in one place, the I/O backplane (how fast you can do reads/writes) is competing with itself: i.e. writes/reads on a follower could be slowing down what the leader can do since the single logical drive can only go so fast.

It seems as though the deployment topology you're suggesting might compromise cluster availability & performance

from neo4j-helm.

Hi,

With Cloud shared storage this probably not an issue as it both highly scalable and available but it provide additional convinience of being able to access data from other pods within cluster

from neo4j-helm.

@nosnow111 again, not so sure about that. While cloud storage is highly scalable & available, usually in the fine print there is a guaranteed IOPS & throughput limit. What would that be in your scenario? Because all 3 members would have to share it. Do you have a reference to the kind of setup you have in mind on a cloud documentation site where we can compare it to what you'd get if you had 3 separate disks?

Keep in mind that because Neo4j must write TXs to disk in order to confirm that your data is safe before signalling that a TX has been successful, Neo4j is quite sensitive to disk access speed. A lot of clouds trade off access speed in favor of durability, availability, and other factors. What I don't want to do is enable a usage pattern that just slows the database down.

from neo4j-helm.

Additional requirements from discussion with @nosnow111 live

• I can't allocate extra PVCs as I want due to organizational restrictions. An admin allocates me one ahead of time, and I must use that, so you can't choose your own name
• For storage administration it's desirable to be able to shell into one pod and be able to cleanup storage for all 3 because they're subdirs on the same volume. Other storage ops, for example putting certificates in different directories.
• What might be other normal storage cleanup options (run a vanilla container, mount the storage, and do linux stuff with that) isn't an option because the list of pre-vetted containers in the registry is minimal and can't be added to by the user.

from neo4j-helm.