[Bug]: PersistentVolumes get deleted when their relative PersistentVolumeClaims are deleted due to StorageClass reclaim policy ignoring the `--pv-reclaim-policy` value. about kadalu HOT 7 CLOSED

Expected behavior
When creating new KadaluStorage and using --pv-reclaim-policy=retain flag, the StorageClass ReclaimPolicy should be set to Retain.

• I believe the feature is misunderstood, I don't blame you though 😄 , thanks for raising the issue as this calls for an improvement in the docs
• Assume you have two actors, one is kubernetes storage class (hereafter just sc) another is kadalustorage cr (hereafter just cr)
• The reclaim policy you mentioned in the cr doesn't get carried into sc that kadalu creates by default
• the policy in cr serves different purpose than the policy in sc, let me explain with two usecases

Usecase 1: When a PVC is deleted, unless cluster is deleted you want PV to be retained
Solution: Copy the sc that kadalu creates for a specific cr onto a new sc and set the parameter reclaimPolicy: Retain, this prohibits removal of cr unless you delete the retained PV(C)s.

Usecase 2: When a PVC is deleted, even when cluster is deleted you want PV to be retained
Solution: You set --pv-reclaim-policy=retain on cr and this doesn't prohibit cr removal and data is still retained in backend bricks. This provides you an option for recovery

• I believe you are trying to achieve Usecase 1 by using Solution of Usecase 2 which isn't intentionally allowed

from kadalu.

I agree, 100% valid points. Thank you for your engagement on the matter! ❤️ The bug can be closed :)

from kadalu.

Hello!

Thanks for your feedback and for taking a look! :)

So, if I understand correctly - even if the PersistentVolume is deleted on the Kubernetes layer, you are saying that the actual volume does stay on the KadaluStorage layer ( would the files be found on, let's say server-cluster-storage-0-0 pod? ).

After the initial deletion of the PVC, I tried recreating it again and hoped that if the above were true, the new PV would contain the data from the previous volume, but it has simply created a new PV.

On the contrary, allowing the StorageClass to be set to Retain does make the recovery of the files more generic ( as the old PV stays and can be reattached to another pod ), but I understand Kudalu does not support it.

Let me know what you think 👍

from kadalu.

would the files be found on, let's say server-cluster-storage-0-0 pod?

• yep, to be exact, they are retained on the backend bricks, for retain on cr, it'll just be a no-op from kadalu perspective for delete volume rpc https://github.com/kadalu/kadalu/blob/devel/csi/volumeutils.py#L638-L645

I tried recreating it again and hoped that if the above were true, the new PV would contain the data from the previous volume, but it has simply created a new PV.

• nope, it doesn't work that way, you might want to use the PV via static provisioning but not through PVC, my knowledge on this area reduced a bit, need to recheck

but I understand Kudalu does not support it.

• no, it's a kubernetes feature, every storage class supports it, as mentioned above, recreate a new sc from kadalu created default sc and then use reclaimPolicy: Retain in this new sc
• it's just that kadalu doesn't provide you an option to mention reclaimPolicy: Retain on default sc's, think about it, it's not at all related to storage layer
• when that param is set, k8s just doesn't send delete volume rpc

from kadalu.

need to recheck

• I think this is one #765 when I last looked into this area

from kadalu.

Hi!

Thanks that makes sense, it clarified a lot. I can see now why this is not a bug.

Would you say adding a potential ability to set the reclaimPolicy: Retain on the default sc could be considered a feature upgrade instead?

from kadalu.

Probably not because of below:

1. These two are separate concerns with a clear owner, i.e, kadalu vs kubernetes, so don't want to club
2. Adding a separate option in cr might work but we have to take more decisions unecessarily
3. Creating a new sc is one time task which doesn't warrant reconciliation

Basically I can see the appeal, at the same time, don't want to introduce yet another option for a standalone feature, pls excuse.

from kadalu.