[BACKPORT][v1.5.4][BUG] The activated DR volume do not contain the latest data. about longhorn HOT 9 CLOSED

Need to test the regular restore volume after the corresponding PR gets merged. Here is one manual test QAs can check besides the regression test:

1. Create a Launch a volume in the 1st cluster
2. Create a large backup for the volume.
3. Continue writing more data to the volume.
4. Create a regular restore volume in the 2nd cluster.
5. During the restoration in-progress, Ask the original volume to create the 2nd backup
6. In the 2nd cluster, manually update backupVolume.Spec.SyncRequestedAt so that the 2nd backup will be shown in the 2nd cluster.
7. See if the restoring volume can finish the restore and get detached automatically ==> This should be YES.

from longhorn.

@roger-ryao let's keep this open until longhorn/longhorn-manager#2621 gets merged.

@shuo-wu Please describe what side effect @longhorn/qa needs to verify for the above PR.

from longhorn.

1. backupVolume.Spec.SyncRequestedAt should be later than backupVolume.Status.LastSyncedAt
2. The UI improvement ticket is #7982

from longhorn.

cc @roger-ryao

Verified on v1.5.4-rc4 20240226

Installed with Kubectl.

kubectl apply -f https://raw.githubusercontent.com/longhorn/longhorn/v1.5.4-rc4/deploy/longhorn.yaml

The test steps: #7945 (comment)

1. Launch 2 clusters and both have Longhorn installed
2. Set up a backup target.
3. Create two volumes and write data in the 1st cluster. Then create volumes' backups.
4. Restore two backup files as DR volumes in the 2nd cluster.
5. Modify the backup poll interval to a large value.
6. Write more data for the volumes in the 1st cluster, and create the 2nd backup.
7. Activate one of the DR volumes in the 2nd cluster. Then verify the data

Only the activating DR volume will trigger the last backup synchronizing mechanism.

from longhorn.

Pre Ready-For-Testing Checklist

• Have the backend code been merged (Manager, Engine, Instance Manager, BackupStore etc) (including backport-needed/*)?
  The PR is at longhorn/longhorn-manager#2616
  longhorn/longhorn-manager#2623

from longhorn.

Verified on v1.5.x-head 20240221

• longhorn v1.5.x-head d82f951
• longhorn-manager v1.5.x-head longhorn/longhorn-manager@103a15f

The test steps

1. #7945 (comment)

Result Passed

• 1. The activated DR volume contains the latest data.
• 2. require/manual-test-plan : longhorn/longhorn-tests#1732

from longhorn.

In the 2nd cluster, manually update backupVolume.Spec.SyncRequestedAt so that the 2nd backup will be shown in the 2nd cluster.

Hi @shuo-wu,
I have two questions about item 6. Manually updating backupVolume.Spec.SyncRequestedAt has any limitations?
For example, must it be later than the current time, or if we update the backupVolume.Spec.SyncRequestedAt at any time, will it trigger the creation of the second backup on the second cluster?
Do we have plans to add this functionality to the UI? If yes, perhaps we need to create a related ticket for that.

from longhorn.

Verified on v1.5.4-rc3 20240222

• longhorn v1.5.4-rc3 70a962e
• longhorn-manager v1.5.4-rc3 longhorn/longhorn-manager@7250ac3

The test steps

Test Method 1 : #7945 (comment)

1. Launch 2 clusters and both have Longhorn installed
2. Set up a backup target.
3. Create two volumes and write data in the 1st cluster. Then create volumes' backups.
4. Restore two backup files as DR volumes in the 2nd cluster.
5. Modify the backup poll interval to a large value.
6. Write more data for the volumes in the 1st cluster, and create the 2nd backup.
7. Activate one of the DR volumes in the 2nd cluster. Then verify the data

Test Method 2 refer:#7947 (comment)

Result Passed

• 1. Test Method 1.
• 2. Test Method 2
• 3. require/manual-test-plan : longhorn/longhorn-tests#1732

Hi @shuo-wu and @mantissahz:
When I activate one of the DR volumes in the 2nd cluster, I notice that another DR volume also starts to pull the latest backup. Is this expected? Because in v1.4 & v1.3, DR volumes syncing up are independent and won't trigger each other.

from longhorn.

Hi @shuo-wu and @mantissahz:
When I activate one of the DR volumes in the 2nd cluster, I notice that another DR volume also starts to pull the latest backup. Is this expected? Because in v1.4 & v1.3, DR volumes syncing up are independent and won't trigger each other.

Hi @shuo-wu & @mantissahz
It seems like my environment issue. I built up a new cluster, and DR volumes syncing up are independent and won't trigger each other on v1.5.4-rc3.

from longhorn.