Comments (5)
Pre Ready-For-Testing Checklist
- Have the backend code been merged (Manager, Engine, Instance Manager, BackupStore etc) (including
backport-needed/*
)?
The PR is at longhorn/longhorn-manager#2615
longhorn/longhorn-manager#2622
from longhorn.
Verified on v1.6.x-head 20240223
- longhorn
v1.6.x-head
3382579 - longhorn-manager
v1.6.x-head
longhorn/longhorn-manager@bf3edb2
The test steps
Test Method 1 : #7945 (comment)
- Launch
2 clusters
and both have Longhorn installed - Set up a backup target.
- Create two volumes and write data in the
1st cluster
. Then create volumes' backups. - Restore two backup files as DR volumes in the
2nd cluster
. - Modify the backup poll interval to a large value.
- Write more data for the volumes in the
1st cluster
, and create the2nd backup
. - Activate one of the DR volumes in the
2nd cluster
. Then verify the data
Test Method 2 refer:#7947 (comment)
Result
Hi @shuo-wu
The activated DR volume does not contain the latest data. However, manually updating backupVolume.Spec.SyncRequestedAt
causes the DR volume to sync up with the latest backup.
Could you help to check it?
supportbundle_91df4b65-3a16-4cb1-8d8f-d901ffb280da_2024-02-23T10-10-56Z.zip
volume name : pvc-3da73260-6e9d-4a18-80ac-ae63864ed6c2
from longhorn.
Weird...
The activation is triggered (and finished) at 2024-02-23T10:06:23Z
. backupVolume.Spec.SyncRequestedAt
should be the same timestamp. But its value in the support bundle is 2024-02-23T10:06:25Z
. I have not found that which one updates it again. BTW, the 2nd backup backup-8fefd2d832114dcb
creation time is 2024-02-23T10:06:24Z
2024-02-23T10:06:23.531811509Z time="2024-02-23T10:06:23Z" level=info msg="Activating volume pvc-3da73260-6e9d-4a18-80ac-ae63864ed6c2 with frontend blockdev" func="manager.(*VolumeManager).Activate" file="volume.go:422"
2024-02-23T10:06:23.849213385Z time="2024-02-23T10:06:23Z" level=info msg="Restore/DR volume finished with the last restored backup backup-c0d99731ad034d59" func="controller.(*VolumeController).checkAndFinishVolumeRestore" file="volume_controller.go:3214" accessMode=rwx controller=longhorn-volume frontend=blockdev migratable=false node=ryao-16x-w1-29d1f2d8-bwq5j owner=ryao-16x-w1-29d1f2d8-bwq5j shareEndpoint= shareState=stopped state=attached volume=pvc-3da73260-6e9d-4a18-80ac-ae63864ed6c2
Is this case always reproducible? Can this be reproduced in other versions (master or v1.5.x)?
from longhorn.
Weird... The activation is triggered (and finished) at
2024-02-23T10:06:23Z
.backupVolume.Spec.SyncRequestedAt
should be the same timestamp. But its value in the support bundle is2024-02-23T10:06:25Z
. I have not found that which one updates it again. BTW, the 2nd backupbackup-8fefd2d832114dcb
creation time is2024-02-23T10:06:24Z
2024-02-23T10:06:23.531811509Z time="2024-02-23T10:06:23Z" level=info msg="Activating volume pvc-3da73260-6e9d-4a18-80ac-ae63864ed6c2 with frontend blockdev" func="manager.(*VolumeManager).Activate" file="volume.go:422"
2024-02-23T10:06:23.849213385Z time="2024-02-23T10:06:23Z" level=info msg="Restore/DR volume finished with the last restored backup backup-c0d99731ad034d59" func="controller.(*VolumeController).checkAndFinishVolumeRestore" file="volume_controller.go:3214" accessMode=rwx controller=longhorn-volume frontend=blockdev migratable=false node=ryao-16x-w1-29d1f2d8-bwq5j owner=ryao-16x-w1-29d1f2d8-bwq5j shareEndpoint= shareState=stopped state=attached volume=pvc-3da73260-6e9d-4a18-80ac-ae63864ed6c2
Is this case always reproducible? Can this be reproduced in other versions (master or v1.5.x)?
I built up the test environment 3 times, and the issue could be reproduced on v1.6.x
. However, I didn't observe the issue on v1.5.4-rc3
. I think I should test it on v1.5.4-rc4
. I didn't test it with the master-head
yet.
from longhorn.
Verified on v1.6.x-head 20240301
- longhorn
v1.6.x-head
3382579 - longhorn-manager
v1.6.x-head
longhorn/longhorn-manager@654ee92
The test steps
Test Method 1 : #7945 (comment)
- Launch
2 clusters
and both have Longhorn installed - Set up a backup target.
- Create two volumes and write data in the
1st cluster
. Then create volumes' backups. - Restore two backup files as DR volumes in the
2nd cluster
. - Modify the backup poll interval to a large value.
- Write more data for the volumes in the
1st cluster
, and create the2nd backup
. - Activate one of the DR volumes in the
2nd cluster
. Then verify the data
Test Method 2 refer:#7947 (comment)
Result Passed
- 1. Test Method 1.
- 2. Test Method 2
- 3. require/manual-test-plan : longhorn/longhorn-tests#1732
from longhorn.
Related Issues (20)
- [TASK] Reference Architecture and Sizing Guidelines for Longhorn v1.7.x HOT 1
- [TEST] Investigate accessing lab behind vpn
- [BACKPORT][v1.6.3][IMPROVEMENT] System restore unable to restore volume with backing image HOT 1
- [BUG] Longhorn cifs backups cannot find credentials HOT 9
- [DOC] Incorrect and invalid links HOT 1
- Expanding the volume through UI but not reflecting it in backend. HOT 1
- [TEST][BUG] system restore stuck because of the volume/PV/PVC restoration
- [BACKPORT][v1.6.3][IMPROVEMENT] Improve and simplify chart values.yaml HOT 1
- [BACKPORT][v1.5.6][IMPROVEMENT] Improve and simplify chart values.yaml HOT 1
- Longhorn 1.6.2 - pvc is not ready for workloads HOT 1
- [BUG] Failed to delete a v2 orphan replica
- [FEATURE] Automatically attach the volumes for trimming filesystem HOT 1
- [TEST][FEATURE] Automatically attach the volumes for trimming filesystem
- [BUG] Fresh RWX volume on a fresh cluster install fails to ever mount (dual stack, IPv6-first cluster)
- [UI][IMPROVEMENT] Tweak some minor UI issues
- [UI][FEATURE] Multiple backup stores support
- [BUG] Request for a guide to the longhorn metric
- [BUG] System backup failed because backup creation failed. HOT 1
- [BACKPORT][v1.6.3][IMPROVEMENT] BackingImage UI improvement
- [BUG] System Restore Stuck at Pending due to Tolerations not Applied HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from longhorn.