kvdo 6.2.7.17 storage read/write issue with kernel 6.0 from KVM VMs about kvdo HOT 2 CLOSED

Hi, Andrea. I apologize for the delay in getting back to you.

Our Fedora testing has started using the 6.0.12 kernel at this point, but that’s testing the newer 8.x branch; the 6.2 code is mainly tested on RHEL 8 and its 4.18-based kernels. So I wouldn’t suggest mixing them like this. Also, our testing tends to use file systems or direct I/O and not running VMs atop VDO.

All that said, I don’t recall seeing any problems that would likely result in what you’re seeing. If VDO was returning an error in the hypervisor environment, it should’ve logged something to that effect, and you said there were “no significant message[s]” from the hypervisor. So I suspect the SCSI errors are coming from some other source.

That doesn’t mean it couldn’t be related to the VDO configuration. I note that the “emulate 512 byte” option is disabled in your VDO device, yet lvremove (at least) was trying to do I/O of 512 bytes. Depending on the flags set in various places, I imagine it might be possible that this size difference could cause an error instead of a read-modify-write sequence.

There are a couple of things you could try to see what’s going on:
(1) use “losetup” with a big backing file and “--sector-size=4096” as storage for a VM to see if the I/O size is the problem
(2) recreate the VDO device with --emulate512=enabled

Depending what you’ve got stored in your VDO device, the losetup approach might be the easier way to figure out if the 4k block size is the actual issue. Unfortunately I don’t think the emulate512 option can be modified for an already-created device, at least in the version of the vdo script I looked at.

If it’s just a test system and you don’t mind hacking config files directly, you could try changing “logicalBlockSize” in /etc/vdoconf.yml from 4096 to 512, instead of recreating the VDO device. But getting that edit wrong could stop VDO from starting at all...

Any of these might give us some insights as to what’s going on with the SCSI errors reported from the guests. Where VDO is reported no problems at all, I doubt it’s anything internal to VDO, and I expect the data is still safe on the volume.

from kvdo.

If it’s just a test system and you don’t mind hacking config files directly, you could try changing “logicalBlockSize” in /etc/vdoconf.yml from 4096 to 512, instead of recreating the VDO device. But getting that edit wrong could stop VDO from starting at all...

Hi, just to let you know that probably vdo is not the origin of my problem: I've uninstalled (kmod + vdo) from the system, but problems persist, so it should be something else.

Thanks for the feedback and ideas.

from kvdo.