When using a setup which cinder is a backend for glance, when trying to create vms from boot volumes, we get the following:
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server [req-375bce64-cf8a-49a8-aebe-b443dc5d9223 d7a95eca1fd7db54051a82eb1f898b36489bc2302e172d843ebab07fcf2fba7e 0413a906e8314a26a434b42f3134d1c5 - - -] Exception during message handling: cinder.exception.ImageCopyFailure: Failed to copy image to volume: Privsep daemon failed to start
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server File "/var/lib/openstack/lib/python3.8/site-packages/cinder/volume/volume_utils.py", line 1152, in copy_image_to_volume
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server driver.copy_image_to_volume(
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server File "/var/lib/openstack/lib/python3.8/site-packages/cinder/volume/driver.py", line 859, in copy_image_to_volume
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server self._copy_image_data_to_volume(
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server File "/var/lib/openstack/lib/python3.8/site-packages/cinder/volume/driver.py", line 878, in _copy_image_data_to_volume
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server properties = volume_utils.brick_get_connector_properties(
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server File "/var/lib/openstack/lib/python3.8/site-packages/cinder/volume/volume_utils.py", line 1323, in brick_get_connector_properties
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server return connector.get_connector_properties(root_helper,
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server File "/var/lib/openstack/lib/python3.8/site-packages/os_brick/utils.py", line 141, in trace_logging_wrapper
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server return f(*args, **kwargs)
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server File "/var/lib/openstack/lib/python3.8/site-packages/os_brick/initiator/connector.py", line 232, in get_connector_properties
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server connector.get_connector_properties(
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server File "/var/lib/openstack/lib/python3.8/site-packages/os_brick/initiator/connectors/iscsi.py", line 68, in get_connector_properties
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server initiator = iscsi.get_initiator()
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server File "/var/lib/openstack/lib/python3.8/site-packages/os_brick/initiator/connectors/iscsi.py", line 990, in get_initiator
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server lines, _err = self._execute('cat', file_path, run_as_root=True,
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server File "/var/lib/openstack/lib/python3.8/site-packages/os_brick/executor.py", line 52, in _execute
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server result = self.__execute(*args, **kwargs)
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server File "/var/lib/openstack/lib/python3.8/site-packages/os_brick/privileged/rootwrap.py", line 172, in execute
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server return execute_root(*cmd, **kwargs)
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server File "/var/lib/openstack/lib/python3.8/site-packages/oslo_privsep/priv_context.py", line 246, in _wrap
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server self.start()
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server File "/var/lib/openstack/lib/python3.8/site-packages/oslo_privsep/priv_context.py", line 258, in start
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server channel = daemon.RootwrapClientChannel(context=self)
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server File "/var/lib/openstack/lib/python3.8/site-packages/oslo_privsep/daemon.py", line 383, in __init__
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server super(RootwrapClientChannel, self).__init__(sock)
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server File "/var/lib/openstack/lib/python3.8/site-packages/oslo_privsep/daemon.py", line 198, in __init__
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server self.exchange_ping()
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server File "/var/lib/openstack/lib/python3.8/site-packages/oslo_privsep/daemon.py", line 211, in exchange_ping
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server raise FailedToDropPrivileges(msg)
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server oslo_privsep.daemon.FailedToDropPrivileges: Privsep daemon failed to start
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server During handling of the above exception, another exception occurred:
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server File "/var/lib/openstack/lib/python3.8/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message)
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server File "/var/lib/openstack/lib/python3.8/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args)
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server File "/var/lib/openstack/lib/python3.8/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args)
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server File "<decorator-gen-751>", line 2, in create_volume
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server File "/var/lib/openstack/lib/python3.8/site-packages/cinder/objects/cleanable.py", line 208, in wrapper
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server result = f(*args, **kwargs)
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server File "/var/lib/openstack/lib/python3.8/site-packages/cinder/volume/manager.py", line 772, in create_volume
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server _run_flow()
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server File "/var/lib/openstack/lib/python3.8/site-packages/cinder/volume/manager.py", line 764, in _run_flow
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server flow_engine.run()
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server File "/var/lib/openstack/lib/python3.8/site-packages/taskflow/engines/action_engine/engine.py", line 247, in run
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server for _state in self.run_iter(timeout=timeout):
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server File "/var/lib/openstack/lib/python3.8/site-packages/taskflow/engines/action_engine/engine.py", line 340, in run_iter
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server failure.Failure.reraise_if_any(er_failures)
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server File "/var/lib/openstack/lib/python3.8/site-packages/taskflow/types/failure.py", line 339, in reraise_if_any
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server failures[0].reraise()
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server File "/var/lib/openstack/lib/python3.8/site-packages/taskflow/types/failure.py", line 346, in reraise
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server six.reraise(*self._exc_info)
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server File "/var/lib/openstack/lib/python3.8/site-packages/six.py", line 703, in reraise
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server raise value
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server File "/var/lib/openstack/lib/python3.8/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server result = task.execute(**arguments)
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server File "/var/lib/openstack/lib/python3.8/site-packages/cinder/volume/flows/manager/create_volume.py", line 1162, in execute
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server model_update = self._create_from_image(context,
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server File "/var/lib/openstack/lib/python3.8/site-packages/cinder/utils.py", line 614, in _wrapper
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server return r.call(f, *args, **kwargs)
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server File "/var/lib/openstack/lib/python3.8/site-packages/tenacity/__init__.py", line 411, in call
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server return self.__call__(*args, **kwargs)
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server File "/var/lib/openstack/lib/python3.8/site-packages/tenacity/__init__.py", line 423, in __call__
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server do = self.iter(retry_state=retry_state)
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server File "/var/lib/openstack/lib/python3.8/site-packages/tenacity/__init__.py", line 360, in iter
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server return fut.result()
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.8/concurrent/futures/_base.py", line 437, in result
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server return self.__get_result()
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.8/concurrent/futures/_base.py", line 389, in __get_result
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server raise self._exception
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server File "/var/lib/openstack/lib/python3.8/site-packages/tenacity/__init__.py", line 426, in __call__
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server result = fn(*args, **kwargs)
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server File "/var/lib/openstack/lib/python3.8/site-packages/cinder/volume/flows/manager/create_volume.py", line 1058, in _create_from_image
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server model_update = self._create_from_image_cache_or_download(
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server File "/var/lib/openstack/lib/python3.8/site-packages/cinder/volume/flows/manager/create_volume.py", line 950, in _create_from_image_cache_or_download
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server model_update = self._create_from_image_download(
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server File "/var/lib/openstack/lib/python3.8/site-packages/cinder/volume/flows/manager/create_volume.py", line 766, in _create_from_image_download
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server volume_utils.copy_image_to_volume(self.driver, context, volume,
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server File "/var/lib/openstack/lib/python3.8/site-packages/cinder/volume/volume_utils.py", line 1169, in copy_image_to_volume
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server raise exception.ImageCopyFailure(reason=ex)
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server cinder.exception.ImageCopyFailure: Failed to copy image to volume: Privsep daemon failed to start
openstack/cinder-volume-f478fc9f4-cxnpd[cinder-volume]: 2022-09-14 13:41:28.442 9 ERROR oslo_messaging.rpc.server