akrog / cinderlib Goto Github PK

Hi,

I create a LPAR at IBM power. I know the wwpn of the lpar. How to attach a volume in IBM SVC that is created by cinderlib by cinderlib? Provided that cinderlib use a sqlite database to storage metadata persistence.

I refer cinderlib connections docs. I think I need to do three steps, but I have some questions for the steps.
1.As a host connect to volume by fibre-channel, how to get its connector information? According to the sample in docs, I get below error exception. “10.10.2.99” is my target host that need to attach volume.

connector_dict = cl.get_connector_properties('sudo', '10.10.2.99', True, False)
Traceback (most recent call last):
File "", line 1, in
File "/usr/lib/python2.7/site-packages/os_brick/utils.py", line 137, in trace_logging_wrapper
return f(*args, **kwargs)
File "/usr/lib/python2.7/site-packages/os_brick/initiator/connector.py", line 256, in get_connector_properties
enforce_multipath=enforce_multipath,
File "/usr/lib/python2.7/site-packages/os_brick/initiator/connectors/base.py", line 56, in get_connector_properties
execute=kwargs.get('execute')))
File "/usr/lib/python2.7/site-packages/os_brick/initiator/linuxscsi.py", line 161, in is_multipath_running
run_as_root=True, root_helper=root_helper)
File "/usr/lib/python2.7/site-packages/os_brick/privileged/rootwrap.py", line 169, in execute
return execute_root(*cmd, **kwargs)
File "/usr/lib/python2.7/site-packages/oslo_privsep/priv_context.py", line 204, in _wrap
self.start()
File "/usr/lib/python2.7/site-packages/oslo_privsep/priv_context.py", line 215, in start
channel = daemon.RootwrapClientChannel(context=self)
File "/usr/lib/python2.7/site-packages/oslo_privsep/daemon.py", line 327, in init
raise FailedToDropPrivileges(msg)
oslo_privsep.daemon.FailedToDropPrivileges: privsep helper command exited non-zero (1)

2.If we have known the LPAR wwpn value, whether or not we can directly create a connector dictionary and append the wwpn value to the dictionary? If yes, what is key of wwpn?

3.I mock a connector_info to get conn_info, but I get below exception.

connector_info
{'wwpn': 'c0507605ff190012 c0507605ff190013', 'wwpns': 'c0507605ff190012 c0507605ff190013', 'host': '10.10.2.99', 'target_wwn': 'c0507605ff190012 c0507605ff190013', u'target_wwpn': u'c0507605ff190012'}
vol.connect(connector_info, attached_host=1)
Traceback (most recent call last):
File "", line 1, in
File "/usr/lib/python2.7/site-packages/cinderlib/objects.py", line 562, in connect
self._raise_with_resource()
File "/usr/lib/python2.7/site-packages/cinderlib/objects.py", line 222, in _raise_with_resource
six.reraise(*exc_info)
File "/usr/lib/python2.7/site-packages/cinderlib/objects.py", line 553, in connect
conn = Connection.connect(self, connector_dict, **ovo_fields)
File "/usr/lib/python2.7/site-packages/cinderlib/objects.py", line 618, in connect
volume._ovo, connector)
File "/usr/lib/python2.7/site-packages/cinder/zonemanager/utils.py", line 80, in decorator
conn_info = initialize_connection(self, *args, **kwargs)
File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_fc.py", line 123, in initialize_connection
return _do_initialize_connection_locked()
File "/usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py", line 271, in inner
return f(*args, **kwargs)
File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_fc.py", line 122, in _do_initialize_connection_locked
return self._do_initialize_connection(volume, connector)
File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_fc.py", line 142, in _do_initialize_connection
host_name = self.helpers.get_host_from_connector(connector)
File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 874, in get_host_from_connector
host_name = None
File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 278, in lsfabric
msg = (('Must pass wwpn or host to lsfabric.'))
File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 165, in run_ssh_info
raw = self._run_ssh(ssh_cmd)
File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 161, in _run_ssh
raise exception.VolumeBackendAPIException(data=msg)
cinder.exception.VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:
command: ['svcinfo', 'lsfabric', '-delim', '!', '-wwpn', 'c']
stdout:
stderr: Error running SSH command.

As the error message “Must pass wwpn or host to lsfabric.”, I think that the cause of the issue is the incorrectly wwpn value. I have tried various “wwpn” key at connector_info. Don’t you know the wwpn key?

I know there some questions have exceeded cinderlib. I don’t have development experience for cinder and os-brick . I don’t know how to get the knowledge quickly. If you would give some help or quick refer doc. I will be very grateful.

Best regards
Ray

I refrence "Resource tracking" session in the cinderlib doc. I think cinderlib could upload all of volume instances to track by Backend.refresh() but the actualy result is not that.
Is my understand for the doc right? Thank you

The below is the detail for the test:
I create two volumes by create_volume and then I run backend.refresh(). I check the volumes again. The volumes are two yet. The original IBM SVC columes don't upload yet.

Test enviroment:
Cinderlib version: v0.3.9
Cinder release: Pike
Storage: IBM SVC V7000
Versions: Unknown
Connection type: FC

Test steps and result:

lvm.volumes
[<cinderlib.Volume object a7847ff7-aaa5-411f-b6dc-21626df666f4 on backend svc1234>, <cinderlib.Volume object c00e8221-10c8-471d-8c77-e88faeb064d2 on backend svc1234>]
lvm.refresh()
lvm.volumes
[<cinderlib.Volume object a7847ff7-aaa5-411f-b6dc-21626df666f4 on backend svc1234>, <cinderlib.Volume object c00e8221-10c8-471d-8c77-e88faeb064d2 on backend svc1234>]

refrence cinderlib doc:
https://docs.openstack.org/cinderlib/latest/topics/tracking.html

Thanks
Ray

Hi Gorka,

Very good demo. I used akrog/cinderlib and container to have a try. Instead of vagrant, I did this in a rhel7 vm running on OpenStack with OCP 3.10 deployed. Considering it's a container, I think it's independent of platform.
It seems failed to pass the volume_group argument. I checked its Dockerfile and cinder 3.5.0 has been installed in the container.

Here is my steps:

[root@cnv-executor-qwang-814-ds-master1 ~]# docker images | grep docker.io/akrog
docker.io/akrog/ember-csi                                                               latest              53f912845fb2        19 hours ago        753 MB
docker.io/akrog/cinderlib                                                               latest              b4abe9383d40        20 hours ago        662 MB

[root@cnv-executor-qwang-814-ds-master1 ~]# docker run --name=cinderlib --privileged --net=host   -v /etc/iscsi:/etc/iscsi   -v /dev:/dev   -v /etc/lvm:/etc/lvm   -v /var/lock/lvm:/var/lock/lvm   -v /lib/modules:/lib/modules   -v /run/udev:/run/udev   -v /etc/localtime:/etc/localtime   -it akrog/cinderlib python
Python 2.7.5 (default, Jul 13 2018, 13:06:57)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-28)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import cinderlib as cl
>>> from pprint import pprint as pp
>>> cl.setup(output_all_backend_info=True)
>>> lvm = cl.Backend(volume_driver='cinder.volume.drivers.lvm.LVMVolumeDriver',
...                  volume_group='cinder-volumes',
...                  iscsi_protocol='iscsi',
...                  iscsi_helper='lioadm',
...                  volume_backend_name='lvm_iscsi')
Traceback (most recent call last):
  File "<stdin>", line 5, in <module>
  File "/usr/lib/python2.7/site-packages/cinderlib/cinderlib.py", line 73, in __init__
    self.driver.check_for_setup_error()
  File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/lvm.py", line 313, in check_for_setup_error
    self.configuration.lvm_suppress_fd_warnings))
  File "/usr/lib/python2.7/site-packages/cinder/brick/local_dev/lvm.py", line 106, in __init__
    if self._vg_exists() is False:
  File "/usr/lib/python2.7/site-packages/cinder/brick/local_dev/lvm.py", line 139, in _vg_exists
    run_as_root=True)
  File "/usr/lib/python2.7/site-packages/os_brick/executor.py", line 52, in _execute
    result = self.__execute(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 126, in execute
    return processutils.execute(*cmd, **kwargs)
  File "/usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py", line 424, in execute
    cmd=sanitized_cmd)
oslo_concurrency.processutils.ProcessExecutionError: Unexpected error while running command.
Command: env LC_ALL=C vgs --noheadings -o name cinder-volumes
Exit code: 5
Stdout: u''
Stderr: u'  WARNING: Failed to connect to lvmetad. Falling back to device scanning.\n  Volume group "cinder-volumes" not found\n  Cannot process volume group cinder-volumes\n'
>>>

If I used another image akrog/ember-csi, ran this container with python entry, executed the python code and got the same error as above. If I entered the container with/without default CMD ember-csi, I got the following message.

[root@cnv-executor-qwang-814-ds-master1 ~]# docker run --name=ember-csi-1 --privileged --net=host   -v /etc/iscsi:/etc/iscsi   -v /dev:/dev   -v /etc/lvm:/etc/lvm   -v /var/lock/lvm:/var/lock/lvm   -v /lib/modules:/lib/modules   -v /run/udev:/run/udev   -v /etc/localtime:/etc/localtime   -it docker.io/akrog/ember-csi ember-csi
Missing required backend configuration

[root@cnv-executor-qwang-814-ds-master1 ~]# docker run --name=ember-csi-2 --privileged --net=host   -v /etc/iscsi:/etc/iscsi   -v /dev:/dev   -v /etc/lvm:/etc/lvm   -v /var/lock/lvm:/var/lock/lvm   -v /lib/modules:/lib/modules   -v /run/udev:/run/udev   -v /etc/localtime:/etc/localtime   -it docker.io/akrog/ember-csi
Missing required backend configuration

What kind of backend should be configured? Did I misunderstand something? Could you please have a look? Thanks.

I have executed some volume test(ceate, clone, snapthot etc).

I find if cinderlib don't use the database persistence plugins. It will cause a ssh running exception when it run the function firstly. The exception will disappear if run the same command again. Change the other methods, the same issue will show again.

If cinderlib set the database persistence plugin, the above qustion will not happen.

The below is my test eviroment and error messages.

Test enviroment:
Cinderlib version: v0.3.9
Cinder release: Pike
Storage: IBM SVC V7000
Versions: Unknown
Connection type: FC

Error messages:
vol = lvm.create_volume(size=1)
Traceback (most recent call last):
File "", line 1, in
File "/usr/lib/python2.7/site-packages/cinderlib/cinderlib.py", line 156, in create_volume
vol.create()
File "/usr/lib/python2.7/site-packages/cinderlib/objects.py", line 446, in create
self._raise_with_resource()
File "/usr/lib/python2.7/site-packages/cinderlib/objects.py", line 222, in _raise_with_resource
six.reraise(*exc_info)
File "/usr/lib/python2.7/site-packages/cinderlib/objects.py", line 439, in create
model_update = self.backend.driver.create_volume(self._ovo)
File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2520, in create_volume
pool = utils.extract_host(volume['host'], 'pool')
File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 742, in select_io_group
% {'iogrp': opts['iogrp'],
File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 719, in get_vdisk_count_by_io_group
def get_vdisk_count_by_io_group(self):
File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 224, in lsiogrp
def lsiogrp(self):
File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 163, in run_ssh_info
def run_ssh_info(self, ssh_cmd, delim='!', with_header=False):
File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 159, in _run_ssh
'err': e.stderr})
cinder.exception.VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: CLI Exception output:
command: ['svcinfo', 'lsiogrp', '-delim', '!']
stdout:
stderr: Error running SSH command.

I am try cinderlib for IBM svc v7000, but I get below error messages. Would you help how to resolve the issue or give some hint? Thank you very much.

Test enviroment:
Cinderlib version: v0.3.9
Cinder release: Pike
Storage: IBM SVC V7000
Versions: Unknown
Connection type: FC

Test code:
import cinderlib as cl
lvm = cl.Backend(volume_driver='cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc.StorwizeSVCFCDriver',
san_ip='...',
san_login='superuser',
san_password='********',
storwize_svc_volpool_name='mdiskgrp1',
volume_backend_name='svc1234')

Error messages:
Traceback (most recent call last):
File "mytest.py", line 11, in
volume_backend_name='svc1234')
File "/usr/lib/python2.7/site-packages/cinderlib/cinderlib.py", line 86, in init
self.driver.do_setup(objects.CONTEXT)
File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/ibm/storwize_svc/storwize_svc_common.py", line 2262, in do_setup
volumes = objects.VolumeList.get_all_by_host(admin_context, self.host)
File "/usr/lib/python2.7/site-packages/cinder/objects/volume.py", line 613, in get_all_by_host
volumes = db.volume_get_all_by_host(context, host, filters)
File "/usr/lib/python2.7/site-packages/cinder/db/api.py", line 274, in volume_get_all_by_host
return IMPL.volume_get_all_by_host(context, host, filters=filters)
AttributeError: 'DB' object has no attribute 'volume_get_all_by_host'

Configuration in cinder.conf
[svc1234]
volume_driver=cinder.volume.drivers.ibm.storwize_svc.storwize_svc_fc.StorwizeSVCFCDriver
san_ip=...
san_login=superuser
san_password=*********
storwize_svc_volpool_name=mdiskgrp1
volume_backend_name=svc1234

When attaching using the RBD backend we get the following error:

Traceback (most recent call last):
	  File "/usr/lib/python2.7/site-packages/ember_csi/ember_csi.py", line 128, in dolog
	    result = f(self, request, context)
	  File "/usr/lib/python2.7/site-packages/ember_csi/ember_csi.py", line 169, in checker
	    return f(self, request, context)
	  File "/usr/lib/python2.7/site-packages/ember_csi/ember_csi.py", line 211, in wrapper
	    return func(self, request, context)
	  File "/usr/lib/python2.7/site-packages/ember_csi/ember_csi.py", line 858, in NodeStageVolume
	    conn.attach()
	  File "/usr/lib/python2.7/site-packages/cinderlib/objects.py", line 715, in attach
	    device = self.connector.connect_volume(self.conn_info['data'])
	  File "/usr/lib/python2.7/site-packages/nos_brick/__init__.py", line 32, in connect_volume
	    self._execute('which', 'rbd')
	  File "/usr/lib/python2.7/site-packages/os_brick/executor.py", line 52, in _execute
	    result = self.__execute(*args, **kwargs)
	  File "/usr/lib/python2.7/site-packages/os_brick/privileged/rootwrap.py", line 143, in custom_execute
	    on_completion=on_completion, *cmd, **kwargs)
	  File "/usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py", line 391, in execute
	    env=env_variables)
	  File "/usr/lib/python2.7/site-packages/eventlet/green/subprocess.py", line 58, in __init__
	    subprocess_orig.Popen.__init__(self, args, 0, *argss, **kwds)
	  File "/usr/lib64/python2.7/subprocess.py", line 711, in __init__
	    errread, errwrite)
	  File "/usr/lib64/python2.7/subprocess.py", line 1327, in _execute_child
	    raise child_exception
	OSError: [Errno 2] No such file or directory

Which is caused by the new centos7 image not having the which command.

When detaching an RBD volume (using NBD) from within a container we get this error:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/site-packages/cinderlib/objects.py", line 528, in detach
    conn.detach(force, ignore_errors, exc)
  File "/usr/lib/python2.7/site-packages/cinderlib/objects.py", line 803, in detach
    ignore_errors=ignore_errors)
  File "/usr/lib/python2.7/site-packages/cinderlib/nos_brick.py", line 137, in disconnect_volume
    real_dev_path = os.path.realpath(dev_path)
  File "/usr/lib64/python2.7/posixpath.py", line 367, in realpath
    path, ok = _joinrealpath('', filename, {})
  File "/usr/lib64/python2.7/posixpath.py", line 373, in _joinrealpath
    if isabs(rest):
  File "/usr/lib64/python2.7/posixpath.py", line 61, in isabs
    return s.startswith('/')
AttributeError: 'NoneType' object has no attribute 'startswith'

Helen Walsh tested cinderlib with VMAX and reported success for Pike, Queens, and Rocky Cinder releases.

She had to modify the assertEqual for sizes as the storage provides slightly bigger sizes due to cylinder sizes. (Support for non perfect size matches has been added)

For Pike* she used:

• Configuration file:

     logs: false
     venv_sudo: false
     size_precision: 2
     backends:
         - image_volume_cache_enabled: True
           volume_clear: zero
           volume_backend_name: VMAX_ISCSI_DIAMOND
           volume_driver: cinder.volume.drivers.dell_emc.vmax.iscsi.VMAXISCSIDrive

• File /etc/cinder/cinder_dell_emc_config.xml:

     <?xml version="1.0" encoding="UTF-8"?>
     <EMC>
       <RestServerIp>w.x.y.z</RestServerIp>
       <RestServerPort>8443</RestServerPort>
       <RestUserName>username</RestUserName>
       <RestPassword>toomanysecrets</RestPassword>
       <Array>000197800128</Array>
       <PortGroups>
         <PortGroup>os-iscsi-pg</PortGroup>
       </PortGroups>
       <SRP>SRP_1</SRP>
       <ServiceLevel>Diamond</ServiceLevel>
       <Workload>none</Workload>
       <SSLVerify>/opt/stack/localhost.domain.com.pem</SSLVerify>
     </EMC>

For Queens and Rocky she used:

   logs: false
   venv_sudo: false
   size_precision: 2
   backends:
       - image_volume_cache_enabled: True
         volume_clear: zero
         volume_backend_name: VMAX_ISCSI_DIAMOND
         volume_driver: cinder.volume.drivers.dell_emc.vmax.iscsi.VMAXISCSIDriver
         san_ip: w.x.y.z
         san_rest_port: 8443
         san_login: user
         san_password: toomanysecrets
         vmax_srp: SRP_1
         vmax_array: 000197800128
         vmax_port_groups: [os-iscsi-pg]

John Griffith (@j-griffith) reported success on running cinderlib with SolidFire.

Tests were done with a modified version of v0.1.0 to pass the project_id and the user_id, which were later added to cinderlib.

Configuration used was:

logs: false
venv_sudo: true
backends:
    - volume_backend_name: solidfire
      volume_driver: cinder.volume.drivers.solidfire.SolidFireDriver
      san_ip: 192.168.1.4
      san_login: admin
      san_password: admin_password
      sf_allow_template_caching = false
      image_volume_cache_enabled = True
      volume_clear = zero