naudit / pystorcli2 Goto Github PK
View Code? Open in Web Editor NEWPython library provides wrapper around storcli tool to manage and control LSI MegaRAID controllers.
License: BSD 3-Clause "New" or "Revised" License
Python library provides wrapper around storcli tool to manage and control LSI MegaRAID controllers.
License: BSD 3-Clause "New" or "Revised" License
Requested by @ulmitov
The design pattern - pystorcli uses singleton when creating the StorCLI object. while pysmart just creates it once and then passes to all rest of objects (somewhat a factory design). I think the factory is much better.
Hi,
Kindly asking to add a new attribute to Drive object - Drive.smart_obj (equals to None).
This is to be able to update it later with current drive's pysmart drive object (or any other object).
pip install pystorcli2
Defaulting to user installation because normal site-packages is not writeable
Collecting pystorcli2
Downloading PyStorCLI2-0.5.0-py3-none-any.whl (22 kB)
Installing collected packages: pystorcli2
Successfully installed pystorcli2-0.5.0
In [1]: import pystorcli2
In [3]: pystorcli2.Controllers()
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
Cell In [3], line 1
----> 1 pystorcli2.Controllers()
AttributeError: module 'pystorcli2' has no attribute 'Controllers'
The following exception is being raised if command's rc is non zero:
In [4]: c['/c0'].has_foreign_configurations()
ERROR:root:RC=59 returned from /usr/sbin/storcli64 /c0 /fall show J
---------------------------------------------------------------------------
CalledProcessError Traceback (most recent call last)
File ~/Desktop/venvs/dev/lib/python3.9/site-packages/pystorcli/storcli.py:225, in StorCLI.run(self, args, stdout, stderr, allow_error_codes, **kwargs)
224 if ret.returncode != 0:
--> 225 raise subprocess.CalledProcessError(
226 ret.returncode, cmd, ret.stdout, ret.stderr)
227 if self.cache_enable:
CalledProcessError: Command '['/usr/sbin/storcli64', '/c0', '/fall', 'show', 'J']' returned non-zero exit status 59.
During handling of the above exception, another exception occurred:
StorCliRunTimeError Traceback (most recent call last)
Cell In[4], line 1
----> 1 c['/c0'].has_foreign_configurations()
File ~/Desktop/venvs/dev/lib/python3.9/site-packages/pystorcli/controller/__init__.py:435, in Controller.has_foreign_configurations(self, securitykey)
432 args.append(f'securitykey={securitykey}')
434 try:
--> 435 fc_data = common.response_data(self._run(args))
436 fcs = 0
438 if 'Total foreign Drive Groups' in fc_data:
File ~/Desktop/venvs/dev/lib/python3.9/site-packages/pystorcli/controller/__init__.py:83, in Controller._run(self, args, allow_error_codes, **kwargs)
81 args = args[:]
82 args.insert(0, self._name)
---> 83 return self._storcli.run(args, allow_error_codes=allow_error_codes, **kwargs)
File ~/Desktop/venvs/dev/lib/python3.9/site-packages/pystorcli/storcli.py:252, in StorCLI.run(self, args, stdout, stderr, allow_error_codes, **kwargs)
250 raise exc.StorCliRunTimeout(err)
251 except subprocess.SubprocessError as err:
--> 252 raise exc.StorCliRunTimeError(err)
StorCliRunTimeError: Command '/usr/sbin/storcli64 /c0 /fall show J' returned with non-zero exit status 59:
This was the output:
12:28:52 [ERROR] RC=59 returned from /usr/sbin/storcli64 /c0 /fall show J
12:28:52 [ERROR] STDERR:
STDOUT:{
"Controllers":[
{
"Command Status" : {
"CLI Version" : "007.2309.0000.0000 Sep 16, 2022",
"Operating system" : "Linux 5.4.0-99-generic",
"Controller" : 0,
"Status" : "Failure",
"Description" : "Incomplete foreign configuration"
},
"Response Data" : {
"Total Foreign PDs" : 16,
"Total Locked Foreign PDs" : 0
}
}
]
}
Hi,
Regarding controller.create_vd,
For raid1 and raid10 must specify PDperArray=2,
otherwise the add vd command will fail with:
"operation not possible for current RAID level, Cannot create configuration with 1 span".
Can you please add a parameter to this method so it can get some additional params to append to the add vd command ?
Probably the additional params will be needed in other methods also.
Hi,
Please add "JBOD'" state to this method:
@property
def state(self):
"""Get/Set drive state
One of the following states can be set (str):
online - changes the drive state to online
offline - changes the drive state to offline
missing - marks a drive as missing
good - changes the drive state to unconfigured good
jbod - sets the drive state to JBOD
Returns:
(str):
dhs - dedicated hotspare to some virtual drive
ghs - global hotspare
bad - bad drive
good - unconfigured good
online - already in virtual drive with good state
offline - already in virtual drive with bad state
"""
args = [
'show'
]
state = self._response_properties(self._run(args))['State']
if state == 'DHS':
return 'dhs'
elif state == 'UBad':
return 'bad'
elif state == 'Onln':
return 'online'
elif state == 'Offln':
return 'offline'
elif state == 'GHS':
return 'ghs'
return 'good'
Hi,
Having this kind of controller:
COMMAND: storcli64 /c0 show J
RC:
0
STDOUT:
{
"Controllers":[
{
"Command Status" : {
"CLI Version" : "007.2309.0000.0000 Sep 16, 2022",
"Operating system" : "Linux 5.4.0-99-generic",
"Controller" : 0,
"Status" : "Failure",
"Description" : "None",
"Detailed Status" : [
{
"Ctrl" : 0,
"Status" : "Failed",
"ErrCd" : 59,
"ErrMsg" : "Incomplete foreign configuration"
}
]
},
"Response Data" : {
"Product Name" : "AVAGO JBOD",
Trying to init it with pystorcli:
In [2]: import pystorcli as pp
In [3]: c=pp.controller.Controllers()
In [4]: c.ids
Out[4]: [0]
In [5]: x=c.get_ctl(0)
---------------------------------------------------------------------------
StorCliMissingError Traceback (most recent call last)
Cell In[5], line 1
----> 1 x=c.get_ctl(0)
File /usr/local/lib/python3.8/dist-packages/pystorcli/controller/__init__.py:478, in Controllers.get_ctl(self, ctl_id)
468 def get_ctl(self, ctl_id: int) -> Optional[Controller]:
469 """Get controller object by id
470
471 Args:
(...)
476 (:obj:Controller): controller object
477 """
--> 478 for ctl in self:
479 if ctl.id == ctl_id:
480 return ctl
File /usr/local/lib/python3.8/dist-packages/pystorcli/controller/__init__.py:457, in Controllers._ctls(self)
454 @ property
455 def _ctls(self):
456 for ctl_id in self._ctl_ids:
--> 457 yield Controller(ctl_id=ctl_id, binary=self._binary)
File /usr/local/lib/python3.8/dist-packages/pystorcli/controller/__init__.py:71, in Controller.__init__(self, ctl_id, binary)
68 self._storcli = StorCLI(binary)
69 self._name = '/c{0}'.format(self._ctl_id)
---> 71 self._exist()
File /usr/local/lib/python3.8/dist-packages/pystorcli/controller/__init__.py:85, in Controller._exist(self)
83 self._run(['show'])
84 except exc.StorCliCmdError:
---> 85 raise exc.StorCliMissingError(
86 self.__class__.__name__, self._name) from None
StorCliMissingError: Object 'Controller' doesnt exist: /c0
The exception is saying that object does not exist.
But the real exception should be:
In [9]: c._storcli.run(['/c0', 'show'])
---------------------------------------------------------------------------
StorCliCmdError Traceback (most recent call last)
Cell In[9], line 1
----> 1 c._storcli.run(['/c0', 'show'])
File /usr/local/lib/python3.8/dist-packages/pystorcli/storcli.py:184, in StorCLI.run(self, args, stdout, stderr, **kwargs)
182 try:
183 ret_json = json.loads(ret.stdout)
--> 184 self.check_response_status(cmd, ret_json)
185 ret.check_returncode()
186 if self.cache_enable:
File /usr/local/lib/python3.8/dist-packages/pystorcli/storcli.py:144, in StorCLI.check_response_status(cmd, out)
142 if cmd_status['Status'] == 'Failure':
143 if 'Detailed Status' in cmd_status:
--> 144 raise exc.StorCliCmdError(
145 cmd, "{0}".format(cmd_status['Detailed Status']))
146 else:
147 raise exc.StorCliCmdError(cmd, "{0}".format(cmd_status))
StorCliCmdError: Command '/usr/sbin/storcli64 /c0 show J' error: [{'Ctrl': 0, 'Status': 'Failed', 'ErrCd': 59, 'ErrMsg': 'Incomplete foreign configuration'}]
On the other hand, the controller exists and maybe pystorcli should create the object (although controller has status failed).
Since we can make it healthy:
In [10]: c._storcli.run(['/c0/fall', 'del'])
Out[10]:
{'Controllers': [{'Command Status': {'CLI Version': '007.2309.0000.0000 Sep 16, 2022',
'Operating system': 'Linux 5.4.0-99-generic',
'Controller': 0,
'Status': 'Success',
'Description': 'Operation on foreign configuration Succeeded'},
'Response Data': {'Total Foreign PDs': 0}}]}
In [11]: c._storcli.run(['/c0', 'show'])
Out[11]:
{'Controllers': [{'Command Status': {'CLI Version': '007.2309.0000.0000 Sep 16, 2022',
'Operating system': 'Linux 5.4.0-99-generic',
'Controller': 0,
'Status': 'Success',
'Description': 'None'},
'Response Data': {'Product Name': 'AVAGO JBOD',
But if pystorcli will not create an object this would not be possible.
Now the init is passing:
In [12]: x=c.get_ctl(0)
In [13]: x
Out[13]: <pystorcli.controller.Controller at 0x7f3601a33cd0>
Hi,
Currenlty the serial, model and other fields are not being formatted and have white spaces and different letter case.
Please apply space stripping and to upper() format.
One of purposes for this is also to have same formatting like in pySMART
Example:
In [9]: print(d['megaraid,23'].hba_obj.facts)
{'Drive /c0/e252/s3': [{'EID:Slt': '252:3', 'DID': 23, 'State': 'UGood', 'DG': '-', 'Size': '446.625 GB', 'Intf': 'SATA', 'Med': 'SSD', 'SED': 'N', 'PI': 'N', 'SeSz': '512B', 'Model': 'INTEL SSDSC2KB480G8', 'Sp': 'U', 'Type': '-'}], 'Drive /c0/e252/s3 - Detailed Information': {'Drive /c0/e252/s3 State': {'Shield Counter': 0, 'Media Error Count': 0, 'Other Error Count': 0, 'Drive Temperature': ' 24C (75.20 F)', 'Predictive Failure Count': 0, 'S.M.A.R.T alert flagged by drive': 'No'}, 'Drive /c0/e252/s3 Device attributes': {'SN': 'PHYF1100065U480BGN ', 'Manufacturer Id': 'ATA ', 'Model Number': 'INTEL SSDSC2KB480G8', 'NAND Vendor': 'NA', 'WWN': '55CD2E415339189F', 'Firmware Revision': 'XCV10132', 'Raw size': '447.130 GB [0x37e436b0 Sectors]', 'Coerced size': '446.625 GB [0x37d40000 Sectors]', 'Non Coerced size': '446.630 GB [0x37d436b0 Sectors]', 'Device Speed': '6.0Gb/s', 'Link Speed': '6.0Gb/s', 'NCQ setting': 'Enabled', 'Write Cache': 'N/A', 'Logical Sector Size': '512B', 'Physical Sector Size': '4 KB', 'Connector Name': 'Port 0 - 3 x1'}, 'Drive /c0/e252/s3 Policies/Settings': {'Enclosure position': '1', 'Connected Port Number': '8(path0) ', 'Sequence Number': 35, 'Commissioned Spare': 'No', 'Emergency Spare': 'No', 'Last Predictive Failure Event Sequence Number': 0, 'Successful diagnostics completion on': 'N/A', 'FDE Type': 'None', 'SED Capable': 'No', 'SED Enabled': 'No', 'Secured': 'No', 'Cryptographic Erase Capable': 'Yes', 'Sanitize Support': 'Not supported', 'Locked': 'No', 'Needs EKM Attention': 'No', 'PI Eligible': 'No', 'Certified': 'No', 'Wide Port Capable': 'No', 'Multipath': 'No', 'Port Information': [{'Port': 0, 'Status': 'Active', 'Linkspeed': '6.0Gb/s', 'SAS address': '0x4433221108000000'}]}, 'Inquiry Data': '40 00 ff 3f 37 c8 10 00 00 00 00 00 3f 00 00 00 00 00 00 00 48 50 46 59 31 31 30 30 36 30 55 35 38 34 42 30 4e 47 20 20 00 00 00 00 00 00 43 58 31 56 31 30 32 33 4e 49 45 54 20 4c 53 53 53 44 32 43 42 4b 38 34 47 30 20 38 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 01 80 00 40 00 2f 00 40 00 00 00 00 06 00 ff 3f 10 00 3f 00 10 fc fb 00 01 fd ff ff ff 0f 00 00 07 00 '}}
In [5]: d['megaraid,23'].hba_obj.serial
Out[5]: 'PHYF1100065U480BGN '
In [6]: d['megaraid,23'].hba_obj.model
Out[6]: 'intel ssdsc2kb480g8'
( in this code hba_obj is an added attribute to pySMART.Drive which holds the pystorcli.Drive object )
Hi,
filing another one.
Init of controllers fails on some cards on this line:
return [ctl['Ctl'] for ctl in common.response_data(out)['System Overview']]
The "System overview" appears to be "IT System Overview" sometimes.
Examples:
Output of HBA card:
CLI Version = 007.2309.0000.0000 Sep 16, 2022
Operating system = Linux 5.4.0-99-generic
Status Code = 0
Status = Success
Description = None
Number of Controllers = 1
Host Name = ubuntu20-cuda
Operating System = Linux 5.4.0-99-generic
StoreLib IT Version = 07.2400.0200.0100
StoreLib IR3 Version = 16.14-0
IT System Overview :
==================
---------------------------------------------------------------------------
Ctl Model AdapterType VendId DevId SubVendId SubDevId PCI Address
---------------------------------------------------------------------------
0 HBA 9400-16i SAS3416(B0) 0x1000 0xAC 0x1000 0x3000 00:3b:00:00
---------------------------------------------------------------------------
Output of RAID card:
CLI Version = 007.2309.0000.0000 Sep 16, 2022
Operating system = Linux 5.4.0-99-generic
Status Code = 0
Status = Success
Description = None
Number of Controllers = 1
Host Name = ubuntu20-cuda
Operating System = Linux 5.4.0-99-generic
System Overview :
===============
----------------------------------------------------------------------
Ctl Model Ports PDs DGs DNOpt VDs VNOpt BBU sPR DS EHS ASOs Hlth
----------------------------------------------------------------------
0 AVAGOJBOD 16 16 0 0 0 0 N/A On 1&2 Y 2 Opt
----------------------------------------------------------------------
Hi,
there is an exception when running some commands, for example:
In [4]: ctrl._run(['show', 'events', 'file=/root/raid_events.log'])
---------------------------------------------------------------------------
JSONDecodeError Traceback (most recent call last)
File ~/Desktop/venvs/dev/lib/python3.9/site-packages/pystorcli2/storcli.py:189, in StorCLI.run(self, args, stdout, stderr, **kwargs)
188 try:
--> 189 ret_json = json.loads(ret.stdout)
190 self.check_response_status(cmd, ret_json)
File /usr/lib/python3.9/json/__init__.py:346, in loads(s, cls, object_hook, parse_float, parse_int, parse_constant, object_pairs_hook, **kw)
343 if (cls is None and object_hook is None and
344 parse_int is None and parse_float is None and
345 parse_constant is None and object_pairs_hook is None and not kw):
--> 346 return _default_decoder.decode(s)
347 if cls is None:
File /usr/lib/python3.9/json/decoder.py:337, in JSONDecoder.decode(self, s, _w)
333 """Return the Python representation of ``s`` (a ``str`` instance
334 containing a JSON document).
335
336 """
--> 337 obj, end = self.raw_decode(s, idx=_w(s, 0).end())
338 end = _w(s, end).end()
File /usr/lib/python3.9/json/decoder.py:355, in JSONDecoder.raw_decode(self, s, idx)
354 except StopIteration as err:
--> 355 raise JSONDecodeError("Expecting value", s, err.value) from None
356 return obj, end
JSONDecodeError: Expecting value: line 1 column 1 (char 0)
During handling of the above exception, another exception occurred:
AttributeError Traceback (most recent call last)
Cell In[4], line 1
----> 1 a._run(['show', 'events', f'file=/root/raid_events.log'])
File ~/Desktop/venvs/dev/lib/python3.9/site-packages/pystorcli2/controller.py:228, in Controller._run(self, args, **kwargs)
226 args = args[:]
227 args.insert(0, self._name)
--> 228 return self._storcli.run(args, **kwargs)
File ~/Desktop/venvs/dev/lib/python3.9/site-packages/pystorcli2/storcli.py:197, in StorCLI.run(self, args, stdout, stderr, **kwargs)
194 return ret_json
195 except json.JSONDecodeError:
196 # :/
--> 197 err = re.search('(^.*)Storage.*Command.*$',
198 ret.stdout, re.MULTILINE | re.DOTALL).group(1)
199 raise exc.StorCliCmdError(cmd, err)
200 except subprocess.TimeoutExpired as err:
AttributeError: 'NoneType' object has no attribute 'group'
This command does not have a json format:
root@ubuntu20-cuda:~# storcli64 /c0 show events file=/root/raid_events.log J
CLI Version = 007.2309.0000.0000 Sep 16, 2022
Operating system = Linux 5.4.0-99-generic
Controller = 0
Status = Success
Description = None
Events = GETEVENTS
Controller Properties :
=====================
------------------------------------
Ctrl Status Method Value
------------------------------------
0 Success handleSuboption Events
------------------------------------
COMMAND: storcli64 /c0/e252/s15 set offline J
RC:
0
STDOUT:
{
"Controllers":[
{
"Command Status" : {
"CLI Version" : "007.2309.0000.0000 Sep 16, 2022",
"Operating system" : "Linux 5.4.0-99-generic",
"Controller" : 0,
"Status" : "Success",
"Description" : "Set Drive Offline Succeeded."
}
}
]
}
STDERR:
Start / End / Elapsed 15:40:25.739667 / 15:40:25.941269 / 0:00:00.201602
2023-03-09 15:40:25 [ERROR] [set_drive_state] has an error
Traceback (most recent call last):
File "/home/umit/dev/dev-tests/testsuites/functional/ssd/test_raid.py", line 358, in set_drive_state
d.state = 'offline'
File "/home/umit/.local/lib/python3.10/site-packages/pystorcli/drive.py", line 585, in state
return common.response_setter(self._run(args))
File "/home/umit/.local/lib/python3.10/site-packages/pystorcli/common.py", line 57, in response_setter
return response_cmd(data)['Detailed Status'][0]['Value']
KeyError: 'Detailed Status'
Hi,
Regarding this code in error.py
INVALID_STATUS = (
255, '', 'Invalid status - used for polling command completion.')
It is a a bit misleading, not sure if 255 is same for all cases but i got this one:
storcli64 /c0/e252/s7 set good J
{
"Controllers":[
{
"Command Status" : {
"CLI Version" : "007.2408.0000.0000 Nov 15, 2022",
"Operating system" : "Linux 5.15.0-25-generic",
"Controller" : 0,
"Status" : "Failure",
"Description" : "Set Drive Good Failed.",
"Detailed Status" : [
{
"Drive" : "/c0/e252/s7",
"Status" : "Failure",
"ErrCd" : 255,
"ErrMsg" : "Operation not allowed."
}
]
}
}
]
}
On a host without LSI cards:
File /usr/local/lib/python3.8/dist-packages/pystorcli/controller/__init__.py:554, in Controllers.ids(self)
550 @ property
551 def ids(self):
552 """(list of str): controllers id
553 """
--> 554 return self._ctl_ids
File /usr/local/lib/python3.8/dist-packages/pystorcli/controller/__init__.py:533, in Controllers._ctl_ids(self)
531 @ property
532 def _ctl_ids(self) -> List[int]:
--> 533 out = self._storcli.run(['show'], allow_error_codes=[
534 StorcliError.INCOMPLETE_FOREIGN_CONFIGURATION])
535 response = common.response_data(out)
537 if "Number of Controllers" in response and response["Number of Controllers"] == 0:
File /usr/local/lib/python3.8/dist-packages/pystorcli/storcli.py:210, in StorCLI.run(self, args, stdout, stderr, allow_error_codes, **kwargs)
208 # output in JSON format
209 cmd.append('J')
--> 210 cmd_cache_key = ''.join(cmd)
212 if self.cache_enable:
213 if cmd_cache_key in self.__response_cache:
TypeError: sequence item 0: expected str instance, bytes found
In [9]: !storcli64 /call show
CLI Version = 007.2309.0000.0000 Sep 16, 2022
Operating system = Linux 5.4.0-99-generic
Status = Failure
Description = No Controller found
Hi,
When setting drive.state= 'good' it fails (i don't have an output to paste right now)
since there is no 'good' state, only 'ugood' or 'unconfigured good'
while drive.state = 'offline''missing''online' works well, it makes a little confusion
Thanks
I'm encountering more and more raid cards that are only compatible with storcli2/percli2, which is problematic with the current library. For the most part the commands are roughly the same, but the output is different. What do we think is the best way to handle different output like this? It would be relatively simple if the keys were just different in name, but some are nested under varying levels of keys to get the same answer; ie
storcli64 output for getting virtual drive facts
"Response Data" : {
"VD0 Properties" : {
"Strip Size" : "256 KB",
storcli2 output for getting virtual drive facts
"Response Data" : {
"Virtual Drives" : [
{
"VD Info" : {
"DG/VD" : "0/1",
"TYPE" : "RAID6",
"State" : "Optl",
"Access" : "RW",
"CurrentCache" : "NR,AWB",
"DefaultCache" : "NR,AWB",
"Size" : "392.896 TiB",
"Name" : ""
},
"PDs" : [
{
"EID:Slt" : "355:6",
...
"Alt-EID" : "-"
}
],
"VD Properties" : {
"Strip Size" : "256 KiB",
"Block Size" : 4096,
"Number of Blocks" : 1054674616
Is it best to keep it in the same library? Create a new library? Probably goes along the lines the of the factory pattern conversation and pyarcconf.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.