emcecs / python-ecsclient Goto Github PK

https://www.emc.com/techpubs/api/ecs/v3-0-0-0/DashBoardRESTHandler_73d8e0cafab49a9f04b3c33704b0e781_overview.htm

Implement and test the following actions:

• Delete a RG
• Add a storage pool (varray) to a RG
• Remove a storage pool (varray) from a RG

Accodring API documentation dashboard can return filterted data with historical periond.

In current time library does not pass any parameters to request for filter or query

Implement the Node endpoint for versions 2.x, 3.0, and 3.1.

Check and add tests for the data store endpoint. Also make sure it can create new VDCs. Since it's not documented in the API, review how it's done in the CLI and ECS CE step 2 script.

Is there an equivalent Python Module to List objects in a given bucket ;
GET
retrieve object
retrieve object’s metadata
retrieve object’s ACLs

PUT, POST, DELETE method as well.

Create a Dockerfile to generate the Docker image with its dependencies.

Allow users to configure the client to get XML output from the API.

def set_acl(self):
    raise NotImplementedError()

Do we have this function added now? Please advise.

Depends on #12

    conf.api_client.management_user.create_local_user_info(name, mu_pass, **mu_dict)
  File "/usr/local/lib/python2.7/site-packages/ecsclient/common/user_management/management_user.py", line 42, in create_local_user_info
    return self.conn.post(url='vdc/users', json_payload=payload)
  File "/usr/local/lib/python2.7/site-packages/ecsclient/baseclient.py", line 111, in post
    return self._request(url, json_payload, http_verb='POST')
  File "/usr/local/lib/python2.7/site-packages/ecsclient/baseclient.py", line 160, in _request
    raise ECSClientException.from_response(req)
ecsclient.common.exceptions.ECSClientException: Unrecognized field "user" (Class com.emc.storageos.data.datasvcmodels.mgmtuserinfo.MgmtUserInfoCreate), not marked as ignorable
 at [Source: HttpInputOverHTTP@5254f7cd[c=95,s=STREAM]; line: 1, column: 35] (through reference chain: com.emc.storageos.data.datasvcmodels.mgmtuserinfo.MgmtUserInfoCreate["user"]) - Bad request body: https://192.168.2.220:4443/vdc/users 400 Bad Request  [first 60 chars of response] {"code":1013,"retryable":false,"description":"Bad request bo

For reference, the values passed to api_client.management_user.create_local_user_info(name, mu_pass, **mu_dict) are:

name = "user1"
mu_pass = "ChangeMe"
mu_dict = {'is_system_monitor': False, 'is_system_admin': False}

Let me know what other data you'd like to help troubleshooting.

Implement and test the following actions:

• Update a VDC
• Delete a VDC

E.g. requests>=2.18,<3.0

Asynchronous client.

Possible options:

• HTTPX
• AIOHTTP
• Tornado

API call: http://doc.isilon.com/ECS/3.1/API/UserSecretKeyService_createNewKeyForUser_5d1d9a3cb3acfef2f1a8320c35bba0dc_b2081f90369a6d96de7811240fd3b244_detail.html

Library method:
https://github.com/EMCECS/python-ecsclient/blob/master/ecsclient/common/user_management/secret_key.py#L57

Add an extra kwargs parameter to billing methods to allow users to set the unit (e.g. sizeunit='MB').

It would be extremely helpful for a set of ansible modules to be built off this codebase for use in automation.

For instance:
ecs_bucket
ecs_user
ecs_objectuser
ecs_vdc

Etc.

Create a command line utility to enable users to use this library from the terminal and automate it in shell scripts.

I am using a single logout() command, but the requests sent by the code I see two been issued. The problem is the first one succeed (200) and the second call fails (probably because the first 200).

The client.bucket.create('bucket') without any arguments appears to default to setting stale_allowed to false. My expectation would be that if an ADO setting is not specified in the bucket create process it should default to using the namespace level ADO setting, not enforcing ADO Disabled.

Return the API error codes and corresponding message back to the user inside the exception.

https://www.emc.com/techpubs/api/ecs/v3-0-0-0/ErrorCodes.htm

The API has changed in 3.1 and it fails to create data store given an IP address. It should instead use a node ID.

Error output:

> Adding Data Stores to Storage Pool:
>       vdc1/sp1/192.168.2.220
Traceback (most recent call last):
  File "/usr/local/bin/ecsconfig", line 11, in <module>
    load_entry_point('ecsdeploy==2.4.0', 'console_scripts', 'ecsconfig')()
  File "/usr/local/lib/python2.7/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python2.7/site-packages/click/core.py", line 1092, in invoke
    rv.append(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python2.7/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python2.7/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/click/decorators.py", line 64, in new_func
    return ctx.invoke(f, obj, *args[1:], **kwargs)
  File "/usr/local/lib/python2.7/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/bin/ecsconfig.py", line 637, in sp
    tasks = add_all()
  File "/usr/local/bin/ecsconfig.py", line 613, in add_all
    sp_tasks.extend(add_one(name))
  File "/usr/local/bin/ecsconfig.py", line 605, in add_one
    sp_tasks.append(sp_add_node(sp_id, node))
  File "/usr/local/bin/ecsconfig.py", line 582, in sp_add_node
    return conf.api_client.data_store.create(**kwargs)
  File "/usr/local/lib/python2.7/site-packages/ecsclient/common/provisioning/data_store.py", line 217, in create
    return self.conn.post('vdc/data-stores/commodity', json_payload=payload)
  File "/usr/local/lib/python2.7/site-packages/ecsclient/baseclient.py", line 120, in post
    return self._request(url, json_payload, http_verb='POST')
  File "/usr/local/lib/python2.7/site-packages/ecsclient/baseclient.py", line 170, in _request
    raise ECSClientException.from_response(req)
ecsclient.common.exceptions.ECSClientException: Usage of node IP address in place of nodeId is no longer supported - Api Parameter Node Id Invalid: https://192.168.2.220:4443/vdc/data-stores/commodity 400 Bad Request  [first 60 chars of response] {"code":1050,"retryable":false,"description":"Api Parameter

https://www.emc.com/techpubs/api/ecs/v3-0-0-0/AlertService_f9db649cacccfe426fd71fb475801da2_overview.htm

• GET /vdc/alerts
• PUT /vdc/alerts/{alertId}/acknowledgment
• GET /vdc/alerts/latest

Create a utilities class with functions for users to validate responses and other useful utilities.

Example for validating a response:

from ecsclient import util, schemas
util.is_valid_response(response, schemas.LICENSE)

EMC 3.0 API Spec and changelog: https://www.emc.com/techpubs/api/ecs/v3-0-0-0/index.htm

Keep backwards compatibility with ECS 2.x

ECS CE requires VDC Keystore feature to support VDC-level SSL certificate chains.

Related ECS CE issue: EMCECS/ECS-CommunityEdition#345

Currently the library validate the token by sending a HTTP request to token verification endpoint at every call, which is not necessary.

Fedor: who is supporting python client? Code in client:

        a = self.message
        if self.ecs_details not in a:
            a += ' - %s' % self.ecs_details
        if self.ecs_description not in a:
            a += ' - %s' % self.ecs_description
        b = ''
        if self.http_scheme:
            b += '%s://' % self.http_scheme
        if self.http_host:
            b += self.http_host
        if self.http_port:
            b += ':%s' % self.http_port
        if self.http_path:
            b += self.http_path
        if self.http_query:
            b += '?%s' % self.http_query
        if self.http_status:
            if b:
                b = '%s %s' % (b, self.http_status)
            else:
                b = str(self.http_status)
        if self.http_reason:
            if b:
                b = '%s %s' % (b, self.http_reason)
            else:
                b = '- %s' % self.http_reason
        if self.http_response_content:
            if len(self.http_response_content) <= 60:
                b += '   %s' % self.http_response_content
            else:
                b += '  [first 60 chars of response] %s' \
                     % self.http_response_content[:60]
        return b and '%s: %s' % (a, b) or a```
Fedor: https://github.com/EMCECS/python-ecsclient/blob/master/ecsclient/common/exceptions.py#L72
Fedor: It's creating new exceptions)
Fedor: every exception raising use it
Fedor: and `first 60 chars of response` is too small. I think we should increase size of message. It's just logs, so I don't understand why client needs to cut messages

Am getting error while using
client.bucket.create(bucket_name='bucket-test1', replication_group='REP-ECS', namespace='namespace1')

File "", line 1, in
File "ecs.py", line 244, in create_bucket
status = self.client.bucket.create(bucket_name, replication_group, namespace)
File "/ecsclient/common/provisioning/bucket.py", line 85, in create
return self.conn.post('object/bucket', json_payload=payload)
File "/ecsclient/baseclient.py", line 120, in post
return self._request(url, json_payload, http_verb='POST')
File "/ecsclient/baseclient.py", line 170, in _request
raise ECSClientException.from_response(req)
ecs.ExecutionError: Can not construct instance of java.lang.Boolean from String value 'ess_aut_ns': only "true" or "false" recognized
at [Source: HttpInputOverHTTP@7025d20c[c=153,s=STREAM]; line: 1, column: 2] (through reference chain: com.emc.storageos.data.datasvcmodels.bucket.ObjectBucketParam["filesystem_enabled"]) - Bad request

https://www.emc.com/techpubs/api/ecs/v3-0-0-0/NFSAccessControlService_b2081f90369a6d96de7811240fd3b244_overview.htm

Can you add a check here and not pass this property if the expiry_time passed in is zero?

https://github.com/EMCECS/python-ecsclient/blob/master/ecsclient/common/user_management/secret_key.py#L94

Thanks,
Ben

Please update readme to include a full example of creating a namespace. If i could see a full example on namespace, then creating a bucket, user, or any resource would be trivial.

Thanks!

We have a requirement to create some namespaces with compliance enabled but I noted when attempting to create buckets in those namespaces using ecsclient an error is returned stating that "Incorrect retention parameter: bucket retention period must be set if retention enforcement is off".

On checking I see that under ecsclient.common.provisioning.bucket the create under the Bucket class does not support specifying retention as specified int he REST API guide for ECS - is there supposed to be another way around this (i.e. am I doing it wrong - probably yes is the answer :) ) As there is a function to alter retention post bucket creation I had assumed that you'd be able to create the bucket and then modify retention but no such luck.

Upon modifying bucket.py to allow for specification of retention the bucket is able to be created successfully and specifying a retention of 0 seems as the default also seems to allow for non-compliance buckets to be created without issue.

Run functional tests against an ECS 3.1 instance to verify whether or not code changes are required.

• Force users to provide the parameters that are needed by the API
• Provide examples on how to create and update VDCs.