intelai / inference-model-manager Goto Github PK

Inference Model Manager for Kubernetes

License: Apache License 2.0

Python 64.66% Shell 21.91% Smarty 0.65% Dockerfile 0.12% Makefile 0.92% Go 11.74%

ai inference kubernetes

inference-model-manager's Issues

Management API HTTP response is not JSON

Make the management API a proper REST API. That is, input and output body is JSON. Now the response contains some text and then json, which makes in incorrect and very hard for the client to parse.

Here is an example of a current response:

HTTP/2 200
server: nginx/1.13.12
date: Thu, 28 Mar 2019 07:41:02 GMT
content-type: application/json; charset=UTF-8
content-length: 235
strict-transport-security: max-age=15724800; includeSubDomains

Endpoints present in default-tenant tenant: [{'name': 'endpoint-1', 'url': 'endpoint-1-default-tenant.imm.net:443', 'status': 'Available'}, {'name': 'endpoint-3', 'url': 'endpoint-3-default-tenant.imm.net:443', 'status': 'Available'}]

It should be:

HTTP/2 200
server: nginx/1.13.12
date: Thu, 28 Mar 2019 07:41:02 GMT
content-type: application/json; charset=UTF-8
content-length: xxx
strict-transport-security: max-age=15724800; includeSubDomains

[{"name": "endpoint-1", "url": "endpoint-1-default-tenant.imm.net:443", "status": "Available"}, {"name": "endpoint-3", "url": "endpoint-3-default-tenant.imm.net:443", "status": "Available"}]

Uninstall fails to delete resources if token not valid

The uninstaller.sh deletes the default-tenant with the imm -k rm t command which assumes your have a valid token. If you don't, the namespace and other resources are not cleaned up.

Tenant and endpoint creation fails if [TENANT|ENDPOINT]_RESOURCES not set

The default [TENANT|ENDPOINT]_RESOURCES settings in imm script creates invalid json which fails to create the tenant or endpoint resource.

https://github.com/IntelAI/inference-model-manager/blob/master/scripts/imm#L127
https://github.com/IntelAI/inference-model-manager/blob/master/scripts/imm#L129

Both have extra } at the end.

Steps to reproduce
$ unset ENDPOINT_RESOURCES
$ ./imm create e ep-1 resnet [enter]
Please provide model version policy (press enter for default)
Please provide tenant name default-tenant
Please provide serving name (default: tf-serving)
{"title": "Unexpected error occurred: ('Invalid JSON', 'Could not parse JSON body - Extra data: line 1 column 171 (char 170)')"}

Extra data is the extra bracket }

Logs from mgt:

INFO:management_api.authenticate.authenticate:Request path: /tenants/default-tenant/endpoints, method POST
ERROR:management_api.utils.errors_handling:Unexpected error occurred: ('Invalid JSON', 'Could not parse JSON body - Extra data: line 1 column 169 (char 168)')
Request: <Request: POST 'https://mgt.imm.net/tenants/default-tenant/endpoints'>  Params: {'tenant_name': 'default-tenant'}
ERROR:management_api.utils.errors_handling:Traceback (most recent call last):
  File "/usr/lib/python3.7/site-packages/falcon/media/json.py", line 15, in deserialize
    return json.loads(raw.decode('utf-8'))
  File "/usr/lib/python3.7/json/__init__.py", line 348, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python3.7/json/decoder.py", line 340, in decode
    raise JSONDecodeError("Extra data", s, end)
json.decoder.JSONDecodeError: Extra data: line 1 column 169 (char 168)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.7/site-packages/falcon/api.py", line 244, in __call__
    responder(req, resp, **params)
  File "/usr/lib/python3.7/site-packages/falcon/media/validators/jsonschema.py", line 45, in wrapper
    jsonschema.validate(req.media, schema, format_checker=jsonschema.FormatChecker())
  File "/usr/lib/python3.7/site-packages/falcon/request.py", line 965, in media
    self._media = handler.deserialize(raw)
  File "/usr/lib/python3.7/site-packages/falcon/media/json.py", line 19, in deserialize
    'Could not parse JSON body - {0}'.format(err)
falcon.errors.HTTPBadRequest: ('Invalid JSON', 'Could not parse JSON body - Extra data: line 1 column 169 (char 168)')

INFO:management_api.authenticate.authenticate:Checking token: eyJhbGciOiJSUzI1NiIsImtpZCI6ImI2YTU0ZWQzMzdjYjY5N2M4YWFmMmZiMjFhMDYzYTExMjZlMWYwNDkifQ.eyJpc3MiOiJodHRwczovL2RleC5pbW0ubmV0OjQ0My9kZXgiLCJzdWIiOiJDaU5qYmoxMWMyVnlMRzkxUFZCbGIzQnNaU3hrWXoxbGVHRnRjR3hsTEdSalBXOXlaeElFYkdSaGNBIiwiYXVkIjoiZXhhbXBsZS1hcHAiLCJleHAiOjE1NTM2ODg3NDcsImlhdCI6MTU1MzY4NTE0NywiYXRfaGFzaCI6IkpLdlhkdGkyZGtoV0ltV0hWcXRJQ3ciLCJlbWFpbCI6InVzZXJAZXhhbXBsZS5jb20iLCJlbWFpbF92ZXJpZmllZCI6dHJ1ZSwiZ3JvdXBzIjpbImFkbWluIl19.wQ69oWwUFdrdrUnLH2z8yCDSQ62OisreWqmdqLEToFm70hSMQl8NPO-Krmn549f7BJOata8Q60VupKuYmYgZOfRAiXVUSSpokp8cfcr5RBUrpiw-rwdrk8aBKH5P_l_gq1JA74AnUb_fvzWUvICaEUFL1nK9kooC8U1Q01hrAGo-K1-wpRtSfZnvqzU0EnDV8xZr7wdmINmY0dpIv3tGWkxS3M2J5WxnM-_d8qw3-Spl_QHXKwHbuqWHLNxpt6nlAcXW9Sv690JwY0_aWpAvRpbH6hgJpj5Hx3rJ_L9tlzUSAVhj5yrS6SyqSy8ITx2lXUkXncW_6_oTnbaGiRiuOQ
INFO:management_api.authenticate.authenticate:Trying key number : 0
INFO:management_api.authenticate.authenticate:Decoded
INFO:management_api.authenticate.authenticate:Using service account token
INFO:management_api.authenticate.authenticate:Decoded token : {'iss': 'https://dex.imm.net:443/dex', 'sub': 'CiNjbj11c2VyLG91PVBlb3BsZSxkYz1leGFtcGxlLGRjPW9yZxIEbGRhcA', 'aud': 'example-app', 'exp': 1553688747, 'iat': 1553685147, 'at_hash': 'JKvXdti2dkhWImWHVqtICw', 'email': '[email protected]', 'email_verified': True, 'groups': ['admin']}
INFO:management_api.authenticate.authenticate:Request path: /tenants/default-tenant/endpoints, method POST
INFO:botocore.vendored.requests.packages.urllib3.connectionpool:Resetting dropped connection: minio.imm.net
INFO:management_api.tenants.tenants_utils:Tenant already exists: True
ERROR:management_api.utils.errors_handling:An error occurred during endpoint creation: (409)
Reason: Conflict
HTTP response headers: HTTPHeaderDict({'Content-Type': 'application/json', 'Date': 'Wed, 27 Mar 2019 12:03:19 GMT', 'Content-Length': '264'})
HTTP response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"inference-endpoints.ai.intel.com \"endpoint-1\" already exists","reason":"AlreadyExists","details":{"name":"endpoint-1","group":"ai.intel.com","kind":"inference-endpoints"},"code":409}

IMM_CONFIG_PATH env variable does not work

When running the scripts/imm, the IMM_CONFIG_PATH env variable does not have any effect since it is always set to ~/.immconfig

https://github.com/IntelAI/inference-model-manager/blob/master/scripts/imm_utils.sh#L18

intelai / inference-model-manager Goto Github PK

inference-model-manager's People

Contributors

Stargazers

Watchers

Forkers

inference-model-manager's Issues

Management API HTTP response is not JSON

Uninstall fails to delete resources if token not valid

Tenant and endpoint creation fails if [TENANT|ENDPOINT]_RESOURCES not set

Does this setup allow inference on GRU too?

Suggestions to improve installer

When endpoint creation fails, resources are not cleaned up properly

IMM_CONFIG_PATH env variable does not work

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent