pmservice / ai-openscale-tutorials Goto Github PK

Watson OpenScale tutorials including sample models, notebooks and applications

Python 1.38% HTML 0.23% JavaScript 0.74% CSS 0.23% Dockerfile 0.03% Jupyter Notebook 97.39%

ai-openscale-tutorials's Introduction

IMPORTANT NOTE: Watson machine learning and Openscale have released newer versions of SDKs available for general use. Current available samples use old SDKs and will be obsolate and removed in next couple months. Please use new SDKs going forward(WML and Openscale) to build new models and monitor using Openscale.

For samples using latest SDKs and runtimes, please go to following location:

https://github.com/IBM/watson-openscale-samples

IBM Watson OpenScale tutorials.

IBM Cloud

Tutorial 1. Working with Watson Machine Learning engine

Step 1: Credit risk prediction model creation, deployment as web-service and monitoring using Watson OpenScale - notebook

Tutorial 2. Working with Custom Machine Learning engine

Step 1: Creation of Custom Machine Learning engine using Kubernetes cluster - deployment instruction
Step 2: Data mart creation, model deployment monitoring and data analysis - notebook

Tutorial 3. Working with Azure Machine Learning Studio engine

Step 1: Data mart creation, model deployment monitoring and data analysis - notebook

Tutorial 4. Working with Amazon SageMaker Machine Learning engine

Step 1: Creation and deployment of credit risk prediction model - notebook
Step 2: Data mart creation, model deployment monitoring and data analysis - notebook

Tutorial 5. Working with Azure Machine Learning Service engine

Step 1: Data mart creation, model deployment monitoring and data analysis - notebook

IBM Cloud Pak for Data

Tutorial 5. Working with IBM SPSS C&DS engine

Step 1: Data mart creation, model deployment monitoring and data analysis - notebook

Tutorial 6. Working with Watson Machine Learning engine on CP4D

Step 1: Credit risk prediction model creation, deployment as web-service and monitoring using Watson OpenScale - notebook

Tutorial 7. Generating an explanation for an image-based model on Cloud Pak for Data v. 2.5.0

Step 1: - Watson OpenScale Explanation for Image Multiclass Classification Model CP4D - notebook

Microsoft Azure Cloud

Tutorial 8. Working with not directly supported engine through Custom ML Engine

Step 1: Credit risk model (scikit-learn) deployment on Azure ML Service - notebook
Step 2: Creation of Custom Machine Learning engine and deployment on Azure Cloud as flask application - deployment instruction
Step 3: OpenScale configuration to work with Custom ML Engine - notebook
Step 4: Creation of scoring endpoint wrapper to automate payload logging on Azure ML Service - notebook

Model Risk Management and Governance Features

Watson OpenScale Model Risk Management

On Cloud

Tutorial 1. OpenScale Model Risk Governance with OpenPages Integration on IBM Cloud - notebook
Tutorial 2. OpenScale Model Risk Management on IBM Cloud - notebook

On Cloud Pak for Data

Tutorial 3. OpenScale Model Risk Governance with OpenPages Integration on Cloud Pack for Data - notebook
Tutorial 4. OpenScale Model Risk Management on Cloud Pak for Data - notebook

Metrics Mapping

Tutorial 5. OpenScale MRM metrics mapping - notebook

ai-openscale-tutorials's People

Contributors

Stargazers

Watchers

Forkers

kmacdonald06 sridharkuna kchallapalli sofgun kishore-patel mtawbi bchalamayya howellyu rakababa aishgrt1 prempiyush francescobasciani msochka jruponen stuartarnell hashmalik singhkumaranuj jinsalex sachamongrain sokollip frollini tauheedahmed robertosam mohinishmd zluo1 tranfm pratyushsingh97 harshit-sh jeanlouisardoint esveee bmat7717 abhinandan-kr adri0792 gauravgoswami arsuryan konsoelski philippe-gregoire 83829339 ychunhui n-otna gsg96 kyokonishito karaszewicz asuliman17 vinodhmohan91 4ahu1 pholkar1 hs312205 ravisrirangam sarweshsuman aggarwalrohit0101 gvjimboe skauntey khairulhabibatibm kartha01 opoyraz moly-malibu mbugo ashishpatel26 techthiyanes yogita-kulkarni19 amirgholipour

ai-openscale-tutorials's Issues

custom-ml-engine-deployment fails with latest depencies.

https://github.com/pmservice/ai-openscale-tutorials/tree/master/applications/custom-ml-engine-containers

Error:

kubectl logs custom-ml-engine-deployment-67944fc864-97w85 
2020-03-20 13:43:42.738823: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libnvinfer.so.6'; dlerror: libnvinfer.so.6: cannot open shared object file: No such file or directory
2020-03-20 13:43:42.739398: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libnvinfer_plugin.so.6'; dlerror: libnvinfer_plugin.so.6: cannot open shared object file: No such file or directory
2020-03-20 13:43:42.739573: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:30] Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.
20/03/20 13:43:47 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
Setting default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).
Using TensorFlow backend.
/opt/conda/lib/python3.7/site-packages/sklearn/externals/joblib/__init__.py:15: FutureWarning: sklearn.externals.joblib is deprecated in 0.21 and will be removed in 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib. If this warning is raised when loading pickled models, you may need to re-serialize those models with scikit-learn 0.21+.
  warnings.warn(msg, category=FutureWarning)
Traceback (most recent call last):
  File "/home/jovyan/run_server.py", line 239, in <module>
    load_resnet50_model()
  File "/home/jovyan/run_server.py", line 30, in load_resnet50_model
    with backend.get_session().graph.as_default() as g:
  File "/opt/conda/lib/python3.7/site-packages/keras/backend/tensorflow_backend.py", line 379, in get_session
    '`get_session` is not available '
RuntimeError: `get_session` is not available when using TensorFlow 2.0.

I had success with the following requirements.

cfenv==0.5.3
Flask==1.0.2
watson-developer-cloud==1.3.5
gevent
requests
tensorflow==1.15.0
keras==2.2.5
ibmcloudenv
livereload
pillow
numpy
pyspark
pandas
scikit-learn==0.20.2

` Creation of Custom Machine Learning engine using Kubernetes cluster` is broken

The top level README.md has instructions for:
Tutorial 2. Working with Custom Machine Learning engine

Step 1: Creation of Custom Machine Learning engine using Kubernetes cluster - deployment instructions

But the Deployment instructions no longer have K8s instructions.

I noticed a new Dockerfile in the custom-ml-engine-bluemix directory, but that Dockerfile uses the manage.py script, which is not in this directory, but is in the custom-ml-engine-containers directory.

But that *-containers dir has an outdated Dockerfile and the README says:contains the code for creating custom deployment of Keras ResNet50 model (images classification) and Spark MLlib (CARS4U) on IBM Cloud, which is the outdated use case for custom-ml (should be the German credit model).

So....There's a mix of Dockerfiles and associated scripts in the 2 directory trees, and these need to be reconciled, as well as the instructions for using K8s updated.

'SSL routines', 'tls_process_server_certificate', 'certificate verify failed' for `Watson OpenScale and Watson ML Engine - ICP.ipynb`

Running Watson OpenScale and Watson ML Engine - ICP.ipynb and under Insert historical confusion matrixes I get an error during response = requests.post(measurements_url, json=measurments, headers=ai_client._get_headers())

It works with verify=False, whichy the other POST methods use, so perhaps it is just missing here.

Need push access to the repo or help uploading some new notebooks

There are four notebooks that need to be pushed to the repo. They are the Model Risk Management notebooks that can be used as part of the tutorials. Who can help me with this?

The files are attached to the next comment.

PYTHON_FLASK should be FLASK_APP

Using these instructions I get an error:

 $ export PYTHON_FLASK=app.py

[master:?][scott@mac2018:~/gitRepos/Sandbox/ai-openscale-tutorials/applications/custom-ml-engine-bluemix]
 $ python -m flask run
Usage: python -m flask run [OPTIONS]

Error: Could not locate Flask application. You did not provide the FLASK_APP environment variable.

I believe it should be:

export FLASK_APP=app.py