!python baseball-pipeline-single.py >> !python pipelines/baseball-pipeline-single.py

(project name has been redacted)

Using endpoint [
https://us-central1-ml.googleapis.com/
]
Listed 0 items.
Using endpoint [
https://ml.googleapis.com/
]
ERROR: (gcloud.beta.ai-platform.models.create) Resource in projects [...] is the subject of a conflict: Field: model.name Error: A model with the same name already exists.
- '@type': type.googleapis.com/google.rpc.BadRequest
  fieldViolations:
  - description: A model with the same name already exists.
    field: model.name
Traceback (most recent call last):
  File "host_xgboost.py", line 46, in <module>
    run()
  File "host_xgboost.py", line 35, in run
    subprocess.check_call([shutil.which('gcloud'),'beta', 'ai-platform','models','create',MODEL_NAME,'--regions','us-central1', "--enable-logging", "--enable-console-logging"])
  File "/usr/lib/python3.7/subprocess.py", line 347, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/usr/bin/gcloud', 'beta', 'ai-platform', 'models', 'create', 'xgboost_FT', '--regions', 'us-central1', '--enable-logging', '--enable-console-logging']' returned non-zero exit status 1

All the data should be pulling from the MW Public GCP Buckets:

• https://storage.googleapis.com/amazing-public-data/black_friday/test.csv
• https://storage.googleapis.com/amazing-public-data/lending_club/lending_club_data.tsv
• ...

But the example don't seem to pull from these.

Note for MW folks: this is https://console.cloud.google.com/storage/browser/amazing-public-data;tab=objects?forceOnBucketsSortingFiltering=false&project=data-describe&prefix=&forceOnObjectsSortingFiltering=false

For example:

• Census-income ttps://archive.ics.uci.edu/ml/machine-learning-databases/adult/adult.data and *.text
• for black-friday https://github.com/data-describe/awesome-data-science-models/tree/master/black-friday/EDA/Data
• on cellular image bipin_workshop_bucket is used should ber the public bucket
• Chicago taxi looks fine as it uses a pubic GCP data set but we need to give instructions on how to authenticate their own GCP project for access
• lending club should be moved from gs://amazing-public-data/lending_club/lending_club_data.tsv
• pitch predictor uses a Collect-Games method https://github.com/data-describe/awesome-data-science-models/blob/master/pitch-predictor/answers/components/collectStats/ParDoFns/collectGames.py#L13, should probably freeze that data and put it in our public bucket

saw some paths with client names, need to remove those before we make public.

Please add in README.md where each example originated.

create IOT example with NASA Bearing Sensor Dataset

Are there any usability shortcomings that prevent using dd.distribution() in the EDA notebooks? These plots are always created manually.

Pitch predictor contains partially implemented components for random forest model (training and deploy) as well as a Seldon deployed model.

Prep the following.

Give overview for each, create readme.md with an overview of the problem, description of the data, and the problem statement. Where possible make references to the GCP tool set. Possibly make a sub directory of each with GCP as the name to create GCP specific training.

• Census
• Beatles
• Lending Club

Make sure each one has filled out the following:

• EDA Census

• EDA Beatles

• EDA Lending Club

• Exercises Census

• Exercises Beatles

• Exercises Lending Club

• Answers Census

• Answers Beatles

• Answers Lending Club

• MLOps Prod tutorial for Census

• MLOps Prod tutorial for Beatles

• MLOps Prod tutorial Lending Club

• Add a new exercise called AI/ML Use discovery worksheet

The Document AI demo should incorporate real-world training data. Right now, it uses a very small set of dummy data in order to train the classification algorithm.

the GCP project name is currently hardcoded to mwpmltr. While this is a straightforward find-and-replace, it would be nicer if it was templated out or dynamically filled based on appropriate environment variables.

include the demo

Create unit tests for each and a corresponding github action to run examples.

Pipeline has not been validated after update

The aim of this issue is to add more IoT related use cases to awesome data science models repository. The use case we have selected for this activity is experimental bearing data sets for condition monitoring based on vibration and motor current signals.

The goal is to move from condition monitoring to predictive maintenance. Modeling approaches to consider include Predict Remaining Useful LifeCycle of bearing so that corrective and timely maintenance measures can be taken. This will help avoid unplanned machine downtime.

Many plants faces unplanned downtime and it assets fail over the period of time and plants should have the ability to predict the life remaining. With the help of data science, the first step is to build an ML tool to analyze the condition or health of machine assets.

Our dataset of Interest(Paderborn Bearing Dataset) consists the following details:

1. The Dataset contains the motor currents and vibration signals with additional measurements of torque, speed, load and temperature
2. There are 26 Damaged bearing and 6 healthy bearing state in the data
3. All the data is collected in 4 different operating conditions.

Dataset - https://mb.uni-paderborn.de/en/kat/main-research/datacenter/bearing-datacenter/data-sets-and-download

With help of Data Science techniques the aim is to analyze the condition of assets, identify features which contribute to failure and build effective ML models for RUL prediction

The requirements.txt state TF1 but we are using TF2

https://github.com/data-describe/awesome-data-science-models/tree/master/chicago-taxi

Ending with commit 7d748cf, additional work on these demos will begin to be targeted towards Google Vertex AI vs the traditional Google AI Platform. AI Platform is still available but expected to be deprecated sometime in the future.

All of the examples in this repository use the data-describe tool to perform exploratory data analysis on the training data. This step needs to be added to the Document AI example.

Rework the current chicago-taxi solution. There are dependency conflicts with the current implementation on AI Platform. May need to upgrade the runtime, keras version, and deployment strategy.