aicoe / elyra-aidevsecops-tutorial Goto Github PK
View Code? Open in Web Editor NEWAIDevSecOps with Thoth and Elyra on ODH/Openshift running on Operate First
License: GNU General Public License v3.0
elyra-aidevsecops-tutorial's People
Contributors
![dependabot[bot] avatar](https://avatars.githubusercontent.com/in/29110?v=4 "dependabot[bot]")
![khebhut[bot] avatar](https://avatars.githubusercontent.com/in/62893?v=4 "khebhut[bot]")
![op1st-prow[bot] avatar](https://avatars.githubusercontent.com/in/331653?v=4 "op1st-prow[bot]")
Stargazers
Watchers
Forkers
pacospace harshad16 martinpovolny fridex durandom michaelclifford pixeljonas chauhankaranraj tumido stefan-bergstein juandecarrion rrma mvesela baierst marekhanus pstovik cliff1987 hwozd natasa-plulikova vpavlin ptitzler isabelizimm kpostoffice margarethaley hemajv aakankshaduggal preska oindrillac shreyanand suppathak antter codificat jesus-bustos eldritchjs adrezni mimotej mirkhala ocarvajarh diego-uchoa szemmour-rh timjslaton ravitri schwesig maniveer2017 wongwiyan almen-ng mayacostantini gregory-pereira theckang rcarrata ctrockelelyra-aidevsecops-tutorial's Issues
Failed to update dependencies to their latest version
Automatic dependency update failed for the current master with SHA a63712c.
The automatic dependency management cannot continue. Please fix errors reported bellow.
Command
$ pipenv update --dev
Standard output
Running $ pipenv lock then $ pipenv sync.
Installing dependencies from Pipfile.lock (56a440)...
Ignoring typing-extensions: markers 'python_version < "3.8"' don't match your environment
Installing initially failed dependencies...
Standard error
Creating a virtualenv for this project...
Pipfile: /tmp/tmpipitjgf1/Pipfile
Using /home/user/.local/share/virtualenvs/user-5PivF_Od/bin/python3.8 (3.8.3) to create virtualenv...
created virtual environment CPython3.8.3.final.0-64 in 733ms
creator CPython3Posix(dest=/tmp/tmpipitjgf1/.venv, clear=False, no_vcs_ignore=False, global=False)
seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/tmp/tmpurvfxhp8)
added seed packages: pip==20.3.3, setuptools==51.3.3, wheel==0.36.2
activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator
Successfully created virtual environment!
�[KVirtualenv location: /tmp/tmpipitjgf1/.venv
Locking [dev-packages] dependencies...
Building requirements...
�[K
Resolving dependencies...
�[K
Success!
�[KLocking [packages] dependencies...
Building requirements...
�[K
Resolving dependencies...
�[K
Success!
�[KUpdated Pipfile.lock (56a440)!
An error occurred while installing tensorflow==2.4.1 --hash=sha256:efa9daa4b3701a4e439b24b74c1e4b66844aee8ae5263fb3cc12281ac9cc9f67 --hash=sha256:eedcf578afde5e6e69c75d796bed41093451cd1ab54afb438760e40fb74a09de --hash=sha256:2357112319303da1b5459a621fd0503c2b2cd97b6c33c4903abd46b3c3e380e2 --hash=sha256:0e427b1350be6dbe572f971947c5596fdbb152081f227808d8becd894bf40282 --hash=sha256:4a04081647b89a8fb602895b29ffc559e3c20aac8bde1d4c5ecd2a65adce5d35 --hash=sha256:36d5acd60aac48e34bd545d0ce1fb8b3fceebff6b8782436defd0f71c12203bd --hash=sha256:55368ba0bedb513ba0e36a2543a588b5276e9b2ca99fa3232a9a176601a7bab5 --hash=sha256:22723b8e1fa83b34f56c349b16a57aaff913b404451fcf70981f2b1d6e0c64fc --hash=sha256:e1f2799cc86861680d8515167f103e2207a8cab92a4afe5471e4839330591f08! Will try again.
[InstallError]: File "/home/user/.local/share/virtualenvs/user-5PivF_Od/lib/python3.8/site-packages/pipenv/cli/command.py", line 594, in update
[InstallError]: do_sync(
[InstallError]: File "/home/user/.local/share/virtualenvs/user-5PivF_Od/lib/python3.8/site-packages/pipenv/core.py", line 2884, in do_sync
[InstallError]: do_init(
[InstallError]: File "/home/user/.local/share/virtualenvs/user-5PivF_Od/lib/python3.8/site-packages/pipenv/core.py", line 1304, in do_init
[InstallError]: do_install_dependencies(
[InstallError]: File "/home/user/.local/share/virtualenvs/user-5PivF_Od/lib/python3.8/site-packages/pipenv/core.py", line 899, in do_install_dependencies
[InstallError]: batch_install(
[InstallError]: File "/home/user/.local/share/virtualenvs/user-5PivF_Od/lib/python3.8/site-packages/pipenv/core.py", line 796, in batch_install
[InstallError]: _cleanup_procs(procs, failed_deps_queue, retry=retry)
[InstallError]: File "/home/user/.local/share/virtualenvs/user-5PivF_Od/lib/python3.8/site-packages/pipenv/core.py", line 703, in _cleanup_procs
[InstallError]: raise exceptions.InstallError(c.dep.name, extra=err_lines)
[pipenv.exceptions.InstallError]: Collecting tensorflow==2.4.1
ERROR: Couldn't install package: tensorflow
Package installation failed...
Environment details
Kebechet version: 1.2.4
Python version: 3.8.3
Platform: Linux-4.18.0-193.41.1.el8_2.x86_64-x86_64-with-glibc2.2.5
pipenv version: pipenv, version 2020.11.15
Dependency graph
Unable to obtain dependency graph:
An error occurred while installing tensorflow! Will try again.
[InstallError]: File "/home/user/.local/share/virtualenvs/user-5PivF_Od/lib/python3.8/site-packages/pipenv/cli/command.py", line 233, in install
[InstallError]: retcode = do_install(
[InstallError]: File "/home/user/.local/share/virtualenvs/user-5PivF_Od/lib/python3.8/site-packages/pipenv/core.py", line 2052, in do_install
[InstallError]: do_init(
[InstallError]: File "/home/user/.local/share/virtualenvs/user-5PivF_Od/lib/python3.8/site-packages/pipenv/core.py", line 1304, in do_init
[InstallError]: do_install_dependencies(
[InstallError]: File "/home/user/.local/share/virtualenvs/user-5PivF_Od/lib/python3.8/site-packages/pipenv/core.py", line 899, in do_install_dependencies
[InstallError]: batch_install(
[InstallError]: File "/home/user/.local/share/virtualenvs/user-5PivF_Od/lib/python3.8/site-packages/pipenv/core.py", line 796, in batch_install
[InstallError]: _cleanup_procs(procs, failed_deps_queue, retry=retry)
[InstallError]: File "/home/user/.local/share/virtualenvs/user-5PivF_Od/lib/python3.8/site-packages/pipenv/core.py", line 703, in _cleanup_procs
[InstallError]: raise exceptions.InstallError(c.dep.name, extra=err_lines)
[pipenv.exceptions.InstallError]: Collecting tensorflow
ERROR: Couldn't install package: tensorflow
Package installation failed...
Notes
For more information, see Pipfile and Pipfile.lock.
Once this issue is resolved, the issue will be automatically closed by bot.
New patch release
Hey, Kebechet!
Create a new patch release, please.
Deliver Container Image
Hey, AICoE-CI!
Please build and deliver the following git tag:
Tag: v0.8.0
deploy via OpenVino Model Server Operator
Is your feature request related to a problem? Please describe.
Have an alternative deployment via https://catalog.redhat.com/software/operators/detail/60649a56209af65d24b7ca9e same tests and metrics should be applied.
Describe the solution you'd like
- deploy https://catalog.redhat.com/software/operators/detail/60649a56209af65d24b7ca9e on zero-prod
- extend deployment to use OpenVINO Model Server
Describe alternatives you've considered
n/a
Additional context
n/a
Acceptance Criteria
- PR is opened in this repo, model is deployed as is, metrics feedback to PR
- model is deployed using OpenVINO, metrics feedback to PR
/priority important-soon
/kind feature
/kind demo
New patch release
Hey, Kebechet!
Create a new patch release, please.
create a repo with demo application, discuss the interface between Data Science and Dev/DevOps, document that interface as a template project on GitHub
- use simple MNIST classification example
- create python script to download dataset and store locally
- create training notebooks
- create Elyra pipeline to run both steps and store model locally
- create Flask app and expose /predict and /metrics endpoint
- export metrics of Flask app (#28)
- test deployment and prediction
- Add a notebook to run prediction
- deploy on Operate first in thoth-deployment-examples namespace (operate-first/apps#157, #17, operate-first/argocd-apps#51)
- add documentation and description in README
- demo tutorial
reference:
create a tutorial that is compatible with https://github.com/redhat-scholars
SSIA ;)
what is this file used for?
Why do we have this file?
https://github.com/thoth-station/elyra-aidevsecops-tutorial/blob/6e35b3ec4b589ede30e06fa22a0529fc99044b83/Pipfile#L1
/kind questions
/sig docs
/assign @pacospace
New minor release
Hey, Kebechet!
Create a new minor release, please.
show off how an Elyra pipeline is using CPU and GPU
Is your feature request related to a problem? Please describe.
As a Data Scientist,
I want to run an Elyra Pipeline,
so that some steps utilize CPU
and so that the training step utilized GPU
The pipeline might have 4 standard steps that end up in a trained model on the repo, so that it could be tagged and released...
The result should be a demo on Op1st.
Describe the solution you'd like
tbd
Describe alternatives you've considered
n/a
Additional context
the basic question is, how we could share that GPU in a Data Scientist team in an efficient way.
Cc: @cfchase
/kind feature
/sig user-experience
embed in behave report once feature is available > 1.2.6
This issue was generated by todo based on a TODO comment in 023e3ba. It's been assigned to @pacospace because they committed the code.
New minor release
Hey, Kebechet!
Create a new minor release, please.
New minor release
Hey, Kebechet!
Create a new minor release, please.
Deliver Container Image
Hey, AICoE-CI!
Please build and deliver the following git tag:
Tag: v0.3.0
integrated AICoE-CI with namespace this tutorial is deployed to
Is your feature request related to a problem? Please describe.
We want to show, that all the images required to work thru the tutorial, can be created/build by an integrated (with the namespace the tutorial is running in) aicoe-ci deployment. images shall be delivered to the OCP internal registry only. any guidance feed into the devsecops procedures/pipes shall come from an operate first hosted thoth service.
Describe the solution you'd like
deploy aicoe-ci to the same namespace as the tutorial, use openshift internal registry
Describe alternatives you've considered
n/a
Additional context
n/a
/kind feature
/priority important-soon
patch image tag fails
Describe the bug
patch image tag fails https://github.com/thoth-station/elyra-aidevsecops-tutorial/blob/51b4b15062a3aaaa5d82c21eaf5e7639d2066f5d/.aicoe-ci.yaml#L15
To Reproduce
Steps to reproduce the behavior:
- Retrigger pipeline
Expected behavior
Correct patch image tag in context path elyra-aidevsecops-tutorial/manifests/overlays/test/imagestreamtag.yaml
Screenshots
+ '[' -z elyra-aidevsecops-tutorial ]
+ registry_repo=elyra-aidevsecops-tutorial
+ '[' -z elyra-aidevsecops-tutorial/manifests/overlays/test/imagestreamtag.yaml ]
+ contextpath=elyra-aidevsecops-tutorial/manifests/overlays/test/imagestreamtag.yaml
+ '[' '!' -z elyra-aidevsecops-tutorial ]
+ yq r '-d*' elyra-aidevsecops-tutorial/manifests/overlays/test/imagestreamtag.yaml metadata.name
13:54:30 main [ERRO] open elyra-aidevsecops-tutorial/manifests/overlays/test/imagestreamtag.yaml: no such file or directory
+ docs='Error: open elyra-aidevsecops-tutorial/manifests/overlays/test/imagestreamtag.yaml: no such file or directory
Usage:
yq read [yaml_file] [path_expression] [flags]
Additional context
Add any other context about the problem here.
No dependency management found
No dependency management found for this repository. If you want to keep your dependencies managed, please submit Pipfile or requirements.in or requirements-dev.in file.
To generate a Pipfile, use:
$ pipenv install --skip-lock --code ./
$ git add Pipfile
$ git commit -m 'Add Pipfile for dependency management'
Make sure your Pipfile or requirements.in or requirements-dev.in is placed in the root of your Git repository.
New patch release
Hey, Kebechet!
Create a new patch release, please.
New minor release
Hey, Kebechet!
Create a new minor release, please.
Update tutorial to consider secure Elyra pipeline (credentials not exposed)
Is your feature request related to a problem? Please describe.
As Operate First user,
I want to see in the tutorial how I can set credentials before submitting my pipeline.
Describe the solution you'd like
See: operate-first/support#155 (comment)
Describe alternatives you've considered
Leave as it is.
Additional context
deploy tutorial to Operate-First
start deploying this tutorial to the Operate-First cluster by opening an issue: https://github.com/operate-first/support/issues/new?assignees=&labels=onboarding&template=onboarding_to_cluster.md&title=
/assign @pacospace
/sig documentation
/sig devops
add a Kubeflow-Serving part
Is your feature request related to a problem? Please describe.
As a Data Scientist, I trained my model and persisted it to the git repo, so that a deployment pipeline can pick it up and deploy it.
Describe the solution you'd like
KFServing to take model to deployment
Describe alternatives you've considered
n/a
Additional context
n/a
/kind feature
/sig pipelines
Failed to update dependencies to their latest version
Automatic dependency update failed for the current master with SHA 37240a2.
The automatic dependency management cannot continue. Please fix errors reported bellow.
Command
$ pipenv update --dev
Standard output
Running $ pipenv lock then $ pipenv sync.
Installing dependencies from Pipfile.lock (d2f2cc)...
Installing initially failed dependencies...
Standard error
Creating a virtualenv for this project...
Pipfile: /tmp/tmpowmxz8i7/Pipfile
Using /home/user/.local/share/virtualenvs/user-5PivF_Od/bin/python3.8 (3.8.3) to create virtualenv...
created virtual environment CPython3.8.3.final.0-64 in 713ms
creator CPython3Posix(dest=/tmp/tmpowmxz8i7/.venv, clear=False, no_vcs_ignore=False, global=False)
seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/tmp/tmpexhmq3kp)
added seed packages: pip==20.3.1, setuptools==51.0.0, wheel==0.36.1
activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator
Successfully created virtual environment!
�[KVirtualenv location: /tmp/tmpowmxz8i7/.venv
Locking [dev-packages] dependencies...
Building requirements...
�[K
Resolving dependencies...
�[K
Success!
�[KLocking [packages] dependencies...
Building requirements...
�[K
Resolving dependencies...
�[K
Success!
�[KUpdated Pipfile.lock (d2f2cc)!
An error occurred while installing tensorflow==2.4.1 --hash=sha256:4a04081647b89a8fb602895b29ffc559e3c20aac8bde1d4c5ecd2a65adce5d35 --hash=sha256:e1f2799cc86861680d8515167f103e2207a8cab92a4afe5471e4839330591f08 --hash=sha256:eedcf578afde5e6e69c75d796bed41093451cd1ab54afb438760e40fb74a09de --hash=sha256:efa9daa4b3701a4e439b24b74c1e4b66844aee8ae5263fb3cc12281ac9cc9f67 --hash=sha256:55368ba0bedb513ba0e36a2543a588b5276e9b2ca99fa3232a9a176601a7bab5 --hash=sha256:22723b8e1fa83b34f56c349b16a57aaff913b404451fcf70981f2b1d6e0c64fc --hash=sha256:0e427b1350be6dbe572f971947c5596fdbb152081f227808d8becd894bf40282 --hash=sha256:36d5acd60aac48e34bd545d0ce1fb8b3fceebff6b8782436defd0f71c12203bd --hash=sha256:2357112319303da1b5459a621fd0503c2b2cd97b6c33c4903abd46b3c3e380e2! Will try again.
[InstallError]: File "/home/user/.local/share/virtualenvs/user-5PivF_Od/lib/python3.8/site-packages/pipenv/cli/command.py", line 594, in update
[InstallError]: do_sync(
[InstallError]: File "/home/user/.local/share/virtualenvs/user-5PivF_Od/lib/python3.8/site-packages/pipenv/core.py", line 2884, in do_sync
[InstallError]: do_init(
[InstallError]: File "/home/user/.local/share/virtualenvs/user-5PivF_Od/lib/python3.8/site-packages/pipenv/core.py", line 1304, in do_init
[InstallError]: do_install_dependencies(
[InstallError]: File "/home/user/.local/share/virtualenvs/user-5PivF_Od/lib/python3.8/site-packages/pipenv/core.py", line 899, in do_install_dependencies
[InstallError]: batch_install(
[InstallError]: File "/home/user/.local/share/virtualenvs/user-5PivF_Od/lib/python3.8/site-packages/pipenv/core.py", line 796, in batch_install
[InstallError]: _cleanup_procs(procs, failed_deps_queue, retry=retry)
[InstallError]: File "/home/user/.local/share/virtualenvs/user-5PivF_Od/lib/python3.8/site-packages/pipenv/core.py", line 703, in _cleanup_procs
[InstallError]: raise exceptions.InstallError(c.dep.name, extra=err_lines)
[pipenv.exceptions.InstallError]: Collecting tensorflow==2.4.1
ERROR: Couldn't install package: tensorflow
Package installation failed...
Environment details
Kebechet version: 1.2.3
Python version: 3.8.3
Platform: Linux-4.18.0-193.29.1.el8_2.x86_64-x86_64-with-glibc2.2.5
pipenv version: pipenv, version 2020.11.15
Dependency graph
Unable to obtain dependency graph:
An error occurred while installing tensorflow! Will try again.
[InstallError]: File "/home/user/.local/share/virtualenvs/user-5PivF_Od/lib/python3.8/site-packages/pipenv/cli/command.py", line 233, in install
[InstallError]: retcode = do_install(
[InstallError]: File "/home/user/.local/share/virtualenvs/user-5PivF_Od/lib/python3.8/site-packages/pipenv/core.py", line 2052, in do_install
[InstallError]: do_init(
[InstallError]: File "/home/user/.local/share/virtualenvs/user-5PivF_Od/lib/python3.8/site-packages/pipenv/core.py", line 1304, in do_init
[InstallError]: do_install_dependencies(
[InstallError]: File "/home/user/.local/share/virtualenvs/user-5PivF_Od/lib/python3.8/site-packages/pipenv/core.py", line 899, in do_install_dependencies
[InstallError]: batch_install(
[InstallError]: File "/home/user/.local/share/virtualenvs/user-5PivF_Od/lib/python3.8/site-packages/pipenv/core.py", line 796, in batch_install
[InstallError]: _cleanup_procs(procs, failed_deps_queue, retry=retry)
[InstallError]: File "/home/user/.local/share/virtualenvs/user-5PivF_Od/lib/python3.8/site-packages/pipenv/core.py", line 703, in _cleanup_procs
[InstallError]: raise exceptions.InstallError(c.dep.name, extra=err_lines)
[pipenv.exceptions.InstallError]: Collecting tensorflow
ERROR: Couldn't install package: tensorflow
Package installation failed...
Notes
For more information, see Pipfile and Pipfile.lock.
Once this issue is resolved, the issue will be automatically closed by bot.
New minor release
Hey, Kebechet!
Create a new minor release, please.
New minor release
Hey, Kebechet!
Create a new minor release, please.
Failed to update dependencies to their latest version
Automatic dependency update failed for the current master with SHA aadb946.
The automatic dependency management cannot continue. Please fix errors reported bellow.
Command
$ pipenv update --dev
Standard output
Running $ pipenv lock then $ pipenv sync.
Installing dependencies from Pipfile.lock (30af66)...
Installing initially failed dependencies...
Standard error
Creating a virtualenv for this project...
Pipfile: /tmp/tmp_hv_886n/Pipfile
Using /home/user/.local/share/virtualenvs/user-5PivF_Od/bin/python3.8 (3.8.3) to create virtualenv...
created virtual environment CPython3.8.3.final.0-64 in 1127ms
creator CPython3Posix(dest=/tmp/tmp_hv_886n/.venv, clear=False, no_vcs_ignore=False, global=False)
seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/tmp/tmp_xhpq3_b)
added seed packages: pip==20.3.3, setuptools==51.3.3, wheel==0.36.2
activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator
Successfully created virtual environment!
�[KVirtualenv location: /tmp/tmp_hv_886n/.venv
Locking [dev-packages] dependencies...
Building requirements...
�[K
Resolving dependencies...
�[K
Success!
�[KLocking [packages] dependencies...
Building requirements...
�[K
Resolving dependencies...
�[K
Success!
�[KUpdated Pipfile.lock (30af66)!
An error occurred while installing tensorflow==2.4.1 --hash=sha256:36d5acd60aac48e34bd545d0ce1fb8b3fceebff6b8782436defd0f71c12203bd --hash=sha256:22723b8e1fa83b34f56c349b16a57aaff913b404451fcf70981f2b1d6e0c64fc --hash=sha256:4a04081647b89a8fb602895b29ffc559e3c20aac8bde1d4c5ecd2a65adce5d35 --hash=sha256:0e427b1350be6dbe572f971947c5596fdbb152081f227808d8becd894bf40282 --hash=sha256:e1f2799cc86861680d8515167f103e2207a8cab92a4afe5471e4839330591f08 --hash=sha256:55368ba0bedb513ba0e36a2543a588b5276e9b2ca99fa3232a9a176601a7bab5 --hash=sha256:efa9daa4b3701a4e439b24b74c1e4b66844aee8ae5263fb3cc12281ac9cc9f67 --hash=sha256:eedcf578afde5e6e69c75d796bed41093451cd1ab54afb438760e40fb74a09de --hash=sha256:2357112319303da1b5459a621fd0503c2b2cd97b6c33c4903abd46b3c3e380e2! Will try again.
[InstallError]: File "/home/user/.local/share/virtualenvs/user-5PivF_Od/lib/python3.8/site-packages/pipenv/cli/command.py", line 594, in update
[InstallError]: do_sync(
[InstallError]: File "/home/user/.local/share/virtualenvs/user-5PivF_Od/lib/python3.8/site-packages/pipenv/core.py", line 2884, in do_sync
[InstallError]: do_init(
[InstallError]: File "/home/user/.local/share/virtualenvs/user-5PivF_Od/lib/python3.8/site-packages/pipenv/core.py", line 1304, in do_init
[InstallError]: do_install_dependencies(
[InstallError]: File "/home/user/.local/share/virtualenvs/user-5PivF_Od/lib/python3.8/site-packages/pipenv/core.py", line 899, in do_install_dependencies
[InstallError]: batch_install(
[InstallError]: File "/home/user/.local/share/virtualenvs/user-5PivF_Od/lib/python3.8/site-packages/pipenv/core.py", line 796, in batch_install
[InstallError]: _cleanup_procs(procs, failed_deps_queue, retry=retry)
[InstallError]: File "/home/user/.local/share/virtualenvs/user-5PivF_Od/lib/python3.8/site-packages/pipenv/core.py", line 703, in _cleanup_procs
[InstallError]: raise exceptions.InstallError(c.dep.name, extra=err_lines)
[pipenv.exceptions.InstallError]: Collecting tensorflow==2.4.1
ERROR: Couldn't install package: tensorflow
Package installation failed...
Environment details
Kebechet version: 1.2.4
Python version: 3.8.3
Platform: Linux-4.18.0-193.41.1.el8_2.x86_64-x86_64-with-glibc2.2.5
pipenv version: pipenv, version 2020.11.15
Dependency graph
Unable to obtain dependency graph:
An error occurred while installing tensorflow! Will try again.
[InstallError]: File "/home/user/.local/share/virtualenvs/user-5PivF_Od/lib/python3.8/site-packages/pipenv/cli/command.py", line 233, in install
[InstallError]: retcode = do_install(
[InstallError]: File "/home/user/.local/share/virtualenvs/user-5PivF_Od/lib/python3.8/site-packages/pipenv/core.py", line 2052, in do_install
[InstallError]: do_init(
[InstallError]: File "/home/user/.local/share/virtualenvs/user-5PivF_Od/lib/python3.8/site-packages/pipenv/core.py", line 1304, in do_init
[InstallError]: do_install_dependencies(
[InstallError]: File "/home/user/.local/share/virtualenvs/user-5PivF_Od/lib/python3.8/site-packages/pipenv/core.py", line 899, in do_install_dependencies
[InstallError]: batch_install(
[InstallError]: File "/home/user/.local/share/virtualenvs/user-5PivF_Od/lib/python3.8/site-packages/pipenv/core.py", line 796, in batch_install
[InstallError]: _cleanup_procs(procs, failed_deps_queue, retry=retry)
[InstallError]: File "/home/user/.local/share/virtualenvs/user-5PivF_Od/lib/python3.8/site-packages/pipenv/core.py", line 703, in _cleanup_procs
[InstallError]: raise exceptions.InstallError(c.dep.name, extra=err_lines)
[pipenv.exceptions.InstallError]: Collecting tensorflow
ERROR: Couldn't install package: tensorflow
Package installation failed...
Notes
For more information, see Pipfile and Pipfile.lock.
Once this issue is resolved, the issue will be automatically closed by bot.
use Ray in a pipeline
Is your feature request related to a problem? Please describe.
As Data Scientist,
I would like to use Ray in one step of a pipeline.
Describe the solution you'd like
- Create notebook that can use Ray using non Ray based image.
- Create image using AICoE pipeline with Ray base image (erikerlandson/ray-ubi#2).
- Crete an AI Pipeline in Elyra using KFP with Tekton with one step that uses Ray.
- Extend Pipeline sections of tutorial separating different cases (only CPU, Ray, Spark, GPU, ...)
Describe alternatives you've considered
References
[1] https://www.operate-first.cloud/users/moc-ray-demo/README.md
[2] Spark/ Ray-like pipeline https://github.com/erikerlandson/spark-tekton-demo/blob/main/deploy/pipelines/pipeline-spark-tekton-demo.yaml
[3] Ray CR https://github.com/erikerlandson/ray-odh-demo/blob/main/deploy/operator/ray-test-cluster-cr.yaml
[4] use cases https://docs.google.com/presentation/d/1UxIPa_fEGfYgu1abeyTpK8FAW9NwfTvsKNeFbQMgB-w/edit#slide=id.p
Additional context
[1] could be extended with other ways to use Ray beside spawning image with Ray from JH.
New minor release
Hey, Kebechet!
Create a new minor release, please.
New patch release
Hey, Kebechet!
Create a new patch release, please.
Failed to update dependencies to their latest version
Automatic dependency update failed for the current master with SHA 6c68867.
The automatic dependency management cannot continue. Please fix errors reported bellow.
Command
$ pipenv update --dev
Standard output
Standard error
Warning: Python 3.7 was not found on your system…
You can specify specific versions of Python with:
$ pipenv --python path/to/python
Environment details
Kebechet version: 1.0.9
Python version: 3.6.8
Platform: Linux-4.18.0-147.8.1.el8_1.x86_64-x86_64-with-redhat-8.2-Ootpa
pipenv version: pipenv, version 2018.11.26
Dependency graph
Unable to obtain dependency graph:
Warning: Python 3.7 was not found on your system…
You can specify specific versions of Python with:
$ pipenv --python path/to/python
Notes
For more information, see Pipfile and Pipfile.lock.
Once this issue is resolved, the issue will be automatically closed by bot.
Deliver Container Image
Hey, AICoE-CI!
Please build and deliver the following git tag:
Tag: 0.7.0
New minor release
Hey, Kebechet!
Create a new minor release, please.
Deliver Container Image
Hey, AICoE-CI!
Please build and deliver the following git tag:
Tag: v0.2.2
Use prometheus-flask-exporter to gather endpoints metrics
Create a notebook using KFP libraries to submit a KFP with Tekton
Is your feature request related to a problem? Please describe.
As an alternative to the current way of running KFP.
Describe the solution you'd like
Have one optional notebook as backup using KFP Python SDK to run an AI pipeline to show case the behavior with Tekton.
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Reference: https://github.com/kubeflow/kfp-tekton/blob/master/sdk/README.md#sdk-packages-overview
Reference: https://pypi.org/project/kfp-tekton/
Reference: https://github.com/elyra-ai/kfp-notebook
Deliver Container Image
Hey, AICoE-CI!
Please build and deliver the following git tag:
Tag: v0.7.0
Browser stuck when running locally.
Just linking this issue here, so that people do not wonder where's the problem:
opendatahub-io/s2i-lab-elyra#9
Happens to me with 0.0.4 or 0.0.5 (quay.io/thoth-station/s2i-lab-elyra:v0.0.5).
Show the use of overlays handled from Jupyterlab and the release of different images matching step of an Elyra pipeline
Is your feature request related to a problem? Please describe.
As a Data Scientist,
I want to run an Elyra Pipeline,
with different dependencies for my notebooks.
Hence I would like to use overlays and create images for each step that satisfy each specific notebook requirements
The result should be a demo on Op1st.
Describe the solution you'd like
A clear and concise description of what you want to happen.
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Add any other context or screenshots about the feature request here.
/kind feature
/sig user-experience
embed in behave report once feature is available > 1.2.6
This issue was generated by todo based on a TODO comment in 3325228 when #148 was merged. cc @pacospace.
Kebechet info
New minor release
Hey, Kebechet!
Create a new minor release, please.
serve model as a function
Is your feature request related to a problem? Please describe.
As a Data Scientist,
I want to OpenShift Serverless/FaaS
as that it can serve my model
Describe the solution you'd like
tbd
Describe alternatives you've considered
n/a
Additional context
n/a
/kind feature
create grafana dashboard for app
- setup servicemonitor
thoth-deployment-examplesnamespace onoperate-first - make prometheus scrape http://elyra-aidevsecops-tutorial-thoth-deployment-examples.apps.cnv.massopen.cloud/metrics
- add grafana dashboard to
operate-firstfor the app
Update serving portion of demo to use Seldon
Currently the tutorial uses a custom flask app to deploy the trained model. We should update this to use Seldon and deploy it on the moc zero cluster.
Acceptance Criteria
- Tutorial model is served using Seldon
New minor release
Hey, Kebechet!
Create a new minor release, please.
No dependency management found
No dependency management found for this repository. If you want to keep your dependencies managed, please submit Pipfile or requirements.in or requirements-dev.in file.
To generate a Pipfile, use:
$ pipenv install --skip-lock --code ./
$ git add Pipfile
$ git commit -m 'Add Pipfile for dependency management'
Make sure your Pipfile or requirements.in or requirements-dev.in is placed in the root of your Git repository.
Deployment is requesting more cpu then required
The deployment of the aidevsecops-test has requested 2 cpu cores, but isn't using them fully.
As we can notice, the deployment is only utilizing 1m cpu.
Failed to update dependencies to their latest version
Automatic dependency update failed for the current master with SHA 5913fa9.
The automatic dependency management cannot continue. Please fix errors reported bellow.
Command
$ pipenv update --dev
Standard output
Running $ pipenv lock then $ pipenv sync.
Installing dependencies from Pipfile.lock (4b931a)...
Installing initially failed dependencies...
Standard error
Creating a virtualenv for this project...
Pipfile: /tmp/tmpceval3bp/Pipfile
Using /home/user/.local/share/virtualenvs/user-5PivF_Od/bin/python3.8 (3.8.3) to create virtualenv...
created virtual environment CPython3.8.3.final.0-64 in 662ms
creator CPython3Posix(dest=/tmp/tmpceval3bp/.venv, clear=False, no_vcs_ignore=False, global=False)
seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/tmp/tmpzrmngsob)
added seed packages: pip==20.3.1, setuptools==51.0.0, wheel==0.36.1
activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator
Successfully created virtual environment!
�[KVirtualenv location: /tmp/tmpceval3bp/.venv
Locking [dev-packages] dependencies...
Building requirements...
�[K
Resolving dependencies...
�[K
Success!
�[KLocking [packages] dependencies...
Building requirements...
�[K
Resolving dependencies...
�[K
Success!
�[KUpdated Pipfile.lock (4b931a)!
An error occurred while installing tensorflow==2.4.1 --hash=sha256:0e427b1350be6dbe572f971947c5596fdbb152081f227808d8becd894bf40282 --hash=sha256:efa9daa4b3701a4e439b24b74c1e4b66844aee8ae5263fb3cc12281ac9cc9f67 --hash=sha256:4a04081647b89a8fb602895b29ffc559e3c20aac8bde1d4c5ecd2a65adce5d35 --hash=sha256:22723b8e1fa83b34f56c349b16a57aaff913b404451fcf70981f2b1d6e0c64fc --hash=sha256:2357112319303da1b5459a621fd0503c2b2cd97b6c33c4903abd46b3c3e380e2 --hash=sha256:55368ba0bedb513ba0e36a2543a588b5276e9b2ca99fa3232a9a176601a7bab5 --hash=sha256:e1f2799cc86861680d8515167f103e2207a8cab92a4afe5471e4839330591f08 --hash=sha256:eedcf578afde5e6e69c75d796bed41093451cd1ab54afb438760e40fb74a09de --hash=sha256:36d5acd60aac48e34bd545d0ce1fb8b3fceebff6b8782436defd0f71c12203bd! Will try again.
[InstallError]: File "/home/user/.local/share/virtualenvs/user-5PivF_Od/lib/python3.8/site-packages/pipenv/cli/command.py", line 594, in update
[InstallError]: do_sync(
[InstallError]: File "/home/user/.local/share/virtualenvs/user-5PivF_Od/lib/python3.8/site-packages/pipenv/core.py", line 2884, in do_sync
[InstallError]: do_init(
[InstallError]: File "/home/user/.local/share/virtualenvs/user-5PivF_Od/lib/python3.8/site-packages/pipenv/core.py", line 1304, in do_init
[InstallError]: do_install_dependencies(
[InstallError]: File "/home/user/.local/share/virtualenvs/user-5PivF_Od/lib/python3.8/site-packages/pipenv/core.py", line 899, in do_install_dependencies
[InstallError]: batch_install(
[InstallError]: File "/home/user/.local/share/virtualenvs/user-5PivF_Od/lib/python3.8/site-packages/pipenv/core.py", line 796, in batch_install
[InstallError]: _cleanup_procs(procs, failed_deps_queue, retry=retry)
[InstallError]: File "/home/user/.local/share/virtualenvs/user-5PivF_Od/lib/python3.8/site-packages/pipenv/core.py", line 703, in _cleanup_procs
[InstallError]: raise exceptions.InstallError(c.dep.name, extra=err_lines)
[pipenv.exceptions.InstallError]: Collecting tensorflow==2.4.1
ERROR: Couldn't install package: tensorflow
Package installation failed...
Environment details
Kebechet version: 1.2.3
Python version: 3.8.3
Platform: Linux-4.18.0-193.29.1.el8_2.x86_64-x86_64-with-glibc2.2.5
pipenv version: pipenv, version 2020.11.15
Dependency graph
Unable to obtain dependency graph:
An error occurred while installing tensorflow! Will try again.
[InstallError]: File "/home/user/.local/share/virtualenvs/user-5PivF_Od/lib/python3.8/site-packages/pipenv/cli/command.py", line 233, in install
[InstallError]: retcode = do_install(
[InstallError]: File "/home/user/.local/share/virtualenvs/user-5PivF_Od/lib/python3.8/site-packages/pipenv/core.py", line 2052, in do_install
[InstallError]: do_init(
[InstallError]: File "/home/user/.local/share/virtualenvs/user-5PivF_Od/lib/python3.8/site-packages/pipenv/core.py", line 1304, in do_init
[InstallError]: do_install_dependencies(
[InstallError]: File "/home/user/.local/share/virtualenvs/user-5PivF_Od/lib/python3.8/site-packages/pipenv/core.py", line 899, in do_install_dependencies
[InstallError]: batch_install(
[InstallError]: File "/home/user/.local/share/virtualenvs/user-5PivF_Od/lib/python3.8/site-packages/pipenv/core.py", line 796, in batch_install
[InstallError]: _cleanup_procs(procs, failed_deps_queue, retry=retry)
[InstallError]: File "/home/user/.local/share/virtualenvs/user-5PivF_Od/lib/python3.8/site-packages/pipenv/core.py", line 703, in _cleanup_procs
[InstallError]: raise exceptions.InstallError(c.dep.name, extra=err_lines)
[pipenv.exceptions.InstallError]: Collecting tensorflow
ERROR: Couldn't install package: tensorflow
Package installation failed...
Notes
For more information, see Pipfile and Pipfile.lock.
Once this issue is resolved, the issue will be automatically closed by bot.
Failed to update dependencies to their latest version
Automatic dependency update failed for the current master with SHA 7bfcbde.
The automatic dependency management cannot continue. Please fix errors reported bellow.
Command
$ pipenv update --dev
Standard output
Running $ pipenv lock then $ pipenv sync.
Installing dependencies from Pipfile.lock (4b931a)...
Installing initially failed dependencies...
Standard error
Creating a virtualenv for this project...
Pipfile: /tmp/tmpbbddunfx/Pipfile
Using /home/user/.local/share/virtualenvs/user-5PivF_Od/bin/python3.8 (3.8.3) to create virtualenv...
created virtual environment CPython3.8.3.final.0-64 in 834ms
creator CPython3Posix(dest=/tmp/tmpbbddunfx/.venv, clear=False, no_vcs_ignore=False, global=False)
seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/tmp/tmpeec7es_d)
added seed packages: pip==20.3.1, setuptools==51.0.0, wheel==0.36.1
activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator
Successfully created virtual environment!
�[KVirtualenv location: /tmp/tmpbbddunfx/.venv
Locking [dev-packages] dependencies...
Building requirements...
�[K
Resolving dependencies...
�[K
Success!
�[KLocking [packages] dependencies...
Building requirements...
�[K
Resolving dependencies...
�[K
Success!
�[KUpdated Pipfile.lock (4b931a)!
An error occurred while installing tensorflow==2.4.1 --hash=sha256:4a04081647b89a8fb602895b29ffc559e3c20aac8bde1d4c5ecd2a65adce5d35 --hash=sha256:0e427b1350be6dbe572f971947c5596fdbb152081f227808d8becd894bf40282 --hash=sha256:36d5acd60aac48e34bd545d0ce1fb8b3fceebff6b8782436defd0f71c12203bd --hash=sha256:22723b8e1fa83b34f56c349b16a57aaff913b404451fcf70981f2b1d6e0c64fc --hash=sha256:efa9daa4b3701a4e439b24b74c1e4b66844aee8ae5263fb3cc12281ac9cc9f67 --hash=sha256:55368ba0bedb513ba0e36a2543a588b5276e9b2ca99fa3232a9a176601a7bab5 --hash=sha256:eedcf578afde5e6e69c75d796bed41093451cd1ab54afb438760e40fb74a09de --hash=sha256:e1f2799cc86861680d8515167f103e2207a8cab92a4afe5471e4839330591f08 --hash=sha256:2357112319303da1b5459a621fd0503c2b2cd97b6c33c4903abd46b3c3e380e2! Will try again.
[InstallError]: File "/home/user/.local/share/virtualenvs/user-5PivF_Od/lib/python3.8/site-packages/pipenv/cli/command.py", line 594, in update
[InstallError]: do_sync(
[InstallError]: File "/home/user/.local/share/virtualenvs/user-5PivF_Od/lib/python3.8/site-packages/pipenv/core.py", line 2884, in do_sync
[InstallError]: do_init(
[InstallError]: File "/home/user/.local/share/virtualenvs/user-5PivF_Od/lib/python3.8/site-packages/pipenv/core.py", line 1304, in do_init
[InstallError]: do_install_dependencies(
[InstallError]: File "/home/user/.local/share/virtualenvs/user-5PivF_Od/lib/python3.8/site-packages/pipenv/core.py", line 899, in do_install_dependencies
[InstallError]: batch_install(
[InstallError]: File "/home/user/.local/share/virtualenvs/user-5PivF_Od/lib/python3.8/site-packages/pipenv/core.py", line 796, in batch_install
[InstallError]: _cleanup_procs(procs, failed_deps_queue, retry=retry)
[InstallError]: File "/home/user/.local/share/virtualenvs/user-5PivF_Od/lib/python3.8/site-packages/pipenv/core.py", line 703, in _cleanup_procs
[InstallError]: raise exceptions.InstallError(c.dep.name, extra=err_lines)
[pipenv.exceptions.InstallError]: Collecting tensorflow==2.4.1
ERROR: Couldn't install package: tensorflow
Package installation failed...
Environment details
Kebechet version: 1.2.3
Python version: 3.8.3
Platform: Linux-4.18.0-193.29.1.el8_2.x86_64-x86_64-with-glibc2.2.5
pipenv version: pipenv, version 2020.11.15
Dependency graph
Unable to obtain dependency graph:
An error occurred while installing tensorflow! Will try again.
[InstallError]: File "/home/user/.local/share/virtualenvs/user-5PivF_Od/lib/python3.8/site-packages/pipenv/cli/command.py", line 233, in install
[InstallError]: retcode = do_install(
[InstallError]: File "/home/user/.local/share/virtualenvs/user-5PivF_Od/lib/python3.8/site-packages/pipenv/core.py", line 2052, in do_install
[InstallError]: do_init(
[InstallError]: File "/home/user/.local/share/virtualenvs/user-5PivF_Od/lib/python3.8/site-packages/pipenv/core.py", line 1304, in do_init
[InstallError]: do_install_dependencies(
[InstallError]: File "/home/user/.local/share/virtualenvs/user-5PivF_Od/lib/python3.8/site-packages/pipenv/core.py", line 899, in do_install_dependencies
[InstallError]: batch_install(
[InstallError]: File "/home/user/.local/share/virtualenvs/user-5PivF_Od/lib/python3.8/site-packages/pipenv/core.py", line 796, in batch_install
[InstallError]: _cleanup_procs(procs, failed_deps_queue, retry=retry)
[InstallError]: File "/home/user/.local/share/virtualenvs/user-5PivF_Od/lib/python3.8/site-packages/pipenv/core.py", line 703, in _cleanup_procs
[InstallError]: raise exceptions.InstallError(c.dep.name, extra=err_lines)
[pipenv.exceptions.InstallError]: Collecting tensorflow
ERROR: Couldn't install package: tensorflow
Package installation failed...
Notes
For more information, see Pipfile and Pipfile.lock.
Once this issue is resolved, the issue will be automatically closed by bot.
Update tutorial to add option to submit pipeline from CLI if necessary
Is your feature request related to a problem? Please describe.
See: aicoe-aiops/ocp-ci-analysis#124 (comment)
Describe the solution you'd like
- Add New section in the documentation on how to use elyra cli.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.