paulescu / hands-on-train-and-deploy-ml Goto Github PK
View Code? Open in Web Editor NEWTrain and Deploy an ML REST API to predict crypto prices, in 10 steps
Home Page: https://www.realworldml.net/subscribe
License: MIT License
Train and Deploy an ML REST API to predict crypto prices, in 10 steps
Home Page: https://www.realworldml.net/subscribe
License: MIT License
Dear @Paulescu ,
I have been working on a personal project with the goal of understanding and implementing some of the most relevant concepts on Machine Learning Operations - MLOps. This personal projects is being developed using this repository as inspiration and guideline.
I am currently finishing this project and everything works fine except for the last step - deployment (using cerebrium). During the running of the Makefile - which "implements" deploy - I am facing a ModuleNotFoundError that I do not understand, which follows bellow.
As mentioned, I do not understand this error, since the structure of the deployment directory is as follows:
Additionaly, I provide my Makefile, so you have more means to understand why I am facing this situation, which besides lots of experimentation I do not understand. I honestly thank you for your time in helping me solve this error and also for the effort of putting together, and free of charge, this project as well all related materials.
Thank you.
Doing a make init
gives me this error.
$ make init
curl -sSL https://install.python-poetry.org | python3 -
Retrieving Poetry metadata
The latest version (1.7.1) is already installed.
poetry install
Updating dependencies
Resolving dependencies... (2.9s)
Package operations: 1 install, 0 updates, 0 removals
• Installing lightgbm (3.3.5): Failed
ChefBuildError
Backend subprocess exited when trying to invoke build_wheel
INFO:root:running bdist_wheel
INFO:root:running build
INFO:root:running build_py
INFO:root:creating build
INFO:root:creating build/lib
INFO:root:creating build/lib/lightgbm
INFO:root:copying lightgbm/callback.py -> build/lib/lightgbm
INFO:root:copying lightgbm/compat.py -> build/lib/lightgbm
INFO:root:copying lightgbm/plotting.py -> build/lib/lightgbm
INFO:root:copying lightgbm/__init__.py -> build/lib/lightgbm
INFO:root:copying lightgbm/engine.py -> build/lib/lightgbm
INFO:root:copying lightgbm/dask.py -> build/lib/lightgbm
INFO:root:copying lightgbm/basic.py -> build/lib/lightgbm
INFO:root:copying lightgbm/libpath.py -> build/lib/lightgbm
INFO:root:copying lightgbm/sklearn.py -> build/lib/lightgbm
INFO:root:running egg_info
INFO:root:writing lightgbm.egg-info/PKG-INFO
INFO:root:writing dependency_links to lightgbm.egg-info/dependency_links.txt
INFO:root:writing requirements to lightgbm.egg-info/requires.txt
INFO:root:writing top-level names to lightgbm.egg-info/top_level.txt
INFO:root:reading manifest file 'lightgbm.egg-info/SOURCES.txt'
INFO:root:reading manifest template 'MANIFEST.in'
WARNING:root:no previously-included directories found matching 'build'
WARNING:root:warning: no files found matching '*.so' under directory 'lightgbm'
WARNING:root:warning: no files found matching '*.so' under directory 'compile'
WARNING:root:warning: no files found matching '*.dll' under directory 'compile/Release'
WARNING:root:warning: no files found matching '*.dll' under directory 'compile/windows/x64/DLL'
WARNING:root:warning: no previously-included files matching '*.py[co]' found anywhere in distribution
WARNING:root:warning: no previously-included files found matching 'compile/external_libs/compute/.git'
INFO:root:adding license file 'LICENSE'
INFO:root:writing manifest file 'lightgbm.egg-info/SOURCES.txt'
INFO:root:copying lightgbm/VERSION.txt -> build/lib/lightgbm
INFO:wheel:installing to build/bdist.macosx-11.0-arm64/wheel
INFO:root:running install
INFO:LightGBM:Starting to compile the library.
INFO:LightGBM:Starting to compile with CMake.
Traceback (most recent call last):
File "<string>", line 95, in silent_call
File "/Users/nbhandari/mambaforge/envs/manother_ml_project_env/lib/python3.10/subprocess.py", line 369, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['make', '_lightgbm', '-I/private/var/folders/gb/xd14qbk12bs52pkzwdsx3qlm0000gp/T/tmpef06d3aq/lightgbm-3.3.5/build_cpp', '-j4']' returned non-zero exit status 2.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/nbhandari/Library/Application Support/pypoetry/venv/lib/python3.11/site-packages/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
main()
File "/Users/nbhandari/Library/Application Support/pypoetry/venv/lib/python3.11/site-packages/pyproject_hooks/_in_process/_in_process.py", line 335, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
File "/Users/nbhandari/Library/Application Support/pypoetry/venv/lib/python3.11/site-packages/pyproject_hooks/_in_process/_in_process.py", line 251, in build_wheel
return _build_backend().build_wheel(wheel_directory, config_settings,
File "/var/folders/gb/xd14qbk12bs52pkzwdsx3qlm0000gp/T/tmpj7eakzvm/.venv/lib/python3.10/site-packages/setuptools/build_meta.py", line 404, in build_wheel
return self._build_with_temp_dir(
File "/var/folders/gb/xd14qbk12bs52pkzwdsx3qlm0000gp/T/tmpj7eakzvm/.venv/lib/python3.10/site-packages/setuptools/build_meta.py", line 389, in _build_with_temp_dir
self.run_setup()
File "/var/folders/gb/xd14qbk12bs52pkzwdsx3qlm0000gp/T/tmpj7eakzvm/.venv/lib/python3.10/site-packages/setuptools/build_meta.py", line 480, in run_setup
super(_BuildMetaLegacyBackend, self).run_setup(setup_script=setup_script)
File "/var/folders/gb/xd14qbk12bs52pkzwdsx3qlm0000gp/T/tmpj7eakzvm/.venv/lib/python3.10/site-packages/setuptools/build_meta.py", line 311, in run_setup
exec(code, locals())
File "<string>", line 334, in <module>
File "/var/folders/gb/xd14qbk12bs52pkzwdsx3qlm0000gp/T/tmpj7eakzvm/.venv/lib/python3.10/site-packages/setuptools/__init__.py", line 103, in setup
return distutils.core.setup(**attrs)
File "/var/folders/gb/xd14qbk12bs52pkzwdsx3qlm0000gp/T/tmpj7eakzvm/.venv/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 185, in setup
return run_commands(dist)
File "/var/folders/gb/xd14qbk12bs52pkzwdsx3qlm0000gp/T/tmpj7eakzvm/.venv/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
dist.run_commands()
File "/var/folders/gb/xd14qbk12bs52pkzwdsx3qlm0000gp/T/tmpj7eakzvm/.venv/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
self.run_command(cmd)
File "/var/folders/gb/xd14qbk12bs52pkzwdsx3qlm0000gp/T/tmpj7eakzvm/.venv/lib/python3.10/site-packages/setuptools/dist.py", line 963, in run_command
super().run_command(command)
File "/var/folders/gb/xd14qbk12bs52pkzwdsx3qlm0000gp/T/tmpj7eakzvm/.venv/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
cmd_obj.run()
File "/var/folders/gb/xd14qbk12bs52pkzwdsx3qlm0000gp/T/tmpj7eakzvm/.venv/lib/python3.10/site-packages/wheel/bdist_wheel.py", line 403, in run
self.run_command("install")
File "/var/folders/gb/xd14qbk12bs52pkzwdsx3qlm0000gp/T/tmpj7eakzvm/.venv/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
self.distribution.run_command(command)
File "/var/folders/gb/xd14qbk12bs52pkzwdsx3qlm0000gp/T/tmpj7eakzvm/.venv/lib/python3.10/site-packages/setuptools/dist.py", line 963, in run_command
super().run_command(command)
File "/var/folders/gb/xd14qbk12bs52pkzwdsx3qlm0000gp/T/tmpj7eakzvm/.venv/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
cmd_obj.run()
File "<string>", line 248, in run
File "<string>", line 199, in compile_cpp
File "<string>", line 99, in silent_call
Exception: An error has occurred while building lightgbm library file
The full version of error log was saved into /Users/nbhandari/LightGBM_compilation.log
at ~/Library/Application Support/pypoetry/venv/lib/python3.11/site-packages/poetry/installation/chef.py:164 in _prepare
160│
161│ error = ChefBuildError("\n\n".join(message_parts))
162│
163│ if error is not None:
→ 164│ raise error from None
165│
166│ return path
167│
168│ def _prepare_sdist(self, archive: Path, destination: Path | None = None) -> Path:
Note: This error originates from the build backend, and is likely not a problem with poetry but with lightgbm (3.3.5) not supporting PEP 517 builds. You can verify this by running 'pip wheel --no-cache-dir --use-pep517 "lightgbm (==3.3.5)"'.
make: *** [init] Error 1
Using mamba as virtual environment. Here are the details:
$ mamba info
__ __ __ __
/ \ / \ / \ / \
/ \/ \/ \/ \
███████████████/ /██/ /██/ /██/ /████████████████████████
/ / \ / \ / \ / \ \____
/ / \_/ \_/ \_/ \ o \__,
/ _/ \_____/ `
|/
███╗ ███╗ █████╗ ███╗ ███╗██████╗ █████╗
████╗ ████║██╔══██╗████╗ ████║██╔══██╗██╔══██╗
██╔████╔██║███████║██╔████╔██║██████╔╝███████║
██║╚██╔╝██║██╔══██║██║╚██╔╝██║██╔══██╗██╔══██║
██║ ╚═╝ ██║██║ ██║██║ ╚═╝ ██║██████╔╝██║ ██║
╚═╝ ╚═╝╚═╝ ╚═╝╚═╝ ╚═╝╚═════╝ ╚═╝ ╚═╝
mamba (1.4.2) supported by @QuantStack
GitHub: https://github.com/mamba-org/mamba
Twitter: https://twitter.com/QuantStack
█████████████████████████████████████████████████████████████
active environment : manother_ml_project_env
active env location : /Users/nbhandari/mambaforge/envs/manother_ml_project_env
shell level : 1
user config file : /Users/nbhandari/.condarc
populated config files : /Users/nbhandari/mambaforge/.condarc
/Users/nbhandari/.condarc
conda version : 23.3.1
conda-build version : not installed
python version : 3.10.12.final.0
virtual packages : __archspec=1=arm64
__osx=14.1.2=0
__unix=0=0
base environment : /Users/nbhandari/mambaforge (writable)
conda av data dir : /Users/nbhandari/mambaforge/etc/conda
conda av metadata url : None
channel URLs : https://conda.anaconda.org/conda-forge/osx-arm64
https://conda.anaconda.org/conda-forge/noarch
package cache : /Users/nbhandari/mambaforge/pkgs
/Users/nbhandari/.conda/pkgs
envs directories : /Users/nbhandari/mambaforge/envs
/Users/nbhandari/.conda/envs
platform : osx-arm64
user-agent : conda/23.3.1 requests/2.31.0 CPython/3.10.12 Darwin/23.1.0 OSX/14.1.2
UID:GID : 502:20
netrc file : /Users/nbhandari/.netrc
offline mode : False
There's nothing in the Github actions section.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.