Comments (3)
Hello @trent-s, thank you for this report! I will update the docker image as soon as I can, in the meanwhile you can install speedster using the Quick installation
steps that you can find here: installation
from nebuly.
Thank you @valeriosofi for your kind and timely response.
I followed the Quick installation steps as you suggested, and still got similar results.
Just FYI, I am posting the latest log here.
root@66f17ec15c70:/# python
Python 3.8.10 (default, Mar 13 2023, 10:26:41)
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
>>> import torchvision.models as models
>>> from speedster import optimize_model
2023-06-21 02:33:42.382076: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2023-06-21 02:33:42.432123: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2023-06-21 02:33:44.244064: W tensorflow/core/common_runtime/gpu/gpu_device.cc:1956] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform.
Skipping registering GPU devices...
>>>
>>> model = models.resnet50()
>>> input_data = [((torch.randn(1, 3, 256, 256), ), torch.tensor([0])) for _ in range(100)]
>>>
>>> optimized_model = optimize_model(
... model,
... input_data=input_data,
... optimization_time="constrained",
... metric_drop_ths=0.05
... )
2023-06-21 02:33:53 | INFO | Running Speedster on GPU:0
2023-06-21 02:33:59 | INFO | Benchmark performance of original model
2023-06-21 02:33:59 | INFO | Original model latency: 0.004133939743041992 sec/iter
============= Diagnostic Run torch.onnx.export version 2.0.0+cu118 =============
verbose: False, log level: Level.ERROR
======================= 0 NONE 0 NOTE 0 WARNING 0 ERROR ========================
2023-06-21 02:34:02 | INFO | [1/2] Running PyTorch Optimization Pipeline
2023-06-21 02:34:02 | INFO | Optimizing with PytorchBackendCompiler and q_type: None.
2023-06-21 02:34:12 | INFO | Optimized model latency: 0.003423929214477539 sec/iter
2023-06-21 02:34:12 | INFO | Optimizing with PytorchBackendCompiler and q_type: QuantizationType.HALF.
2023-06-21 02:34:23 | INFO | Optimized model latency: 0.004581451416015625 sec/iter
2023-06-21 02:34:23 | INFO | Optimizing with PyTorchApacheTVMCompiler and q_type: None.
2023-06-21 02:41:47 | INFO | Optimized model latency: 0.007593631744384766 sec/iter
2023-06-21 02:41:47 | INFO | Optimizing with PyTorchApacheTVMCompiler and q_type: QuantizationType.HALF.
2023-06-21 02:50:06 | INFO | Optimized model latency: 0.007908344268798828 sec/iter
2023-06-21 02:50:06 | INFO | Optimizing with PyTorchApacheTVMCompiler and q_type: QuantizationType.DYNAMIC.
2023-06-21 02:58:06 | WARNING | The optimized model will be discarded due to poor results obtained with the given metric.
2023-06-21 02:58:06 | INFO | [2/2] Running ONNX Optimization Pipeline
2023-06-21 02:58:06 | INFO | Optimizing with ONNXCompiler and q_type: None.
Inconsistency detected by ld.so: dl-version.c: 205: _dl_check_map_versions: Assertion `needed != NULL' failed!
from nebuly.
@valeriosofi i would like to ask you if there is any progress on this issue
from nebuly.
Related Issues (20)
- [chatllama]Do I need to split the llama model manully? HOT 2
- [Chatllama] facebook/opt-350m is missing in rlhf/model_list.py HOT 2
- module not found:chatllama.rlhf.dataset HOT 1
- Support for torch 2.0 HOT 1
- Issues with accelerate and deepspeed training HOT 4
- [chatllama]How models enable inference HOT 1
- [Chatllama] what's supposed to be in the Actor checkpoint dir? HOT 3
- [chatllama]Puzzled about the update of the critic model
- [Speedster] Optimization failed with PytorchBackendCompiler HOT 4
- yolov8 + nebuly | AttributeError: type object 'DummyClass' has no attribute 'models' HOT 10
- Evaluating accuracy of only the reward model
- torch2.0 support on speedster HOT 2
- Yolov8-Pose Model
- [ Speedster] With Hugging Face notebook code on nebulydocker/nebullvm container: RuntimeError: Expected all tensors to be on the same device HOT 5
- How to generate and perform inference for an ONNX model HOT 2
- Forward Forward Algorithm Questions HOT 2
- [Speedster] TensorRt OSError: [WinError 127] The specified procedure could not be found
- [Speedster] optimize_model took 10 hours, and it's not over yet
- nebullvm LICENSE and commercial use?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from nebuly.