Comments (11)
If you only need offline distribution of your application. use jar:///
is sufficient.
- put your retinaface.pt (and synset.txt, serving.properties files) into a .zip file
- add the .zip file and put it into your distribution .jar file (e.g. /META-INF/models/retinaface.zip)
- use `.optModelUrls("jar:/META-INF/models/retinaface.zip") to load the model
from djl.
By the way, you might want to take a look this demo if you want to work without network, see: https://github.com/deepjavalibrary/djl-demo/tree/master/development/fatjar
from djl.
synset.txt and serving.properties files are optional:
synset.txt
usually used by ImageClassificationTranslator, you might not need for your caseserving.properties
allows you adding arguments and options that will be used for load the model and translator. This makes it easy to distribute the .zip file. Withserving.properties
, you don't need to add them inCritieria
- you can specify which engine to use:
engine=PyTorch
- you can specify which tralsator/translatorFactory use use (can be override by
.optTranslatorFactory()
):transaltorFactory=ai.djl.pytorch.zoo.nlp.qa.PtBertQATranslatorFactory
- You can add arguments that needed by Translator:
width=640
- you can specify which engine to use:
See: https://docs.djl.ai/docs/serving/serving/docs/configurations_model.html
from djl.
- You can load model from your jar file and class path,
.optModelUrls("jar:///my_model.jar")
, you have to put your model in a .zip file - If you are using OnnxRuntime or PyTorch, you can use
Model.laod(InputStream)
api to load from InputStream:
try (Model model = Model.newInstance("resnet18", "PyTorch")) {
model.load(class.getResourceAsStream("..."));
...
}
from djl.
Thank you for your prompt response! :)
Regarding the first option, wouldn't specifying a path to the JAR file still necessitate an absolute path? Since it's later initiated from an EXE and the working directory could be anything, which is beyond my control. However, I do have a system property ("compose.application.resources.dir") that indicates the file's location. Unfortunately, this can be a UNC path.
As for the second approach, it's quite intriguing. I'm attempting to incorporate the RetinaFace detection sample code into my photo app for face detection.
Could you provide guidance on how the official sample should be modified to employ the model.load() approach? How to apply the FaceDetectionTranslator
here?
from djl.
@StefanOltmann
The jar url is not to the jar file, It the file in the classpath (it doesn't really need a jar, any files in the classpath should be fine):
The jar:///ai/djl/utils/model.zip
equals:
ai.djl.util.Utils.class.getResource("model.zip")
file URL should also work: file:///Users/home/model/model.zip
from djl.
Translator
is not involved in model loading. Currently Criteria API cannot handle Streaming model loading (It's possible, but need major refactor). The main reason is because, a model usually contains multiple files, We don't have a good way to streaming in multiple files. API doesn't limit to .zip
file, but underlying implementation only access .zip
in many cases.
Can you provide more context why you need a use InputStream?
from djl.
Can you provide more context why you need a use InputStream?
I want to include RetinaFaceDetector into Ashampoo Photos, a JVM based Desktop app. It should come with the model and engine included, so that it can be installed and be used offline without any downloading of missing resources from the web.
People might put the installation, which has a resource directory including „retinaface.pt“ onto a network drive. So the model should also be loaded from there.
Loading resources using class.getResourceAsStream() (like icons, etc.) is the most reliable way to load resources.
This just has been proven true as loading from a UNC path using the existing API fails.
It must not be a stream. If I can give the whole model as byte array, this would help, too. I assume that Criteria API also reads the whole file bytes behind the scenes.
Can you give me a optModelFromBytes() ?
from djl.
Okay, I will test it and report back. Thank you.
from djl.
Where do I get synset.txt and serving.properties from?
from djl.
The jar:///
indeed works from a network drive. Great.
from djl.
Related Issues (20)
- RuntimeError: The size of tensor a (16) must match the size of tensor b (80) at non-singleton dimension 3 HOT 1
- Hundreds wrong detection on yolov8 HOT 14
- SIGSEGV faults
- When I build a project supporting DJL in Android Studio, I encounter the following error. How can I resolve it? HOT 3
- Cannot use trained gluonTS model
- Adding ppc64le support for PyTorch engine HOT 4
- Please report location when HuggingFaceTokenizer.newInstance fails with I/O error HOT 11
- How to support paddlepaddle on an aarch64 FT Phytium S2500/64 machine?
- Run tensorflow model on arm architecture HOT 2
- Could not initialize class ai.djl.onnxruntime.engine.0rtNDManager HOT 4
- NDArrays.concat has a different behaviour as np.concatenate HOT 4
- Potential Race Condition or Garbage Collection Issue during JNI Environment Usage HOT 1
- NumberFormatException: Cannot parse null string when loading inside of Docker HOT 1
- Extracting local model ZIP multiple times HOT 3
- Error encountered when predicting with xgboost on Linux arm64 HOT 2
- SIGSEGV calling NDList.encode() HOT 3
- OnnxRuntime 引擎官网案例报错 HOT 9
- Missing Features Exception while inputting the custom dataset.
- Prediction from clip-ViT-B-32-multilingual-v1 incorrectly returns 768 elements rather than 512 HOT 8
- HuggingFace model multilingual-e5-small fails to open on Windows due to max path limitations HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from djl.