Comments (10)
We have released the API for FastSAM. Please see for details at fastsam/decoder.py.
import model
from fastsam import FastSAM ,FastSAMDecoder
model = FastSAM('./weights/FastSAM.pt')
fastsam = FastSAMDecoder(model,device=DEVICE,retina_masks=True,imgsz=1024,conf=0.4,iou=0.9)
Encoder
image_embedding = fastsam.run_encoder(image)
Decoder
ann = fastsam.run_decoder(image_embedding,point_prompt=[[506, 340]], point_label=[1])
from fastsam.
We have released the API for FastSAM. Please see for details at fastsam/decoder.py.
import model
from fastsam import FastSAM ,FastSAMDecoder model = FastSAM('./weights/FastSAM.pt') fastsam = FastSAMDecoder(model,device=DEVICE,retina_masks=True,imgsz=1024,conf=0.4,iou=0.9)Encoder
image_embedding = fastsam.run_encoder(image)Decoder
ann = fastsam.run_decoder(image_embedding,point_prompt=[[506, 340]], point_label=[1])
Thanks for the updates. However, it seems that the 'image_embeddings' are actually wrapped results including masks and boxes.
Is it possible to get the intermediate results such as the encoded feature maps, such as that in the original SAM repository? THe encoded features will be more valuable for adapting to down-stream tasks.
Looking forward to your reply.
from fastsam.
We can also convert the model to ONNX to remove the dependence on PyTorch, which has a very big size compared to ONNXRuntime. My code for the original SAM is here.
We have referred to the YOLOv8 to onnx tutorial to convert onnx, and the code to inference directly with onnx is planned to be released in the near future.
from fastsam.
Hello! I really like this project. Do you plan to support splitting this model into Encoder and Decoder like the original SAM? In that way, the Decoder part can be run very fast, and we can apply it to some applications like AnyLabeling. I'd love to help integrate into AnyLabeling if we can find a way to split the model. Thank you very much!
Our approach is divided into two phases: All-instance Segment and Prompt-guided Selection. The first stage can be considered as the encoder and the second stage as the decoder. The integration into AnyLabeling is also feasible. Thank you for your suggestion. We will package it as a SAM-like Encoder and Decoder in the near future.
from fastsam.
- It's best if you can divide the model into 2 small models: Encoder and Decoder. Encoder part (your backbone) can be calculated separately. In this way, we can precalculate the encoder, each time user adds some boxes or points, we only have to calculate the decoder again, with the result from the encoder, and the prompt. This action can be really fast.
- Convert model to ONNX format to reduce the dependencies. The binary distribution of AnyLabeling is ~130MB (you can see it here). That means users only have to download ~130MB to run the model. If we use .pth model, we need PyTorch, which is about 700MB to 1GB+, this will cost much more time and space to download the program.
We can calculate the encoder as following:
image_embedding = run_encoder(image)
And calculate the decoder after that:
image = run_decoder(
image_embedding,
prompt,
)
from fastsam.
We can also convert the model to ONNX to remove the dependence on PyTorch, which has a very big size compared to ONNXRuntime. My code for the original SAM is here.
from fastsam.
Hello! I really like this project. Do you plan to support splitting this model into Encoder and Decoder like the original SAM? In that way, the Decoder part can be run very fast, and we can apply it to some applications like AnyLabeling. I'd love to help integrate into AnyLabeling if we can find a way to split the model. Thank you very much!
Our approach is divided into two phases: All-instance Segment and Prompt-guided Selection. The first stage can be considered as the encoder and the second stage as the decoder. The integration into AnyLabeling is also feasible. Thank you for your suggestion. We will package it as a SAM-like Encoder and Decoder in the near future.
Thanks. I'm waiting for it too.
from fastsam.
We have created a new branch to integrate AnyLabeling. We have divided the functionality into three functions: point_prompt, box_prompt, and text_prompt, which can be seen as decoders for the model. Could you provide a more detailed description of the specific functionalities we need to encapsulate?😊
from fastsam.
Is there any update on the onnx export part for both the decoder and encoder parts?
from fastsam.
any update?
from fastsam.
Related Issues (20)
- Problems during training
- Assertion Error
- Model returns None when used in Google Colab
- text model is not correct box
- How to convert annotation files of the SA-1B dataset into the format compatible with YOLOv8
- about network: FastSAM is only to train a YOLOV8-seg? HOT 17
- SA-1B dataset HOT 1
- Working with FastSAM results HOT 2
- visualize mask branch
- Why the number of parameters of FastSAM different from Yolov8-seg?
- fine-tuning FastSAM with medical CT scans
- Hello, I am trying to reproduce your validation data on the coco dataset, but I have encountered the following problems
- Hello, I am using the verification code published by you, but I have encountered many problems. Could you please publish a usage instruction document Thank you.
- whole image predict
- Return Nothing With CUDA DEVICE
- Finetuning the CLIP model used in FastSAM
- Segment by similarity to a reference image
- Where is the tensorrt C++ deployment
- Wrong license in readme after commit 279ceab2b656801917fd2d84e182a80f5d20bf8a HOT 1
- object proposal cocoeval - how did you get your SAM result? HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from fastsam.