Comments (7)
We're working on releasing the training code but cannot give a precise ETA yet. It will take at least a few more weeks. I'll keep you posted.
from scenic.
We just added fine-tuning instructions to the README.
The config is very similar to the from-scratch config. The only crucial difference is that instead of config.init_from.codebase = 'clip'
, it uses config.init_from.checkpoint_path = 'path/to/parent/checkpoint'
.
However, some other changes may be necessary to get good fine-tuning result. They depend on the target dataset, so I can make only general recommendations:
- If the dataset is small, very short fine-tuning schedules might be optimal. I would try values for
config.num_training_steps
starting as low as 500 steps and then keep doubling the value until performance is optimal. - The default learning rate is a good starting point, but it should be tuned. Try going up and down by a factor of 2 or 3.
- Carefully go through the preprocessing settings and check if they make sense for your application. For example
remove_forbidden_labels
is only needed to measure zero-shot LVIS performance and therefore not included in the fine-tuning config. Also, if you have a dataset with a small, fixed label space, then you should probably get rid of theadd_random_prompts
op and simply add all categories that are not present in an image as negative labels directly in the dataset decoder. - Input resolution should be tuned. If your dataset contains mostly large objects, a smaller input resolution may lead to better results (and would be faster).
from scenic.
We just published the training code.
Please let us know if you have any questions.
from scenic.
@mjlm hello, thanks for your fancy work! When the all of codes can be updated? Please offer the full codes which include the configs for experiments published in the paper. Thanks!
from scenic.
Hi there! Exciting news about the training code, thanks a lot!
When could we expect finetuning instructions? Is it mostly similar to training detection from scratch (some smaller learning rate here and there)? Or are there any other tips to be aware of? :)
thanks so much!
from scenic.
Hi,
is there a complete example how to train Owl-vit? From my dataset having a lot of images, what I have to do? Such commands are quite cryptic. Thanks
from scenic.
hi, i have a dataset of only images, no text_queries and i want to use it as image one shot detection at the end, can i finetune the same way ?
from scenic.
Related Issues (20)
- UniVRD model release? HOT 2
- scenic/scenic/projects /mbt bottleneck shape shape?
- CUDA12.3 install
- Dockerfile for running SCENIC+CUDA
- What is the file format of the checkpoint files? HOT 1
- Focal loss in OWL-ViT HOT 6
- playground.ipynb of OWL-ViT starts having dependency and import error on Google Colab since Nov. 15 HOT 4
- AttributeError: module 'numpy.linalg._umath_linalg' has no attribute '_ilp64' HOT 3
- Clip implementation with Hugging Face Datasets
- Inconsistent Object Detection Results on Constant Video Stream and Text Query
- Knowledge Distillation - OwlViT Model
- OWL-ViT inference playround: ModuleNotFoundError: No module named 'jax.experimental.gda_serialization' HOT 1
- Full training example with custom dataset HOT 1
- I would like to ask for the code release of UniVRD (ICCV 2023). HOT 1
- Port Vid2Seq to use new train_lib
- not module named ’grain.python‘
- local checkpoint error
- Requirements for Vid2Seq
- finetune owl on image dataset with no caption
- CLIP feature compression package for the ActivityNet Captions dataset
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from scenic.