Comments (7)
embedding as an object that has save_to_npy() and to_list() method
Sounds reasonable. Let me have a look.
also let me know of you planning something else in mind?
Zero-shot image labelling will be also exposed to Python soon --it's now implemented in C++ examples but I'll move it to the main lib and then expose to Python.
In fact all these started just for fun, but there seems to be a community interest in this so I'm thinking of pushing more features over time and optimizing performance.
from clip.cpp.
Yes there was a bug in batch inference, I'll expose it to Python after I make sure that it's completely fixed.
We can also support writing directly to a Numpy file (*.npy). Numpy file structure is quite simple and I can implement it in C++ without any dependency on Numpy. In this case, it could be something like
model.load_preprocess_encode_images(
image_paths # List[str]
to_numpy_file='image_embeddings.npy' # str, optional
)
WDYT?
from clip.cpp.
mmm.., I don't think it's great idea not right now at least, here's why since you support saving in specific format you should or expected to support to be loaded in that same format, second saving the embedding is not the real issue and beyond clip.cpp core, it's getting them in the first place is the deal breaker and expensive to do in python, at least not right now see Better Solution
if it's really not that hard you could do that (don't recommend it - so out of the scope for now), i.e:
# embedding as an object that has save_to_npy() and to_list() method
embeddings = model.load_preprocess_encode_images(image_files)
embeddings.save_to_npy()
embeddings.to_list() # which will return the python list
Better Solution
I am planning to refactor the python interface (not so much just some naming convention and some new methods) to be as close as much to that of sentence_transformers
package for ease of use and familiarity for devs, I will raise an issue for how the interface will look like for a review to start right into it but it need batch inference
also let me know of you planning something else in mind?
from clip.cpp.
Zero-shot image labelling will be also exposed to Python soon --it's now implemented in C++ examples but I'll move it to the main lib and then expose to Python.
keep up the good work ❤️
In fact all these started just for fun, but there seems to be a community interest in this so I'm thinking of pushing more features over time and optimizing performance.
isn't that how 90% of great projects and frameworks started😊.
from clip.cpp.
Hey @Yossef-Dawoad, with #75, now we're using GGUF and older model files are not usable. can you please update the notebook?
from clip.cpp.
Sure, will do it tomorrow.
any luck 🤞 of getting batch inference fixed ?
from clip.cpp.
Batch inference is working on the C/C++ side, it needs to be exposed to Python. I still have some considerations about how to expose it. The issue is, if we try to read all the embeddings at once from the native side to Python, it might be a bad user experience, so instead a generator or a custom data structure that can read embeddings lasily could be preferable. I'll try to pack it ASAP. After the GGUF support in clip.cpp, I started to work on lmm.cpp, sister of this project. So I can try to release batch inference in Python tomorrow but if it takes longer than I expected it might be delayed after the release of lmm.cpp.
from clip.cpp.
Related Issues (20)
- python binding: OSError libggml.so: cannot open shared object file HOT 4
- Migrate to GGUF HOT 4
- Move ZSL implementation to `clip` lib as a function
- Support downloading models in Python bindings HOT 1
- Introduce Java bindings
- Support batch inference for models other than patch32 HOT 3
- Implement bicubic interpolation
- Can u please make exe of this project? HOT 2
- no module named 'gguf' HOT 2
- Metal support? HOT 5
- Building with -DCLIP_BUILD_IMAGE_SEARCH=ON for image-search fails, ‘cos_gt’ is not a member of ‘unum::usearch’ HOT 4
- Vision only model memory issue
- Slower image encode the lower the quantization HOT 1
- Memory leak: clip_tokenize and clip_image_preprocess
- Question about the CLIP model in clip.cpp and llama.cpp HOT 1
- include clip.cpp in another project
- Python Bindings, optionally pass PIL image
- Python Bindings distributions for different architectures
- Python Binding Example: AttributeError: dlsym(0x8ea2fac0, make_clip_image_u8): symbol not found HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from clip.cpp.