Comments (8)
Tagging @oarriaga for possible interest as per patrick-kidger/equinox#143
from eqxvision.
Hey great work with the repo! I can make a PR. have you thought about where to host the weights? Currently for https://github.com/oarriaga/paz I have another repo hosting any pre-trained weights.
from eqxvision.
Hi,
The weights is the tricky bit. I was initially planning to not train on my own and leverage existing weights of say torchvision (or timm) for this task (#4).
- The pro being that it will reduce the effort of finding computational resources for ever increasing number of computer vision models (and their size variants).
- The con is that it does not guarantee that after successful loading of weights from other framework will work as intended due to slight implementation level deviations of few layers. I was initially plan to test porting of weights before diving into full-fledged training.
Thoughts?
from eqxvision.
Yes I think training is not a viable option. I meant to ask where to host any weights of pre-trained models that have already been ported. For example, for the VGG model I coded in equinox, I have already ported the weights too, and I serialized them in an equinox file. Thus, you could host that file in another repo and download it automatically when the user instantiates the model i.e. keras-style.
I think that model deviations might not exist. Specially for networks that only do Conv2D, MaxPool, Dropout. Maybe BatchNorm could be an issue in equinox? But even then the forward pass should be simple. There were a few caveats when porting the weights from VGG mostly the Flatten layer which required to move the axis before the reshape. But for the other models that you have coded, I think porting weights should not be an issue. Specially if one is porting from another channels first repo framework like pytorch.
from eqxvision.
Ahh right. I should have explained it better.
So what I meant by conversion using torchvision
or timm
was that we can directly rely on the weights hosted by these libraries and do something like URLs -> download -> convert -> load. This way we don't have to host the .eqx
weights as well.
I doubt conversion will be time consuming. Thoughts?
On the convertor
front, perhaps, it is not feasible to have a common convertor
function across package and we can start small with per-model convertor. Or a mix of two, have common conversion rules and then assembling can be handled per model. You have more experience on this so you'll be able to weigh in better. The starting point can be the conversion-code
you already have!
One thing I wanted to add to VGG implementation was supporting _bn
variants as well for VGGs to be in line with models supported on torchvision. Perhaps we should discuss this in a PR in more detail.
from eqxvision.
I see, yes this makes sense. I am still thinking about whether or not it might be computationally inefficient to have a function that translates the weights every single time a users requests it, or rather do it only once and host "duplicated" weights somewhere else. It seems like memory vs time complexity debate.
My translation code is not generic enough and it goes from keras -> equinox. I think you are considering going from timm's repo so it will not work directly.
I can gladly make the PR with the VGG model that I have, but let me know how you want to proceed with the translation or weight hosting.
from eqxvision.
Yes, that is true regarding the memory vs time complexity debate. Any insights as to how much time it takes to make a single conversion? If you wish to share the script for conversion I can then tweak and check for PyTorch weights. I am not familiar with Keras' saving and loading of weights to comment on it. At the moment, I am willing to increase load time by 10% of the entire process (60 seconds to download weights then 6 seconds to convert and load). After seeing some numbers on the time spent at different steps it'll be easier to settle the debate. What say?
Lets start with VGG integration and chase weight conversion/hosting in a separate thread?
from eqxvision.
OK great a basic PR with the model is here now #10
from eqxvision.
Related Issues (13)
- Adding deserialization from torch saved weights HOT 1
- Adding training jupyter notebooks HOT 7
- Need to run `mkdocs` twice. HOT 1
- Adding grad check in test cases HOT 1
- Removing dependency on `torch` and `torchvision`
- Adding a table comparing Imagenet performance of ported networks. HOT 2
- Finetuning VGGs on a small dataset
- inference mode HOT 2
- Adding pre-trained meta data
- equinox.custom_types > replace with jaxtyping HOT 2
- BatchNorm is no longer experimental HOT 1
- Is this project still maintained? HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from eqxvision.