Comments (2)
The models trained on CIFAR-10 that we discuss in the appendix actually do this, but that's because the patch size itself is 1.
For the ImageNet experiments, stride = 1 would result in a very large "internal resolution", which would make the model computationally infeasible.
And this wouldn't really result in a model like MobileNet -- our model is actually quite a bit more simple than MobileNet.
The fact that our model uses the patch embedding layer is a key aspect of our work -- achieving the highest possible accuracy was not our goal. Specifically, we intended to investigate the effect of using patches as the input representation, in the context of recent work on ViT and MLP-Mixer and their variants. If you remove the patch embedding stem, you would ultimately build a very different model.
There are definitely various ways you could tweak the architecture to increase performance (in terms of accuracy or throughput) while decreasing its simplicity, but the patch embedding stem itself would not be the first choice. :) This is definitely important future work, but it's secondary to the point of the paper.
from convmixer.
I'm going to close this issue, but feel free to reopen it or open a new issue if you have more questions.
from convmixer.
Related Issues (16)
- is single gpu training possible? HOT 2
- License HOT 2
- Cifar10 baseline doesn't reach 95% HOT 13
- Training time HOT 1
- Training scheme modifications for small GPUs HOT 4
- Could you please release the training logs of convmixer on imagenet and cifar?
- CIFAR-10 training settings
- something about loss
- weight location HOT 1
- What's new about this model? HOT 5
- 你好,convmixer能否直接提取图片的2D特征?
- Do you use this model on other downstream work? (like semantic segmentation ) HOT 3
- padding=same? HOT 1
- Request more experiment results to compare to other architecture. HOT 1
- Segmentation ConvMixer architecture ?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from convmixer.