Comments (9)
Thanks a lot!
from convnext.
I was also wondering about this. It seems the 32x32 size of CIFAR-10 is incompatible with this model due to the down-sampling layers.
from convnext.
@Yuancheng-Xu It seems like it can. The downsampling layers should be set to a smaller kernel and stride size (2 and 2 respectively). Without this, the output of the downsampling layers is effectively the same size as the kernel.
In addition, you might want to choose a smaller kernel and padding size for the Block convolutional layers
Here's a notebook showing the training progress https://juliusruseckas.github.io/ml/convnext-cifar10.html
from convnext.
@Yuancheng-Xu I managed to get accuracy to 87% by making a few changes to the code in the link above. Basic changes are mentioned in this repository https://github.com/shamikbose/Fujitsu_Assessment
Main changes were as follows:
- The downsampling convolutional layers were modified (4x4 -> 2x2) for the smaller image size in the dataset
- This improved accuracy from 70% to 80%
- Keeping CIFAR-10 training recipes in mind, the architecture was modified to be a 3-block architecture instead of a 4-block one
- This improved accuracy from 80% to 85%
- Kernel size was changed (7 -> 3)
- This improved accuracy from 85% to 87%
from convnext.
Hey @shamikbose, I tried training the ImageNet100 dataset for custom input_size = 32, but the accuracy that I am getting is too low. What could I change in the architecture (I tried with making the kernel and stride small)? Any other approach that might help me to get good accuracy?
from convnext.
@iamsh4shank The parameters used for ImageNet100 are mentioned in the paper. You should be able to reproduce it using those values.
from convnext.
Actually ig it was for input_size 224 but on changing it to 32 I get accuracy really low
from convnext.
With image size 32, try the parameters mentioned here #134 (comment)
from convnext.
I did try changing the Conv layer (https://github.com/facebookresearch/ConvNeXt/blob/main/models/convnext.py#L28) with kernel size 3 and padding 1. Also, I changed the downsampling layer (https://github.com/facebookresearch/ConvNeXt/blob/main/models/convnext.py#L74) with kernel size 2 and stride 2. It did not change the accuracy much. I am getting test accuracy like 4-5 percent
from convnext.
Related Issues (20)
- head layer can't be replaced in convnex like other(classifier layer) pytorch model? HOT 1
- Ask for release model on COCO
- Infinite loss value when training under amp HOT 1
- Why LayerNorm before conv in downsampling layers ?
- Error in Convnext training with Swin transformer framework - The testing results of the whole dataset is empty.
- can convNext add deformable conv in the block
- LayerNorm, what is going on? HOT 1
- load pretrained model, there are errors. RuntimeError: Error(s) in loading state_dict for ConvNeXt: Missing key(s) in state_dict HOT 1
- Tensorrt Model?
- input dimensions
- Welcome update to OpenMMLab 2.0
- A warning appears in DDP using ConvNeXt HOT 1
- Missing code or wrong reference HOT 2
- Finetuning on Imagenet100 using ConvNeXt-T for ImageNet1k
- I can not use 'timm.create_model' to create my convnext HOT 1
- convnext_tiny Loss not change HOT 1
- Error:0.3.2timm.create_model did not have layer_scale_init_value
- How can I make predictions on my testing dataset for using command line?
- Resize Test Images for size less than 32
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from convnext.