Comments (3)
Yeah, your explanation was spot on; I was indeed just counting incorrectly. Sorry for the slow reply!
from e2cnn.
Hi @drewm1980 ,
would it be possible to share the code you used to define your models? That would make it easier for me to understand the architecture.
Be careful that if you preserve the number of fields in each layer (e.g. you always write something like ft = FieldType(gs, [gs.regular_repr]*8)
regardless of which group is used in gs
), the cost of your conv layer will grow quadratically with the group size.
This is because the size of each feature will grow linearly with the group size, but the cost of convolution/linear layer grows quadratically with the feature sizes.
The fact you have ~8 times more params in the FlipRotOnR2(4) model, makes me think your features are actually ~8 times larger.
Is that correct?
In general, it's not usually necessary to scale the feature size linearly with the group size.
Sometimes, if your group is not too large, it is not even necessary to scale up the feature size at all, preserving the original computational cost.
To keep the number of channels more or less fixed, you can do something like this:
# "gs" is your gspace, e.g. FlipRotOnR2(4)
C = 64
c = int(C / gs.fibergroup.order())
ft = FieldType(gs, [gs.regular_repr]*c)
I would try to start with a model where you don't scale up the number of channels, and then gradually increase it if necessary to get better performance.
Let me know if this helps
Best,
Gabriele
from e2cnn.
Thanks for the response! I'll try to work up a minimized example I can share, but it might be week or two before I'm back on the relevant project. This may just be expected behavior; I'll try increasing the non-equivariant network's channels to match (x 8 everywhere), and re-compare the sizes. If they're comparable, then I just wasn't aware enough of the cost of scaling the equivariance group.
from e2cnn.
Related Issues (20)
- wrapping pytorch operations - grid_sample HOT 4
- Import Error with Torch 1.9.0+cu111 HOT 2
- equivariant Transformer HOT 5
- ZeroPad2D on GeometricTensor
- Cannot pass weights of R2Conv as a positional argument HOT 2
- Counting FLOPs for e2cnn HOT 1
- equivariance in C8 space HOT 1
- Module export HOT 3
- About the equivalence of wide_resnet HOT 5
- Need a size parameter for e2cnn.R2Upsampling Class HOT 1
- about attribute R2conv.filter HOT 2
- Learning of kernels HOT 2
- O(2) group, irreps, and PyTorch DDP. HOT 2
- checking equivariance for the angles that are not 90n HOT 2
- about to set special rotation equivariant HOT 2
- Cannot import name container_abcs in python 3.6 version (e2cnn_py36)
- shriking size of output image
- Use of np.float and np.int etc
- Difference between trivial output type and regular output type with group pooling HOT 1
- Export Linear HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from e2cnn.