Comments (3)
My bad, I should have motivated this better (and also, perhaps FeatureDropout is not a very good name).
"FeatureDropout", also known as "Dropout2d/Dropout3d" in PyTorch (http://pytorch.org/docs/master/nn.html#torch.nn.Dropout2d) slightly modifies the Dropout algorithm so that only channels (the first two dimensions as in NCHW ordering; the important thing is that it matches the output of Conv) are randomly masked out; the actual pixels aren't randomly masked (since the correlations between adjacent pixels means that the masking is just decreasing the effective learning rate.)
It's a fairly important technique and shows up in some models that we are interested in exporting.
from onnx.
(and also, perhaps FeatureDropout is not a very good name)
Instead of coming up with names for additional dropout variants, consider adding a parameter to the existing Dropout that gives the list of axes to tie together (i.e., an empty list by default, [0] for batchwise dropout, [1] for dropping pixels/voxels in all channels, [2,3] for 2d spatial dropout, [2,3,4] for 3d spatial dropout, all assuming NC012... layout). This keeps the namespace simple and gives much greater flexibility in what can be represented.
from onnx.
Although dropout is nop in inference, but they are still in the model saved after training. Also, no one prevent anyone from loading the model in CNTK or PyTorch or Caffe2 and do fine tuning. When you load ONNX in CNTK, it just covert ONNX model to CNTK internal presentation, in theory you can continue training, replace some nodes...etc.
Regarding FeatureDropout, it isn't clear to me from the code you share what it does? Can you provide more detail?
from onnx.
Related Issues (20)
- [Feature request] checking an input rank is within a specific range HOT 6
- [Feature request] Shape Inference for Einsum instead of Rank Inference HOT 1
- Building failed with Vitis AI. HOT 1
- AttributeError while installing ONNX HOT 11
- Replace const string references with string_views HOT 1
- [ONNX] Onnx file loads very slowly with onnxruntime, which is exported (dynamic_shapes=True) with `torch.onnx_dynamo_export`. HOT 2
- Optimizing Node Ordering in ONNX Graphs: Ensuring Correct Sequence for Model Generation HOT 1
- When will outstanding security vulnerabilities be fixed? HOT 2
- ScatterND's doc gives wrong result for 3d indice case HOT 2
- Equivalent Implementations to ScatterElements operator HOT 3
- onnx.load() -> Can't Open this page HOT 1
- Square Activation is not supported by onnxruntime HOT 1
- Inference Layer by Layer or feature extraction HOT 2
- Support for collective operations / MPI primitives HOT 4
- When is the next release, 1.16.0? HOT 3
- RUNTIME_EXCEPTION : Non-zero status code returned while running Reshape node. HOT 6
- INT8 quantization model from torch to onnx slower than FP32 model on cpu HOT 1
- When exporting from torch, the result of the input varies depending on the dummy input. HOT 2
- [Feature request] Use protobuf's generator to generate pyi stubs HOT 1
- Does inline_local_functions preserve ir_version? HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from onnx.