Comments (8)
A note to future readers. Because we need to explicitly specify the size of the input we must use the "input_shape" parameter, ie. if before we used
model = Sequential()
model.add(Dense(64, input_dim=20, init='uniform'))
Now we would use
model = Sequential()
model.add(Dropout(0.5, input_shape=(20,)))
model.add(Dense(64, init='uniform'))
(This took me a while to figure out, since I haven't tried anything yet like convolutions which use the "input_shape" parameter, and since I hadn't read about the Abstract base layer class in the documentation, which Dropout inherits from.)
from keras.
Dropout on the input layer is actually pretty common, and was used in the original dropout paper IIRC. It would be nice if the following syntax worked (which it currently does not):
model = Sequential()
model.add(Dropout(0.1))
model.add(Dense(784, 20))
from keras.
31ea018 both allows to start models with dimension-agnostic layers like Activation and Dropout, and allows for custom output target shapes (must be specified by the user).
from keras.
Hey,
Yes you can. The layer takes in a 4D tensor and a probability as input.
Not sure why you would want to do that, but here are is another simple solution:
Do it outside keras. In Python.
from keras.
In the documentation/docstring of keras.layers.core.Activation
it is specified that
# Input shape
Arbitrary. Use the keyword argument input_shape
(tuple of integers, does not include the samples axis)
when using this layer as the first layer in a model.
Maybe add this to keras.layers.core.Dropout
to clarify that this applies here to. @devinplatt's answer should reach the docs..
from keras.
That would require to know what tensor type is expected as input (tensor2, tensor3, tensor4...). We need this info at compile time, so before any input has been seen by the network. So it would have to be specified by the user, it can't be automatically inferred.
There would be two potential ways to do that. Either have the user pass the tensor type as an argument to Dropout (only used/needed when Dropout is the first layer in a network), or introduce an "input" layer that takes a similar argument, and that can be optionally used as first layer in a network (required when the first "real" layer is one that does not have a fixed shape, like activation, dropout, etc).
from keras.
I like the input layer approach, especially if we make it optional (i.e., use it only when it's needed by the first "real" layer). Otherwise, all general "adimensional" layers will need to have their dimensions specified, which does not make much sense since it's not really a layer property, but a side effect from the input size.
from keras.
Here's a different idea that would be even simpler: at compile time, if the first layer is "adimensional", go through the model (list or tree depending on the model type) until you find the first dimensional parent, and set an input attribute on the first layout based on the tensor type of the first dimensional parent.
This would require no change in interface. It would fail in case there is no dimensional parent in the model, in which case the user could still manually set an input attribute on the first layer...
I'll have a go at adding this later today.
from keras.
Related Issues (20)
- Can we use Clustering at the bottom of 3D unet in image segmentation any one plz show implementation
- Loading saved frozen model fails HOT 1
- Saving an `Optimizer` initially instantiated with a `float` learning_rate and later replaced with a `LearningRateSchedule` includes an extra Variable. HOT 1
- Enhance Error Messages for Tuple Input Validations for `IRFFT`, `FFT`, `FFT2`, `RFFT`, `ISTFT` in `keras/ops/math.py`
- Keras 3.2.1 breaks LoRA with ModelParallel HOT 6
- Enormous memory usage after batched forward passes with TensorFlow 2.16.1 (CPU) HOT 7
- model.keras deserialization outputs wrong results HOT 1
- image_dataset_from_directory: Shuffle buffer size arbitrarily set to 1024 when batch_size is None HOT 1
- TrackedList type error for appending NamedTuple HOT 4
- Error when using LearningRateSchedule in distributed context HOT 2
- Newly introduced `weights` in `Embedding` layer breaks the downstream layers such as `ReversibleEmbedding` in KerasNLP HOT 1
- `torch.scatter` equivalent op in Keras HOT 2
- Input is Empty HOT 2
- [Torch backend] GPU utilization and slow running issue when using MeanIoU or its child classes
- 'str' object has no attribute 'shape' HOT 1
- keras/preprocessing/text module is missing in Keras 3 HOT 1
- cannot load fine-tuned model HOT 4
- πΊοΈ Keras Development Roadmap
- kl divergence outputs confusing result when the label contains negative HOT 3
- IntegerLookup with XLA Compilation Fails to Enable JIT in TensorFlow 2.16.1 and Keras 3.2.1 HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. πππ
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google β€οΈ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from keras.