Comments (5)
There seem to be two negative effects of this:
- A repeated Dataset (
dataset.repeat(10)
for example) cannot stop training viaOutOfRangeError
orStopIteration
, we have to setsteps
ormax_steps
, which is inconsistent with canned Estimators. - If a user doesn't shuffle the dataset, AdaNet may repeatedly use the first
max_iteration_steps * batch_size
samples each time, thus fitting to a subset of the training data.
Am I right? @cweill
from adanet.
@shendiaomo: You are correct on both counts. For this reason, we request that the user configures the max_iteration_steps
to be the number of repetitions desired, which unfortunately requires the user to do some extra math (max_iteration_steps = num_examples / batch_size * num_epochs_per_iteration).
Assuming each adanet iteration trains over several epochs, 2. should be less of an issue in practice if your base learners are randomly initialized. They will tend to learn different biases, and form a strong ensemble regardless.
from adanet.
@shendiaomo: You are correct on both counts. For this reason, we request that the user configures the
max_iteration_steps
to be the number of repetitions desired, which unfortunately requires the user to do some extra math (max_iteration_steps = num_examples / batch_size * num_epochs_per_iteration).Assuming each adanet iteration trains over several epochs, 2. should be less of an issue in practice if your base learners are randomly initialized. They will tend to learn different biases, and form a strong ensemble regardless.
Great! Thanks for the explanation. However, that's sort of not handy to do the math, imagine someone wants to replace the DNNClassifier in her application into adanet.Estimator, there may be lots of work. Will you have a plan to improve this? Or will the Keras version avoid the same situation?
from adanet.
@shendiaomo: You are correct on both counts. For this reason, we request that the user configures the
max_iteration_steps
to be the number of repetitions desired, which unfortunately requires the user to do some extra math (max_iteration_steps = num_examples / batch_size * num_epochs_per_iteration).Assuming each adanet iteration trains over several epochs, 2. should be less of an issue in practice if your base learners are randomly initialized. They will tend to learn different biases, and form a strong ensemble regardless.
From the tutorials:
max_iteration_steps=TRAIN_STEPS // ADANET_ITERATIONS,
If I want to train with 100 epochs over one Adanet iteration, meaning num_examples/batch_size
steps per epoch, should I set max_iteration_steps
to that value?
I have a sample size of 5265 and batch sizes of 50, so I have about 105 update steps per epoch. Should my max_iteration_steps
be 10500?
from adanet.
Pinging
from adanet.
Related Issues (20)
- relation to AutoML tables in GCP HOT 2
- Question regarding architecture HOT 3
- adanet.ensemble.Ensembler not used in Tutorials? HOT 5
- Adding different loss to tf.estimator.Head HOT 2
- Correct place to add custom metric_fn? HOT 3
- Early stopping 'best-practice' using Adanet
- Interpretation of serialized "architecture_summary" HOT 2
- simple_dnn should not work !
- [Bug] AutoEnsembleEstimator() cannot be instatiated if tf< 2.x HOT 1
- I am having errors and do not know what to do... HOT 1
- Evaluation issue using TPUEstimator HOT 2
- RuntimeError with adanet_tpu tutorial using TF2.2 and Adanet 0.9.0
- AttributeError: module 'adanet' has no attribute 'AutoEnsembleEstimator'
- Is there any code to compute FLOPs and parameter size of adanet?
- Error on running tutorial notebook in Google colab
- Using AdaNet ModelFlow with pre-trained models HOT 1
- Got `AttributeError: 'NoneType' object has no attribute 'logits'` error while applying tutorial
- Is this repository deprecated? HOT 1
- Design suggestions of _IterationBuilder._best_loss from adanet.core.iteration
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from adanet.