Some questions about training hyperparameters about ddad HOT 1 CLOSED

Training diffusion models can be resource-intensive. When dealing with large datasets, it's advisable to consider reducing the number of epochs. From our experiments, a practical guideline is to use around 500 epochs for every 1000 samples.

Alternatively, a more nuanced approach involves monitoring the reconstruction loss of the training dataset after each 500 (or preferably 250) epochs. If this loss converges to zero, it signals overfitting, increasing the risk of retrieval rather than effective reconstruction.

For optimal experimental configuration, especially when having a custom dataset, incorporating a validation set is highly recommended, if feasible. This allows for a more refined assessment of model performance and aids in making informed decisions about training parameters.

from ddad.