ebagdasa / pytorch-privacy Goto Github PK

I'm new to DP, it's hard for me to understand some privacy mechanisms and calculation of privacy costs.
I found that in the code, it seems that all batches in each epoch need to calculate gradients and meet DP.
for i, data in tqdm(enumerate(trainloader, 0), leave=True):...
For example, if there are total 600 examples and the batch size is set to 100, then there will be 6 batches in each epoch. In the code, all the 6 batches should be used for gradient calculation (with DP).
However, I found this to be inconsistent with the traditional DP-SGD process in Abadi et al.'s work "Deep learning with differential privacy", where a probability q is used to randomly sample a batch in each epoch, and then calculate privacy cost use Moments Accountant. That is to say, each epoch only uses one randomly sampled batch to calculate gradient, and the size of this batch is uncertain, and the expected value is qN.
So, I wonder to know that, in your code, using this way (multiple batches per epoch) to process each epoch is to meet the privacy cost calculation in TF?
Maybe I am not expressing clearly enough, thank you for your patience.
Besides, I will be very grateful if there is any reference or intuitive explanation.Thanks!:)

There is a typo in training code example:
python training.py --params utils/params.yaml
instead of :
python training.yaml --params utils/params.yaml