dayu11 / differentially-private-deep-learning Goto Github PK
View Code? Open in Web Editor NEWThis repo implements several algorithms for learning with differential privacy.
This repo implements several algorithms for learning with differential privacy.
n = params[0].grad_batch.shape[0] AttributeError: 'Parameter' object has no attribute 'grad_batch'
Dear dayu11,
In your code, the parameter "warmup_epoch" is -1 by default, and it seems that you did not assign a new value to this parameter in your experiment.
However, if "warmup_epoch" is -1, it seems that at the train function:
Differentially-Private-Deep-Learning/vision/RGP/cifar_train.py
Lines 261 to 263 in 77096c8
This part of code will never be executed.
So, Is that correct or I am missing something?
Hi dayu11,
I'm appreciated that you open source the code of your brilliant work. I'm confused about the standard deviation of Gaussian noise in your implementation. In vanilla DP-SGD, the standard deviation equals to sigma, a.k.a. noise multiplier, times clip threshold.
Differentially-Private-Deep-Learning/vision/GEP/main.py
Lines 219 to 221 in 55eb9a1
I don't get the reason why you divide it by the batch size.
Is there some details I missed?
Hi dayu11,
I'm a beginner in differential privacy. I'm grateful that you open source this outstanding work, my question may be silly but I wonder if there is a way to use DP independently?
I don’t know if I’ve expressed clear enough, I mean if I add noise to dataset or output(like DP-logits) instead of gradients, how can I achieve it? What are the key points in the code that I need to understand and apply to it?
I would be grateful if you could take the free time to answer this dumb question. Thanks for your help!!
cifar_train.py --eps 8 --delta 1e-5 --rank 16 --lr 0.5 --clipping 1 --batchsize 1000 --n_epoch 400 --width 10
Using the above parameters to train the cifar10 dataset under RGP, the accuracy is just over 40%. How to deal with it?
Test environment: win10, cuda11.1
I run it directly in pycharm IDE with the following settings:
parser.add_argument('--dataset', default='cifar10', type=str, help='dataset name')
parser.add_argument('--arch', default='resnet28', type=str, help='model name') # resnet28
parser.add_argument('--resume', '-r', action='store_true', help='resume from checkpoint')
parser.add_argument('--sess', default='resnet', type=str, help='session name')
parser.add_argument('--weight_decay', default=1e-4, type=float, help='weight decay (default=1e-4)')
parser.add_argument('--batchsize', default=1000, type=int, help='batch size')
parser.add_argument('--n_epoch', default=400, type=int, help='total number of epochs')
parser.add_argument('--lr', default=0.5, type=float, help='base learning rate (default=0.4)')
parser.add_argument('--momentum', default=0.9, type=float, help='momentum coeeficient')
parser.add_argument('--eps', default=8, type=float, help='eps value')
parser.add_argument('--width', default=10, type=int, help='model width')
parser.add_argument('--delta', default=1e-5, type=float, help='delta value')
parser.add_argument('--rank', default=16, type=int, help='rank of reparameterization')
parser.add_argument('--clipping', default=1., type=float, help='clipping threshold')
parser.add_argument('--warmup_epoch', default=-1, type=int, help='num. of epochs for warmup')
Hi,
I'd like to reproduce the experiments in Table 4.
And I could run Method RGP and RGP (N.P.) with this repository, but where is the proper code/code version for Method Full (N.P.) and DP-SGD?
I guess they use the original fairseq codebase or MC-BERT with opacus or pyvacy or sth like that.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.