Comments (4)
That is very weird. Could you share details about your "model" and your Opacus version information?
from opacus.
Thank you for your reply
The currently used version of opacus is 1.1.0
The model is defined as follows:
import torch
import numpy as np
class FeaturesLinear(torch.nn.Module):
def __init__(self, field_dims, output_dim=1):
super().__init__()
self.fc = torch.nn.Embedding(sum(field_dims), output_dim)
self.bias = torch.nn.Parameter(torch.zeros((output_dim, )))
if np.__version__ < "1.24":
self.offsets = np.array((0, *np.cumsum(field_dims)[:-1]), dtype=np.long)
else:
self.offsets = np.array((0, *np.cumsum(field_dims)[:-1]), dtype=np.int64)
def forward(self, x):
"""
:param x: Long tensor of size ``(batch_size, num_fields)``
"""
x = x + x.new_tensor(self.offsets).unsqueeze(0)
return torch.sum(self.fc(x), dim=1) + self.bias
class Model(torch.nn.Module):
"""
A pytorch implementation of Logistic Regression.
"""
def __init__(self, **kwargs):
super().__init__()
embedding_nums = 100000
if 'embedding_nums' in kwargs:
embedding_nums = int(kwargs['embedding_nums'])
input_dim = 10
if 'input_dim' in kwargs:
input_dim = int(kwargs['input_dim'])
self.linear = FeaturesLinear(field_dims=[embedding_nums for _ in range(input_dim)])
def forward(self, x):
"""
:param x: Long tensor of size ``(batch_size, num_fields)``
"""
return torch.sigmoid(self.linear(x).squeeze(1))
from opacus.
I see. The version is too old. Could you please update to the latest version?
Right now, by functorch, Opacus could support any type of model.
from opacus.
Thanks for your reply, I will try a newer version of pacus.
from opacus.
Related Issues (20)
- IndexError: pop from empty list HOT 1
- Implementing augmentation multiplicity using Functorch [Error: AttributeError: 'Tensor' object has no attribute '_forward_counter] HOT 1
- UnsupportedModuleError: [IllegalModuleConfigurationError('Model needs to be in training mode')] HOT 1
- LLM finetuning with Opacus HOT 2
- Add context manager to toggle on/off privacy in training loop
- Some issue with loading model with 'weight' as opposed to 'pretrained=True' HOT 1
- Error: Trying to add hooks twice to the same model HOT 2
- OverflowError: cannot convert float infinity to integer HOT 2
- ModuleValidator.fix() causes layer gradients to be None. HOT 1
- Integrating Opacus for a custom pytorch model raises errors but works fine on its own HOT 2
- `BatchSplittingSampler` return wrong length HOT 5
- Error occurred when executing GroundingDinoSAMSegment (segment anything): Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu! (when checking argument for argument mat2 in method wrapper_CUDA_mm)
- Why GDP and RDP give different result for the same config HOT 1
- Making a custom transformer architecture work with opacus HOT 3
- ValueError: Per sample gradient is not initialized. Not updated in backward pass? HOT 4
- Spectral normalization in Opacus HOT 4
- BatchMemoryManager for training private GANs
- Wrapper references can be easily replaced, consider using properties instead HOT 3
- Error in DPOptimizer: Inconsistency between batch_first argument of PrivacyEngine and DPMultiheadAttention HOT 2
- Grad Sample Module: Use full backward hook to save activations and backprop values. HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from opacus.