Comments (3)
Seems like you are trying to profile your custom implementation of the ResNet model?
If this is the case, you want to patch the flatten
method, similar to this: https://github.com/msr-fiddle/pipedream/blob/master/profiler/torchmodules/torchgraph/graph_creator.py#L157. This is to ensure that we can construct the computation DAG for the given model. At a high level, this method needs to unwrap the TensorWrapper
object, call torch.flatten
, then rewrap the result as a TensorWrapper
object.
from pipedream.
I use my custom implementation of ResNet because of another error:
Traceback (most recent call last):
File "main.py", line 596, in <module>
main()
File "main.py", line 287, in main
verbose=args.verbose, device="cuda")
File "../torchmodules/torchsummary/torchsummary.py", line 77, in summary
model(*model_input)
File "/root/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 574, in __call__
result = self.forward(*input, **kwargs)
File "/root/anaconda3/lib/python3.7/site-packages/torchvision/models/resnet.py", line 149, in forward
x = self.avgpool(x)
File "/root/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 574, in __call__
result = self.forward(*input, **kwargs)
File "/root/anaconda3/lib/python3.7/site-packages/torch/nn/modules/pooling.py", line 554, in forward
self.padding, self.ceil_mode, self.count_include_pad, self.divisor_override)
RuntimeError: Given input size: (2048x1x1). Calculated output size: (2048x-5x-5). Output size is too small
It is said that this might be caught by wrong parameter of avgpool
layer in torchvision.
Thanks for your suggestion! This worked for my situation. I will close this issue.
from pipedream.
Yup, that's correct -- you need to use a version of the VGG model that wraps tensors with TensorWrappers
from pipedream.
Related Issues (20)
- Handling uneven number of batches per replicated instance of a layer
- GPU Peer2Peer communication via --num_ranks_in_server argument HOT 1
- Resource temporarily unavailable
- To run PipeDream_2BW branch without --recompute_step
- The BLEU score of translation model seems abnormal. The model doesn't seem to train effectively.
- GPT2 355m model convergence with 2BW training
- Is there AllReduce in data parallelism? HOT 6
- How is the Double-Buffered Weight Mechanism implemented?
- Supporting T5
- The arguments of self.start_helper_thread() should be more flexible instead of fixed as int64.
- Question about time complexity of PipeDream-2BW's planner algorithm
- Question about PipeDream's optimizer
- AttributeError: module 'models.resnet50.resnet50' has no attribute 'model' HOT 1
- Is there any 2bw code that will run on the native GPU HOT 1
- AttributeError: module 'torch.distributed' has no attribute 'P2POp' HOT 1
- Running in docker will give you an error that you can't find a physical address HOT 1
- what is the role of pre_hook_pytorch_latest.patch? HOT 1
- When I was testing the pipedream code with version-updated torch, I encountered the following error (1.1.0 -> 1.11.0): HOT 2
- optimizer got an empty parameter list when rank=1 HOT 1
- same train_loader but got different loader size HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pipedream.