Comments (2)
KaimingHe
commented on August 28, 2024
This code is a direct revision on the official PyTorch ImageNet training code and its multi-node version could be done following that. The speedup ratio would be pretty reasonable just like the official PyTorch ImageNet training code. In general, I suggest you follow the "ImageNet in 1 hour" paper for multi-node training: specifically the linear lr recipe and the related discussions on other relevant hyper-parameters. Depending on how many nodes you use (basically the batch size), there could be a slight accuracy drop, similar to what is observed also in supervised training.
from moco.
wenyuqing
commented on August 28, 2024
Hi, have you successfully implemented multi-node version of moco? I meet some problem when running with multi-node.
The buffer of queue seems not synchronized and the variable ptr is not 0 at the begining. I am confused about this issue.
from moco.
Related Issues (20)
- cannot reproduce the results of moco-v2 HOT 2
- Question about transfering to COCO with Mocov1 and Mocov2 checkpoint
- Issue about dequeue_and_enqueue HOT 3
- Question about the queue for key encoder HOT 2
- Why labels are all zeros, should first columns of labels be ones? HOT 4
- Issue with batch size HOT 1
- Low Accuracy
- One question about single GPU HOT 2
- How to load the Hyperparameters without command line code Argument Parser?
- About training HOT 2
- what information is leaked due to intra-batch communication? HOT 2
- What is the label format of the cifar-10 dataset? HOT 1
- Concerns about feature dimensionality in MoCo self-training
- Can you tell me dataset structure and how images are named in the dataset HOT 1
- why pretrain from encoder_q? HOT 1
- Question about queue dimension
- How is BN in key-encoder updated (in Moco v1)？ HOT 1
- Why is labels = zeros(N) set to zero? HOT 3
- The size of the dictionary HOT 2
- About License
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from moco.