Comments (3)
Hi qingyang, here's the clarification to some of your questions.
Does the python3 setup.py install of BytePS replace mxnet/3rdparty/ps-lite with byteps/3rdparty/ps-lite?
No.
If no, how does BytePS initialize it's own server and scheduler at the beginning of training, and at the same time, prevent the initialization of server and scheduler in import mxnet (they may occupy the DMLC_PS_ROOT_PORT)?
If you do not intend to use BytePS and MXNet KVStore at the same time, you don't need to worry about this. BytePS server is inited through "import byteps.server". You can check the code of byteps/server/__init__.py
.
BTW: we do not maintain mxnet > 1.5.0
from byteps.
Ok, I get it. I fond "import byteps.server" in launch.py... Then I should add it to my training python file.
But I still need to manually stop MXNet's initialization of server and scheduler in "import mxnet".
Maybe MXNet 1.5.x is different from 1.5.0 . LOl.
from byteps.
Oh, I get it. In server and scheduler side. I only need to run "import byteps.server" and nothing else.
Then I don't have all those problems mentioned before.
Thanks again.
from byteps.
Related Issues (20)
- How to use gradient accumulate in BytePS torch DDP? HOT 5
- The byteps in K8S Pod doesn't have DMLC_WORKER_ID configured.
- Stuck in the bps.init(). HOT 7
- Is it right to do allreduce immediately for non-zero ranks in bytescheduler? HOT 2
- 啥时候支持sparse模型?
- 有计划支持纯cpu吗?我们worker也用cpu机器的 HOT 2
- benchmark with cross barrier error
- Successfully installed BytePS but cannot import byteps.torch or byteps.tensorflow HOT 2
- Running multiple workers on a single GPU machine
- Release BytePS docker image support for TF2
- 安装报错 HOT 1
- support for fault tolerance and straggler mitigation
- broadcast and is_initialized api are not supported with pytorch.
- Supported environment
- 安装问题
- Mistakes of Workload calculation HOT 5
- How does the tensorflow scheduler plugin used in the tf_benchmark_cnn.py HOT 1
- segmentation fault while launching the worker HOT 1
- Is there any benchmark comparison with Megatron-LM ?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from byteps.