This project was done as part of the ECE751: Distributed Computing course at the University of Waterloo. By Vineel Nagisetty; Husayn Kara;
- Introduction
- Usage
- More Info
Our project implements Downpour Stochastic Gradient Descent (Downpour-SGD), an asynchronous distributed deep neural network training algorithm using Thrift IDL (for Python) and Pytorch. We also implement several novel variations to parts of the Downpour-SGD - namely in the data splitting as well as parameter aggregation steps.
- The code for the front-end, back-end and client servers is found in the
src/frontEnd/serverFE.py
,src/backEnd/serverBE.py
, andsrc/client/client.py
files respectively - The code for the fron-tend and back-end service handlers (that implement a majority of the Downpour-SGD algorithm) is found in the
src/frontEnd/serviceFE.py
andsrc/backEnd/serviceBE.py
files respectively - The Deep Neural Network model architecture code is found in the
src/utils/model.py
file - The code for data splitting and parameter aggregation methods is found in the
src/utils/splitData.py
andsrc/utils/aggregateFeedback.py
- The code for plotting the results is found in the
src/utils/plotter.py
- To reproduce the results shown in the report, please install Thrift and all modules from the
requirements.txt
file and run the front-end, back-end and client servers (in that order)
In order to get a better understanding of our implementation and results, our report.