stonne-simulator / stonne Goto Github PK

Hi Francisco,

I looked through the accelerator code and try to understand how the interaction between the STONNE engine and DRAM is modeled.
Can you briefly explain how DRAM bandwidth/latency is modeled?
From the paper, you mentioned Gbuffer has double buffering and DRAM is modeled with DRAMsim.
Can you also point me to the corresponding codes?

Thanks.
Xin

Inside STONNEModel.cpp, from line numbers 542 to 547, to print individual values, the current object is not referenced. Is that by design? The benchmarked time values are stored in the variables associated to that object before. See code snippet below

std::cout << "Time mem(ms): " << time_mem << std::endl;

As opposed to

std::cout << "Time mem(ms): " << this->time_mem << std::endl;

I tried to run the BERT benchmark and unfortunately the required files were deleted from the repository. Was there some bug, or are you planning on re-uploading them soon?

Thank you for always answering my questions kindly.

I recently use the stonne simulator again.

In my case, "torch.add/torch.sub / torch.Basic matrix arithmetic functions such as "matmul" are used a lot.

"Matmul" confirmed that there was a function simulated by the STONNE engine.

However, there seems to be no add/sub. I wonder if I understand well.

In addition, I wonder where I can find the list of functions simulated in STONNE engine.

Thank you for releasing useful software.

I hope you are always happy.

Hi Francisco Muñoz-Martínez,
Can we train an algorithm on STONNE? If not, can you guide me through the ways on (any documentation) to change the simulator to accelerate training?

Compiling objects...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
[1/1] c++ -MMD -MF /Users/hao.chen/work/ai/stonne/pytorch-frontend/stonne_connection/build/temp.macosx-10.9-x86_64-3.8/torch_stonne.o.d -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/hao.chen/opt/anaconda3/envs/stonne/include -arch x86_64 -I/Users/hao.chen/opt/anaconda3/envs/stonne/include -arch x86_64 -I/Users/hao.chen/work/ai/stonne/pytorch-frontend/stonne_connection/../../stonne/include/ -I/Users/hao.chen/work/ai/stonne/pytorch-frontend/stonne_connection/../../stonne/external/ -I/Users/hao.chen/opt/anaconda3/envs/stonne/lib/python3.8/site-packages/torch/include -I/Users/hao.chen/opt/anaconda3/envs/stonne/lib/python3.8/site-packages/torch/include/torch/csrc/api/include -I/Users/hao.chen/opt/anaconda3/envs/stonne/lib/python3.8/site-packages/torch/include/TH -I/Users/hao.chen/opt/anaconda3/envs/stonne/lib/python3.8/site-packages/torch/include/THC -I/Users/hao.chen/opt/anaconda3/envs/stonne/include/python3.8 -c -c /Users/hao.chen/work/ai/stonne/pytorch-frontend/stonne_connection/torch_stonne.cpp -o /Users/hao.chen/work/ai/stonne/pytorch-frontend/stonne_connection/build/temp.macosx-10.9-x86_64-3.8/torch_stonne.o -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_clang"' '-DPYBIND11_STDLIB="_libcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1002"' -DTORCH_EXTENSION_NAME=torch_stonne -D_GLIBCXX_USE_CXX11_ABI=0 -std=gnu++14
FAILED: /Users/hao.chen/work/ai/stonne/pytorch-frontend/stonne_connection/build/temp.macosx-10.9-x86_64-3.8/torch_stonne.o

and I get warning like following
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Your compiler (g++) is not compatible with the compiler Pytorch was
built with for this platform, which is clang++ on darwin. Please
use clang++ to to compile your extension. Alternatively, you may
compile PyTorch from source using g++, and then you can also use
g++ to compile your extension.

See https://github.com/pytorch/pytorch/blob/master/CONTRIBUTING.md for help
with compiling PyTorch from source.
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

so I change the command to CXX=clang++ python setup.py install but still failed.

I have a question about the simulated_matmul() function.

The function seems to be porting the matmul() function to STONNE in pythorch.

Do torch.matmul() and torch_stonne.simulated_matmul() work exactly the same?

According to the notes, it should work the same way.

However, it does not seem to perform exactly the same motion in the following situations.

import torch
import torch_stonne

a = torch.tensor([[3],[0],[1],[2]])  # 1x4 matrix
b = torch.tensor([[0,2,2,3]])  # 4x2 matrix

r=torch_stonne.simulated_matmul("", a,  b, "../../simulation_files/maeri_128mses_128_bw.cfg", "test_tile.txt", 0)
print(r)
print(torch.matmul(a,b))

I want you to check it out

Hi Francisco Muñoz-Martínez,

I have read your awesome paper and now I plan to simulate SIGMA on STONNE. After reading your codes, I found that in your SIGMA configuration file:

[MSNetwork]
ms_size=128
[ReduceNetwork]
type="ASNETWORK"
[SDMemory]
dn_bw=128
rn_bw=128
controller_type="SIGMA_SPARSE_GEMM"

According to the codes, I think it used the default trees-like DSNetworkTop Distribution Network and Augmented Reduction Tree(ART) as Reduction Network. But in the SIGMA paper, they used Benes Topology Distribution Network and Forwarding Adder Network(FAN) Reduction Network.

I wonder if the two building blocks of SIGMA are implemented in the current repo and whether the current configuration is capable to simulate the correct behaviour(timing, energy consumed and the on-chip area) of SIGMA?

Thanks a lot.

Hi all,

I have recently started using STONNE for my research. It is an awesome tool for modeling deep neural network accelerators. I am grateful to the authors for making it open-source. However, I ran into several issues while installing STONNE. I want to share the sequences of steps and commands that lead to a successful installation for me. The below steps have worked well for my lab mates too.

Below are the steps:

1. Use VM or WSL with Ubunto 18.04
2. Install Anaconda
3. Setup a virtual environment using conda
   conda create --name stonne python=3.8
4. Activate stonne environment
   conda activate stonne
5. Clone Stonne
   git clone https://github.com/stonne-simulator/stonne.git
6. Build STONNE user interface

cd stonne
make all

The main issues with installation occur during the Pytorch-FrontEnd Installation of STONNE
7) Before beginning frontend installation, install numpy in the stonne env
conda install -c anaconda numpy
8) If you don't have CUDA access e.g no graphic card, you need to export these environment variables before frontend installation

export MAX_JOBS=1
export NO_CUDA=YES
export NO_CUDA=1

9. Pytorch-frontend installation, it is important to set the CMAKE_PREFIX_PATH

cd pytorch-frontend/
export CMAKE_PREFIX_PATH=${CONDA_PREFIX:-"$(dirname $(which conda))/../"}
python setup.py install

10. You will encounter some module missing errors. You can use conda to install modules one after the other module depending on the error or you install all of them at once using requirement.txt
    The above step should build STONNE torch, the installation takes somewhere between 15-20mins depending on your machine.

11. Once the installation is complete, then you need to connect the torch with stonne.

cd stonne_connection/
python setup.py install

12. You need torchvision for accessing various CNN models and datasets. The authors recommend building the vision from the source, but it throws several errors. The simple workaround that works without messing up the installed pytorch-stonne with below command:
    pip install torchvision --no-deps
13. You can check the installation by running test-2.py in folder stonne-connection

Hope this helps in installing STONNE smoothly.

hello dear,
Is it possible to install STONNE on ubuntu18.04? which OS is recommended?
when I wanted to install vision this error was occured( i have attached it).
How I can solve this problem,please help

[](url
error.docx
)

Hello,
I'm trying to execute the MADDPG rl algorithm with stonne. But, I receive the following error. It says the tile cannot be parsed. But, MADDPG uses MLP neural network with nn.linear and I just did nn.SimulatedLinear.So, can you let me know where the problem might be?
python main.py simple_spread simple_spread --n_episodes 60000 --discrete_action
Episodes 1-2 of 60000
Traceback (most recent call last):
File "main.py", line 251, in
run(config)
File "main.py", line 125, in run
torch_agent_actions, step_1_cnt, step_2_cnt = maddpg.step(torch_obs, step_1_cnt, step_2_cnt, explore=True)
File "/home/kailash/Desktop/maddpg-pytorch-copy/algorithms/maddpg.py", line 92, in step
action, counter_list = a.step(obs, counter_list, explore=explore)
File "/home/kailash/Desktop/maddpg-pytorch-copy/utils/agents.py", line 75, in step
action = self.policy(obs)
File "/home/kailash/anaconda3/envs/maddpg-pytorch/lib/python3.6/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/kailash/Desktop/maddpg-pytorch-copy/utils/networks.py", line 47, in forward
h1 = self.nonlin(self.fc1(self.in_fn(X)))
File "/home/kailash/anaconda3/envs/maddpg-pytorch/lib/python3.6/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/kailash/anaconda3/envs/maddpg-pytorch/lib/python3.6/site-packages/torch/nn/modules/simulatedLinear.py", line 76, in forward
output = torch_stonne.simulated_linear_forward(self.class.name, input, self.weight, self.path_to_arch_file, self.path_to_tile, self.sparsity_ratio, True) # The last true is to transpose the matrices
RuntimeError: dogsandcats_tile_fc.txt could not be opened for parsing

Hello, I'm trying to build stonne-simulator on my conda environment(env name: Test) with python 3.7.
Here is what I stepped before.

1. clone the repository and create environment(conda create -n python=3.7
2. change directory to pytorch-frontend and run setup.py (python setup.py install)
3. change directory to pytorch-frontend/stonne_connection and run setup.py (python setup.py install)
   -> in step 3, it occurred following statemetnt.
   Traceback (most recent call last):
   File "setup.py", line 2, in
   from torch.utils import cpp_extension
   File "/home/inje/anaconda3/envs/Test/lib/python3.7/site-packages/torch/init.py", line 189, in
   from torch._C import *
   ImportError: /home/inje/anaconda3/envs/Test/lib/python3.7/site-packages/torch/lib/libtorch_python.so: undefined symbol: _ZN5torch3jit7testing9FileCheck10check_nextERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE

The error also occurred on prebuilted torch environment(python3.7+ torch1.7+cuda102).
Is there any problem that I miss it? I'm hopefully wait for your comments!
Thanks for reading

Hi, Thanks for this great work,
I encountered the errors when I tried to build pytorch-frontend.

I knew there must be some path error I made,
But I can't find out. Could you please help me resolve this?

So far i understood that the stonne simulator works for MAERI and SIGMA accelerator , so can I change the stonne for our now accelerator architecture to check,

Hi,
Sorry for the bothering,
I got three questions about memory hierarchy assumption, Tiling and Sparsity supports,

1. Memory: if we are calling a SimulatedConv2d or simply run ./stonne with arguments,
   my understanding is we are feeding the tensor in PyTorch to an emulator, if that is correct, what kind of memory hierarchy should we assume? Did we assume that everything is prepared in the scratchpad, or we can assume that the bandwidth is sufficient to feed all the computation needs?
2. Tiling: About tiling, I saw that the stonne has connections for users to customize tiling, I wondered if there is any philosophy behind that tiling so we can have a tradeoff between latency and throughput, or the tiling totally depends on the size of accelerator and the size of computation tensors
3. Sparsity Supports: I saw that in Stonne, we are directly connecting with tensor in Pytorch, in that case, all the sparsity are represented as bit-map, (am I right?), so my question is in that case, did Stonne supports other sparsity data structure for now?

I believe the scripts required to run the benchmarks were never committed.

Hi, I plan on implementing part of the STONNE framework but was wondering how feasible it would be to implement an emerging non-volatile memory technology such as STT-RAM. This would replace the SDMemory module and we would hope to track the behavior/performance of the memory technology on the python frontend module. Any advice as to how one could patch in a similar piece of software would be great.

Thank you, Michael

Helllo,

I tried to download the weights for the mobilenet from benchmarks > object_detection > ssd-mobilenets using the download_weights script, and I got an error message. I think that the file with the weights was deleted from a contributor's google drive.

Hello,

We are currently testing STONNE to simulate some neural network architectures. We have a question, in the README.md file you mentioned a parameter call stats_path inside the SimulatedConv2d operation (i.e. I guess it is in the SimulatedLinear as well). However we could not find any parameter with this name in SimulatedConv.py or in simulatedLinear.py. Could you please update if there are plans to implement this functionality? Your help is much appreciated.

Regards,

Isai Roman.

Hello, I have 2 questions:

1. What is the version of PyTorch frontend that you have included in the repository? [v1.7.0 or v.17.1]
2. Could you provide a file list with the customizations you made in PyTorch? I ran diff using both PyTorch v1.7.0 and v1.7.1 but I couldn't get the result I was looking for. It seems that almost all files were changed.

Hello.
I'm Jongsang Yoo , who is studying HW accelerator.
First of all, thank you for providing a good program as an open source. I think it will be very helpful for studying.

There is a problem with the PyTorch Frontend installation.
I have excutered all the commands in the readme file you provided. ReadMe

(stonne) jognsang@DESKTOP-TV817D3:~/lab/STONNE/stonne/pytorch-frontend$ python setup.py install
Building wheel torch-1.7.0a0+633fa53
-- Building version 1.7.0a0+633fa53
cmake -DBUILD_PYTHON=True -DBUILD_TEST=True -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/home/jognsang/lab/STONNE/stonne/pytorch-frontend/torch -DCMAKE_PREFIX_PATH=/home/jognsang/anaconda3/envs/stonne -DNUMPY_INCLUDE_DIR=/home/jognsang/anaconda3/envs/stonne/lib/python3.8/site-packages/numpy/core/include -DPYTHON_EXECUTABLE=/home/jognsang/anaconda3/envs/stonne/bin/python -DPYTHON_INCLUDE_DIR=/home/jognsang/anaconda3/envs/stonne/include/python3.8 -DPYTHON_LIBRARY=/home/jognsang/anaconda3/envs/stonne/lib/libpython3.8.so.1.0 -DTORCH_BUILD_VERSION=1.7.0a0+633fa53 -DUSE_NUMPY=True /home/jognsang/lab/STONNE/stonne/pytorch-frontend
Traceback (most recent call last):
  File "setup.py", line 748, in <module>
    build_deps()
  File "setup.py", line 327, in build_deps
    build_caffe2(version=version,
  File "/home/jognsang/lab/STONNE/stonne/pytorch-frontend/tools/build_pytorch_libs.py", line 54, in build_caffe2
    cmake.generate(version,
  File "/home/jognsang/lab/STONNE/stonne/pytorch-frontend/tools/setup_helpers/cmake.py", line 329, in generate
    self.run(args, env=my_env)
  File "/home/jognsang/lab/STONNE/stonne/pytorch-frontend/tools/setup_helpers/cmake.py", line 141, in run
    check_call(command, cwd=self.build_dir, env=env)
  File "/home/jognsang/anaconda3/envs/stonne/lib/python3.8/subprocess.py", line 359, in check_call
    retcode = call(*popenargs, **kwargs)
  File "/home/jognsang/anaconda3/envs/stonne/lib/python3.8/subprocess.py", line 340, in call
    with Popen(*popenargs, **kwargs) as p:
  File "/home/jognsang/anaconda3/envs/stonne/lib/python3.8/subprocess.py", line 858, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/home/jognsang/anaconda3/envs/stonne/lib/python3.8/subprocess.py", line 1720, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'build'

However, the following error occurs.
How can I solve this problem?
Please give me some advice.

My environment is Window 10 + WSL 2 (Ubuntu 22.04).

ps. I confirmed that STONNE works with the STONNE User Interface.

Can we use Stonne without GPU support?

Hi Francisco,

I hope you are doing well.

Sorry to bother you again, but I found the data in the output file doesn't have the unit. Do you mind telling me the units of these values, such as energy, area in the output file?

Besides, I cannot find the whole execution time for the whole operation(It just tells me how many cycles when "running a convolution layer"), nor the frequency. If there is a default frequency, could you please tell me?

Can you tell me the data size it reads from the GlobalBuffer. Should it be 8bits? or 32bits? or 64bits?

BEST
Yile

Hello Adrian,

Hey I've been having serious memory issues with big simulation runs (i.e. for instance take MobileNetV2, TPU 256x256 and corresponding architecture attached). The process was killed by the OOM killer in our servers in the lab because of memory starvation (Please check the picture; the memory needs are in the order of TB). So I start digging in and I partially solved the biggest memory issues changing the abstract classes of the hardware components. Since they do not have any virtual destructor, the destructor of the derived classes were not called (reference about this topic, https://www.quantstart.com/articles/C-Virtual-Destructors-How-to-Avoid-Memory-Leaks/). Could you please help us fix this in your master branch? If you need to check my commit, please let me know.

Regards,

Isai.

tpu_256_256_pes.cfg.zip