Three questions about Utilization about stonne HOT 1 CLOSED

Hi, sorry for the late response

I'm not the designer of STONNE so I cannot answer you with all the detail I would like to, but I will try to clarify your questions as much as possible:

1. Memory: you are right, tensor data is directly fed into the STONNE simulator. In this version of the simulator, as far as I remember we only consider that tensor data is already prepared in the scratchpad when starting the simulation, so memory transfer times are not considered on it. Maybe you can find some extra detail about this in the STONNE paper: https://arxiv.org/abs/2006.07137. Consider also taking a look at the MAERI's paper (https://anands09.github.io/papers/maeri_asplos2018.pdf), which is the architecture that STONNE simulates.
   Well, and as you can see not considering that memory transfer times is a problem. For this reason, the authors of STONNE also developed SST-STONNE, an extended version of STONNE that can also simulate a more detailed memory hierarchy using the SST Simulator. It's currently available on this repo: https://github.com/stonne-simulator/sst-elements-with-stonne/tree/sparse_dataflows_project. However, there is not any documentation prepared yet for it ;-(. We are planning to document it this summer.

2. Tiling: you are right, the tiling configuration affects both latency and throughput, and it also depends on both the size of the accelerator and the dimensions of the computation tensors. The taxonomy used for it is described in the mRNA paper (https://ieeexplore.ieee.org/abstract/document/8695674), which is a tool for generating automatic mappings for the MAERI architecture based on the architecture parameters (number of multiplier switches, distribution network bandwidth, and reduction network bandwidth, essentially). Selecting a good tiling (or better called, mapping) is crucial to ensure you achieve the maximum throughput, but selecting the best mapping is a non-trivial task. For that reason, we directly integrated mRNA into STONNE code along with other extensions to it, so you can generate those tilings (mappings) automatically just by including -generate_tile=performance into the STONNE simulation parameters. In case you want to use it from the PyTorch Frontend, take as an example this configuration file: https://github.com/stonne-simulator/stonne/blob/master/minibenchmarks/dogsandcats_tile_stonnemapper.txt

3. Sparsity support: yes, on this version we only support sparsity through bitmap representation. If needed, we can try to also add support for CSR representation (this is already included on the SST-STONNE version, so I hope we could easily export to this version too)

If you have any extra question, do not hesitate on ask again :-)

from stonne.