skku-eslab / cnn-on-flash Goto Github PK

Now the memory budget only includes submatrices allocated on memory.
For more general usage, it is necessary to include total memory usage of the process that executes gemm in the memory budget.

Currently, on-flash gemm only support ARM CPU.

GPU support is needed for usage in various environments and performance comparison between processors.

CNN-on-flash seems to support only GEMM operators.

However, real machine learning apps usually use layer-wise operators; such as convolution layer, fully-connected layer.

If you want to apply it to convolution layer, supporting for im2col is imperative.

We need to add partial im2col operation.

It is necessary to operate im2col on input tensor for convolution using gemm.
im2col should be done while a partial matrix is in memory and the other parts are in storage.

While building this project, warnings like following example occur many times.

/home/odroid/Desktop/test-cof/include/pointers/allocator.h: In instantiation of ‘void flash::unmap_file(flash::flash_ptr<X>) [with T = float]’:
/home/odroid/Desktop/test-cof/drivers/gemm.cpp:72:26:   required from here
/home/odroid/Desktop/test-cof/include/pointers/allocator.h:49:9: warning: unused variable ‘ret’ [-Wunused-variable]
     int ret = munmap(

When I look into the files that make warnings, 'unused variables' are actually used in GLOG sentences but not counted as used.
This bothers us when we modify and debug the codes. It should be solved.

Is it possible to apply CNN-on-flash for other operations like Winograd?

When I set PROGRAM_BUDGET less than the size of three submatrices, it never ends but continues the loop.

This is not proper behavior because failed execution should be informed explicitly and terminated.

So it is necessary to add the functionality to judge if the budget is not enough and if termination is needed.

Convolution implementation using gemm needs weight filters of a convolutional layer to be reshaped.
To reduce the maximum memory usage, reshape procedure should progress while partial matrices are loaded from storage and reshaped sequentially, not the whole matrix loaded at the same time.

So I propose to make partial reshape operation.

Readme is simply telling how to build and how to execute GEMM.

It would be good to add an explanation about methodology and example case evaluation results.

Hi,
I want to see memory usage when performing GEMM operation using CNN-on-flash.

How can I print out this?

I've checked your response to the last issue about evaluation.

You plotted memory usages and inference times of only on-flash GEMM, but not in-memory GEMM.

It will be meaningful to compare the execution time between in-memory and on-flash GEMM.