Inquiry: Pruning process without spatial decomposition? about channel-pruning HOT 14 CLOSED

@OPPOA113 I'm working on it right now. I also want to modify the code to support other DNNs . I think what @yihui-he was talking about can be found in https://github.com/yihui-he/channel-pruning/blob/e8a5fe0b97eaf5de693cc93a9d1024fdf919dab8/lib/net.py#L1343 and https://github.com/yihui-he/channel-pruning/blob/e8a5fe0b97eaf5de693cc93a9d1024fdf919dab8/lib/net.py#L1376 . Changing those two "if-conditions" to "False" might be the solution. Ahh, and also https://github.com/yihui-he/channel-pruning/blob/e8a5fe0b97eaf5de693cc93a9d1024fdf919dab8/lib/net.py#L1373 should be deleted, since now we don't insert any layers into the model. Ill post if I get it working.

from channel-pruning.

@OPPOA113 Nevermind, still trying to figure it out

from channel-pruning.

@OPPOA113 still not working, I did this:
idxs, W2, B2 = self.dictionary_kernel(X_name, None, d_c, convnext, None)
P_shape = [W2.shape[0], W2.shape[1], 1, 1]
self.WPQ[(conv_P, 0)] = W2.reshape(P_shape)
But I get an error because W2 is too big to be reshaped. Im not sure what should I change.

from channel-pruning.

The code was rewrite once, I need to adapt it to current released code.
actually you can implement it yourself, just delete VH and ITQ in R3 function.
Duplicate of #16

from channel-pruning.

Thanks!!!

from channel-pruning.

@slothkong
Hi
Have you successfully changed the code for pruning only?And what should be modify.

from channel-pruning.

@OPPOA113 , seems like "channel pruning" depends on "channel decomposition" ㅜㅜ. There is a set of additional layer names conv_H, conv_V and conv_P https://github.com/yihui-he/channel-pruning/blob/e8a5fe0b97eaf5de693cc93a9d1024fdf919dab8/lib/net.py#L1334-L1336

We are not interested in conv_H or conv_V, only conv_P. However, conv_P is introduced in the new model self.WPQ when we execute "channel decomposition" ( calling the function ITQ_decompose() yields the following values: W1, W2, B, W12)

https://github.com/yihui-he/channel-pruning/blob/e8a5fe0b97eaf5de693cc93a9d1024fdf919dab8/lib/net.py#L1388-L1392

Finally, in "channel pruning", we use conv_P to set self.WPQ

https://github.com/yihui-he/channel-pruning/blob/e8a5fe0b97eaf5de693cc93a9d1024fdf919dab8/lib/net.py#L1414-L1419

TODO: we have find a way to define W2 without executing ITQ_decompose() .
@yihui-he, if you are reading this, please comment^^

--- EDIT: Or maybe we can simply add:
P_shape = [W2.shape[0], W2.shape[1], 1, 1]
self.WPQ[(conv_P, 0)] = W2.reshape(P_shape)
self.WPQ[(conv_P, 1)] = B2

under the code line : https://github.com/yihui-he/channel-pruning/blob/e8a5fe0b97eaf5de693cc93a9d1024fdf919dab8/lib/net.py#L1407

from channel-pruning.

@slothkong
thanks,how it was going on ,it work finally? actually, i do not understand the code completely.
if you have finish,please let me know. thank you !! i will follow you work!

from channel-pruning.

@slothkong Sorry to bother you, would you like to tell me whether you implement it ?

from channel-pruning.

@YaoZhongtian I was unable to debug it. Seems like I never fully understood how @yihui-he's code worked. So even thought I managed to prune the hold model and write the protobuf file, it's accuracy was awefully bad.

The author told he was going to release some code for this after he was done with preparations for some conference. But I guess this repo is dead now

from channel-pruning.

@slothkong Thank for your reply. I just started trying to solve it.

from channel-pruning.

@YaoZhongtian any solution for this?

from channel-pruning.

@Tveek Actually, I can't understand the code completely. Just waiting for the author to release the code of this part.

from channel-pruning.

Is there anyone know how the speed-up is determined? And what is the relationship between the speed-up ratio and the corresponding channel preserved ratio?Thank you!

from channel-pruning.