Comments (4)
mask实现并不影响pruning和fine-tuning的过程
from network-slimming.
感谢你的回答,可能我说的不是很清楚,举个例子,在mask_impl的实现中,假如现在对resnet进行剪枝,先用main_mask进行bn层稀疏化的训练;之后利用prune_mask进行剪枝,这步中剪枝后的模型权重保存了resnet的cfg变量;第三步重新用main_mask对上一步剪枝后的权重进行finetune,由此有三个小问题:
(1)但是我看main_mask.py中并没有用到第二步的cfg变量来对剪枝后模型进行重构(load的时候没有使用cfg变量);
(2)第二步保存的权重中为0的部分并没有进行剪枝吧,还是保存了下来,这样如何在第三步中载入剪枝后的模型呢?
(3)剪枝后其实只是权重置0,最后存储下来的模型权重好像并没有太大变化吧?
期待你的回答,谢谢~
from network-slimming.
(1) 不需要cfg,因为mask的信息包含了cfg
(2) mask模型的意义就在于没有实质的剪枝发生,模型的大小仍然保持一样。只是inference的时候mask模型等价于prune后的模型
(3) 是的
from network-slimming.
这里mask版的实现作者提到能解决整个层被裁剪的问题。若模型中间某整个层被裁剪了,即整个层对应BN层的gamma都被置0,那么在retrain过程中,pruned model应该无法有效收敛呀,因为在被裁整层之前梯度都被截断了,都是0。这块不知道是不是我理解的问题?
from network-slimming.
Related Issues (20)
- 预测单张照片时,准确率只有0.0几 HOT 2
- Minor Bugs caused by old version
- Sparse Confusion HOT 1
- channel_selection layer intraining process HOT 2
- 剪枝后保存的权重文件和newmodel加载不上 HOT 3
- RuntimeError: Given groups=1, weight of size [15, 14, 1, 1], expected input[64, 16, 32, 32] to have 14 channels, but got 16 channels instead HOT 1
- If the remaining channel for a layer is zero, it reports zero division error HOT 2
- 原论文中的memory是指 HOT 6
- TypeError: item() takes no arguments (1 given) HOT 1
- mnn加载剪枝模型错误 HOT 1
- 经稀疏训练剪枝后模型变小,但是refine微调后模型又变大了
- 关于L1 regular HOT 1
- m.weight.grad.data.add_的问题
- 问题咨询:剪枝后通道数为0 HOT 7
- RuntimeError: CUDA error: device-side assert triggered HOT 1
- About other visions
- 题外话:模型压缩如何入门?对于自己的网络架构该如何着手去写剪枝代码?
- ResNet和DenseNet这种每一层的输入会作为后续多个层的输入,且其BN层是在卷积层之前,在这种情况下,稀疏化是在层的输入末端得到的,一个层选择性接受所有通道的子集去做下一步的卷积运算。
- 对HRNet剪枝,第一个bn的grad是None的问题
- Regarding the calculation of FLOPs after model compression HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from network-slimming.