minghz / meng_proj Goto Github PK

Every operation needs a gradient by default, but in my opinion, gradients shouldn't need to be implemented for the fix point rounding operation.

There are no derivatives for a rounding operation, and I'm treating the process as a constant, which means the fix point rounding has no effect in back propagation. It only affects the cost and final results in inference.

Currently, I only round the tensors in-between layers.

I must round the weight and bias tensors as well.

They should be unique to the major layer groups, as I've defined.

Currently test data is generated in cifar10_eval.py, which can be run in parallel to cifar10_train.py.

Although this setup simplifies testing and training by segregating the two, I feel it would make everything simpler by just having test and train data in one script. So I'll try to merge the functionality of _eval into _train