why my metal shader is much slow than MPSCNN about forge HOT 5 CLOSED

The reason why MPSCNN is faster than your convolution kernel is that Apple has a team of very smart people who spent all their time writing and optimizing such kernels. :-)

Note that you don't need to do 4 texture reads from the input texture in your loop, only one. See my (also slow) version of this kernel here (it's called conv3x3): https://github.com/hollance/Forge/blob/master/Forge/Forge/Shaders.metal

I know the MPSCNN kernels also don't use textures for their weights and biases but MTLBuffers, although that in itself probably wouldn't make a huge speed difference.

The biggest reason for the speed difference is most likely that MPSCNN uses a faster algorithm. There are many ways you can compute convolution (im2col, FFT, Winograd, etc). Apple has the resources to try all of them. And they also have inside knowledge of how the GPU works, something we can only guess at.

I would like to add a very fast conv kernel to Forge at some point, just to show how it can be done, but my time is limited...

from forge.

other reason is that MPSCNN is using float16

from forge.

i am very expecting for your fast conv kernel :-)

from forge.

in objc, there is no datatype like float16 , but datatype "half" is supported in metal kernel , how can i use float16 in my code ?

from forge.

i ask the question in apple forum , https://forums.developer.apple.com/message/229368

from forge.