Which complex Softmax activation function shall I use in complex-valued neural networks? about cvnn HOT 4 CLOSED

Well that is an hyperparameter of your network. You can refer to the papers dealing with CVNNs and check what they do. But using either softmax_real_with_abs or softmax_real_with_avg is a very good start I think.

from cvnn.

I think for an appropriate complex Softmax activation function, when the imaginary part of complex numbers is 0, it can degenerate to the real Softmax function.

from cvnn.

That might be an option. However, I think the chances that all complex valued outputs have a zero imaginary part is practically null.

I also ignore what tensorflow's softmax function does in the case of, for example: softmax([0,0,0]) if it randomly choses one or just keep them all at zero. If the second case, then your best choice for what you said is using softmax_of_softmax_real_with_avg and it will have the property you said.

from cvnn.

Thank you very much.

from cvnn.