shhslin / neural-network-quantum-state Goto Github PK

Memory Issue, i.e. allocating too much memory, may occur at different part of the calculation.

1.) Local Energy calculation data storage:
For a batch size N_b = 10000, system size 10x10 system, at evaluation time, the configuration is copy 10x10 times, and then evaluate (forward pass) the amplitude.
The data size would easy be of the order 10^9, considering the padding and channels.

2.) plain GD (no SR) back propagation.
with plain GD method, the gradient can be calculated directly from the back propagation of the loss function. Again, if the batch size is too large, memory issue may occur.

Currently, most of variables are in tf.float32 or np.float32. It might result in the error in the outcome. If the condition number of the overall problem is large, say 1e3, the comparison for error around 1e-5 is meaningless. It might be necessary to change the implementation with tf.float64.

Potential Problem:
need to check the whether all related function support type tf.complex128
might not be GPU friendly.

Issue:
Currently the implementation does not include complex-valued wave function. This is because that with a complex-valued function parametrized by real variables, the gradient will be automatically be cast as real type.

See this issue

Another issue is the complex derivative is not implemented in tensorflow, which is the reason why the function are all parameterized by read variables. Even the complex multiplication/ convolution in this implementation are split into (RR-II), i(RI + IR) kind of structure.

Importance:
All complex Hamiltonian could not be solved.
e.g. FQHE related Hamiltonian

Possible Solution:
split the output into real and imaginary parts , then take the gradient separately.

Problem:

As in DMRG algorithm, if one started with large bond dimension, it is likely one would get stuck at local minimum. It is suggested to increase the bond dimension along the optimization process.

It seems also to be crucial to include this process in NNQS training when one start to go deeper.

Difficulties:
The restore function in tensorflow does not support restore part of the matrix. A wrapper is needed to restore part of the variables, i.e. tensor, if the dimension mismatch.