Autoencoder is identity function on atom coordinates? Equivalence to EDM about geoldm HOT 2 OPEN

Hi Jiaqi,

Thanks a lot for your interest and your close observation! The published version exactly follows my description (https://github.com/MinkaiXu/GeoLDM#train-the-geoldm) that the encoder remained "untrained".

I just followed your interesting observation take a look at the whole model. I guess, but not 100% sure, this phenomenon comes from the initialization of the EGNN layers:

1. x in EGNN encoders is updated with weighted relative direction from neighbors. I found that it looks like the MLPs for computing the weights will be initialed with very small values.
   https://github.com/MinkaiXu/GeoLDM/blob/main/egnn/egnn_new.py#L76
2. Then since x is updated with residual connections, with the small aggregations, the updates become almost identical transformations.
   https://github.com/MinkaiXu/GeoLDM/blob/main/egnn/egnn_new.py#L98

I think your observation also helps us to understand the model's behavior that I could make it work with any KL regularization over z_x (as in the paper appendix ablation study). Seems like the latent code need to keep most of the structural information and leave this part of modeling complexity to the latent diffusion. Considering this perspective, I think your understanding of equivalence to EDM is also true. But I think very likely my current implementation of GeoLDM is still not perfect, and actually some reasonably distorted z_x can lead to better results :)

from geoldm.

Hi Minkai,

Thank you for your quick response! That makes sense. I also believe there should be some better way to implement GeoLDM. Looking forward to future updates and welcome further discussion!

Best,
Jiaqi

from geoldm.