ONNX is a cross-platform model representation and inference runtime that is used by the NVidia Jetson (for example). While the idea of an open model representation sounds good. There is not official support for converting back to pytorch from ONNX. i.e. You cannot use pytorch as the inferene runtime after saving as ONNX.

In order to get accurate training on the multi-GPU instance of PyTorch one must keep the batch size the same. Please see explanation here (https://discuss.pytorch.org/t/accuracy-difference-on-multi-gpu-with-nn-dataparallel/65481/12)

As for the model saving for the multi-GPU instance, one must chance the code to the following (see the following for more information https://discuss.pytorch.org/t/save-checkpoints-trained-on-multi-gpus-for-load-on-single-gpu/97881/9):

#Function to update saved model if validation loss is minimum
def update_saved_model(model, path):
    if not os.path.isdir(path):
        os.mkdir(path)
    for f in os.listdir(path):
        os.remove(os.path.join(path, f))
    if (NGPUS>1):    
        
        if isinstance(model, nn.DataParallel):
            torch.save(model.module.state_dict(), path+'best_model.pth')
        else:
            torch.save(model.state_dict(), path+'best_model.pth')
    else:
        torch.save(model, path+'best_model.pth')

For example, train on full diffraction input (512x512) with output cropped to (128x128) because only the illuminated region in real-space is valid, so why predict for parts of the FOV that are invalid?

Could you please provide a mechanism for querying the NN input and output sizes? (currently everything is 128x128, but it sounds like people want to move to 512x512). PtychoNN only supports square detectors at the moment, but it might be good to design the API with support for rectangular detectors.

This should probably be a parameter of the Tester/model classes.

pvaPY

https://github.com/epics-base/pvaPy/blob/master/documentation/streamingFramework.md

Dear author, I'd like to ask what the parameters point_size and overlap in the code mean and what their functions are?

In order to distribute this package on conda-forge, this codebase needs a license. @mcherukara, should determine who is the copywrite owner and have them/himself add a the appropriate open-source license to the repo. (ANL-1032 if this is Argonne-owned software)

Probably, the training data is copied in memory. Probably when converted from numpy to pytorch Tensor.