[Help]: sbas.topo_ra_parallel() taking too long about pygmtsar HOT 6 CLOSED

Notebook becomes stuck at 15%. Error in NotebookApp:

from pygmtsar.

The logged error is not related to PyGMTSAR, see
tornadoweb/tornado#2271
python/cpython#85269
To resolve it, try to upgrade all your python libraries (especially tornado). Also, do you use chunk size 1024 (instead of default 512) and dask distributed LocalCluster with tcp and connect timeouts 60s? For the reference, an example configuration for 2 stitched scenes 15m for pygmtsar-large Docker image available on GitHub: https://github.com/mobigroup/YamchiDam/tree/main/notebooks

from pygmtsar.

Yes, I am using chunk size 1024:

It took about 12 hours.

I think the log error is not related, probably connection issue.

from pygmtsar.

Not only does sbas.topo_ra_parallel() take very long, it is called a second time in sbas.geocode_parallel(). Is this really necessary? It would only make sense on the condition that sbas.get_topo_ra() is empty.

    def geocode_parallel(self, pairs=None):
        # find any one interferogram to build the geocoding matrices
        if pairs is None:
            pairs = self.find_pairs()
        # build trans_dat and topo_ra grids
        self.topo_ra_parallel()
        # define the target interferogram grid
        intf_grid = self.open_grids(pairs, 'phasefilt').min('pair').astype(bool)
        # build geographic coordinates transformation matrix for landmask and other grids
        self.intf_ll2ra_matrix_parallel(intf_grid)
        # build radar coordinates transformation matrix for the interferograms grid stack        
        self.intf_ra2ll_matrix_parallel(intf_grid)

from pygmtsar.

geocode_parallel() should be called after merge_parallel() when all the subswaths merged together and so it builds the topography matrix for the merged grid. That's possible to enhance it combining single subswath matrices but for now merge_parallel result is unpredictable sometime (GMTSAR binary is used to do it).

Timeouts can be tuned by this code (see the large Docker image or the same notebooks on GitHub):

import dask, dask.distributed
# increase timeouts to work using default chunk size (512) even on large areas
dask.config.set({'distributed.comm.timeouts.tcp': '60s'})
#print (dask.config.get("distributed.comm.timeouts.tcp"))
dask.config.set({'distributed.comm.timeouts.connect': '60s'})
#print (dask.config.get("distributed.comm.timeouts.connect"))

from dask.distributed import Client, LocalCluster
cluster = LocalCluster(n_workers=1)
client = Client(cluster)
client

I think sbas.topo_ra_parallel() requires about 1 hour on 8 core Apple Air laptop for a single scene topography processing using 15m resolution. So for your case it would be about 15 minutes in case you use all the cores (check your "client" variable output). "LocalCluster(n_workers=1)" can be not optimal for your hardware (I use it to limit the used RAM as possible because 2GB RAM per core is not enough for 2 stitched scenes processing by common way and the trick is required), try just "LocalCluster()".

from pygmtsar.

Using larger cluster seems to make this faster.

from pygmtsar.