Getting "IndexError: Encountered an index error" when running train.py on datasets generated using scenegenv7.py about gnn-pressure-estimation HOT 3 CLOSED

Hallo Sugandhrise,

The reason depends on network-by-network.
In designing INP, the designer (creation software) often adds auxiliary nodes/ edges. You need to clean the INP file or isolate "unused" nodes and edges in code before generating datasets or training models.
If the INP file is available, we are happy to help.

Cheers.

from gnn-pressure-estimation.

Hallo Sugandhrise,

The reason depends on network-by-network. In designing INP, the designer (creation software) often adds auxiliary nodes/ edges. You need to clean the INP file or isolate "unused" nodes and edges in code before generating datasets or training models. If the INP file is available, we are happy to help.

Cheers.

Hey Huy,

I have tried it on Net1.inp from https://github.com/KIOS-Research/epanet-function-test/blob/master/Net1.inp and C-Town.inp from https://www.batadal.net/data/CTOWN.INP.

Could you please elaborate more on what you mean by unused nodes and edges?

Thanks

from gnn-pressure-estimation.

Update: I used the same .zip file for dataset_paths and test_data_path and the same .inp file for input paths and test_input_path, so the command for executing train.py became this (notice it's only NET.zip and NET.inp):

python train.py --model gatres_small --epochs 500 --batch_size 8 --device 'cuda' --mask_rate 0.95 --dataset_paths /home/sugandh/gnn2/gnn-pressure-estimation/datasets/NET/datasets/NET.zip --input_paths /home/sugandh/gnn2/gnn-pressure-estimation/inputs/NET.inp --save_path /home/sugandh/Downloads/ --test_data_path /home/sugandh/gnn2/gnn-pressure-estimation/datasets/NET/datasets/NET.zip --test_input_path /home/sugandh/gnn2/gnn-pressure-estimation/inputs/NET.inp

and I got this error:

Traceback (most recent call last):
  File "/home/sugandh/gnn2/gnn-pressure-estimation/train.py", line 611, in <module>
    train(args, model=model, do_load=False)
  File "/home/sugandh/gnn2/gnn-pressure-estimation/train.py", line 520, in train
    train_ds, val_ds, test_ds = get_default_datasets(args)
  File "/home/sugandh/gnn2/gnn-pressure-estimation/train.py", line 86, in get_default_datasets
    test_ds = get_stacked_set(zip_file_path=args.test_data_path,  # fullnode
  File "/home/sugandh/gnn2/gnn-pressure-estimation/utils/DataLoader.py", line 440, in get_stacked_set
    test_train_ds = WDNDataset(zip_file_paths=[zip_file_path],
  File "/home/sugandh/gnn2/gnn-pressure-estimation/utils/DataLoader.py", line 127, in __init__
    graph_template, array, keep_list = self.collect(input_path, zip_file_path, feature, edge_attrs, removal,
  File "/home/sugandh/gnn2/gnn-pressure-estimation/utils/DataLoader.py", line 252, in collect
    array = np.take(array, taken_indices, axis=-1)
  File "/home/sugandh/anaconda3/envs/gnn/lib/python3.9/site-packages/numpy/core/fromnumeric.py", line 192, in take
    return _wrapfunc(a, 'take', indices, axis=axis, out=out, mode=mode)
  File "/home/sugandh/anaconda3/envs/gnn/lib/python3.9/site-packages/numpy/core/fromnumeric.py", line 59, in _wrapfunc
    return bound(*args, **kwds)
IndexError: index 10 is out of bounds for axis 1 with size 10

When I started digging into the code, I think the error is introduced in DataLoader.py due to the variables "taken_indices" and "array" having the wrong sizes (again difference of 1 in their lengths). Or the error is being introduced in the data generation phase itself.

from gnn-pressure-estimation.