Thank you Lu for such exciting work.

RIght now, I want to use DeepONet for flow problems. My input is of the dimension (Samples, Height, Width, Time) or images corresponding to different time steps depicting the evolution of flow. Being specific, my input is velocity from. t = 1 to m time steps and I want to predict the velocity for next n time steps [ batch, :,:,:m] -> [ batch,:,:, m: m+n].

How should I preprocess the data for such a problem? Is there any available code implementation for such flow problem using deepOnet?

Also, can we use DeepOnet like FNO-2D-time( more of RNN type structure)?

HELLO, Lu：
where can i find the code and dataset of "Compressible Euler equations with non-equilibrium chemistry" ?

Hi, Professor Lu! I really want to reproduce this work for a fair comparison between foo and deponent. However, I encountered some troubles when I try to reproduce some cases, for example, the NS case, I do not know how to obtain the prediction result and replace the first time step result of the original input with the prediction one in the training, just as what fno does.

It may be caused by my awkward usage about deepxde. So could I ask for the other codes to share in this repository?

Or will you update the pytorch one?

Thank you!

Xinyuan

NotImplementedError: <tensorflow_addons.optimizers.weight_decay_optimizers.AdamW object at 0x00000136F434F6D0> to be implemented for backend tensorflow.compat.v1.
in src\darcy_rectangular_pwc\deeponet.py.

    model.compile(
        tfa.optimizers.AdamW(1e-4, learning_rate=3e-4),
        lr=3e-4,
        decay=("inverse time", 1, 1e-4),
        metrics=["mean l2 relative error"],
    )

where I have add lr=3e-4, if I use the original code

    model.compile(
        tfa.optimizers.AdamW(1e-4, learning_rate=3e-4),
        decay=("inverse time", 1, 1e-4),
        metrics=["mean l2 relative error"],
    )

I got anther error
ValueError: No learning rate for <tensorflow_addons.optimizers.weight_decay_optimizers.AdamW object at 0x00000284AC2BF100>.

Hi

I was trying to use the POD-DeepONet code on Burgers dataset and ran into ValueError: No variables to save error while compiling the model. I think this is related to Tensorflow. Do you have PyTorch version for POD-DeepONet?

Thanks

Hi Prof. Lu,

Thank you for your great code and dataset.

I want to apply deeponet to a multivariate system (e.g. containing u,v,p and others). How can I organize data and input them? For example, I'm using CartesianDeepONet, current data input for branch net is (n_samples, n_spatial_dims), how can I add more variables to it?

Thank you!

Hello Prof. @lululxvi

Thank you for putting together DeepXDE. I was trying to run the Burgers code with a PyTorch backend however got a variety of errors. The first one was in the following lines

def periodic(x):
   x *= 2 * np.pi
   return tf.concat(
       [tf.math.cos(x), tf.math.sin(x), tf.math.cos(2 * x), tf.math.sin(2 * x)], 1
   )

tf was not getting imported properly so I used torch.cat instead. I then got Leaf Variable was used in in-place operation for the line x *= 2 * np.pi. So I fixed that.

Then I got a datatype mismatch error while training the network, so I changed all float32 in code to float64 and also set the default float to float64 -

dde.config.set_default_float("float64")

I then got -

mat1 and mat2 shapes cannot be multiplied (128x4 and 1x128)

So likely, a "transpose" has to be taken in some layer and is missing but I cannot figure out where exactly. Could you please help me with this so I can run using a PyTorch backend.

Additionally, Here is my version info-

I am trying to run the code on darcy flow and getting following error. Please look into it. Thanks

AttributeError Traceback (most recent call last)
/tmp/ipykernel_238106/3930051686.py in
1 m = 29 ** 2
2 activation = "relu"
----> 3 branch = tf.keras.Sequential([
4 tf.keras.layers.InputLayer(input_shape=(m,)),
5 tf.keras.layers.Reshape((29, 29, 1)),

AttributeError: 'function' object has no attribute 'keras'

你好！
我的deepxde版本是0.11.2，然后运行burgers方程的code时报错

请问要怎么修改呀？期待回复~

Hi Prof. Lu,

In FNO architure, the input x is not periodic. Will this have a negative impact on the results?

Thank you!

卢教授您好!
从你的代码和数据集学到很多！
我想问一下input或者output的数据类型可以是复函数吗

你好！
我在40G的A100卡上训练您的burgers方程的deepONet.py文件时，报错oom：

请问下怎么修改来完成多卡训练呀？抱歉我对DeepXDE的使用不是很熟悉，期待回复~

Hi Prof. Lu,

Thank you for providing the code for the experiments!

I'm trying to run the models on the Burgers' equation, but I get errors on the PyTorch and TF sides.

For example, when running FNO with PyTorch, something is wrong with Adam:

Traceback (most recent call last):
  File ".../deeponet-fno/src/burgers/fourier_1d.py", line 322, in <module>
    FNO_main(training_data_resolution, save_index)
  File ".../deeponet-fno/src/burgers/fourier_1d.py", line 225, in FNO_main
    optimizer.step()
  File ".../miniconda3/envs/fair-torch/lib/python3.11/site-packages/torch/optim/lr_scheduler.py", line 69, in wrapper
    return wrapped(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../miniconda3/envs/fair-torch/lib/python3.11/site-packages/torch/optim/optimizer.py", line 280, in wrapper
    out = func(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^
  File ".../miniconda3/envs/fair-torch/lib/python3.11/site-packages/torch/optim/optimizer.py", line 33, in _use_grad
    ret = func(self, *args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../miniconda3/envs/fair-torch/lib/python3.11/site-packages/torch/optim/adam.py", line 141, in step
    adam(
  File ".../miniconda3/envs/fair-torch/lib/python3.11/site-packages/torch/optim/adam.py", line 281, in adam
    func(params,
  File ".../miniconda3/envs/fair-torch/lib/python3.11/site-packages/torch/optim/adam.py", line 442, in _multi_tensor_adam
    device_grads = torch._foreach_add(device_grads, device_params, alpha=weight_decay)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: The size of tensor a (2) must match the size of tensor b (16) at non-singleton dimension 3

And when trying to run DeepONet-POD with TF, there is an error complaining about tf.Saver:

Traceback (most recent call last):
  File ".../deeponet-fno/src/burgers/deeponet_POD.py", line 131, in <module>
    main()
  File ".../deeponet-fno/src/burgers/deeponet_POD.py", line 122, in main
    model.compile(
  File ".../miniconda3/envs/fair-tf/lib/python3.9/site-packages/deepxde/utils/internal.py", line 22, in wrapper
    result = f(*args, **kwargs)
  File ".../miniconda3/envs/fair-tf/lib/python3.9/site-packages/deepxde/model.py", line 131, in compile
    self._compile_tensorflow_compat_v1(lr, loss_fn, decay, loss_weights)
  File ".../miniconda3/envs/fair-tf/lib/python3.9/site-packages/deepxde/model.py", line 158, in _compile_tensorflow_compat_v1
    self.saver = tf.train.Saver(max_to_keep=None)
  File ".../miniconda3/envs/fair-tf/lib/python3.9/site-packages/tensorflow/python/training/saver.py", line 934, in __init__
    self.build()
  File ".../miniconda3/envs/fair-tf/lib/python3.9/site-packages/tensorflow/python/training/saver.py", line 946, in build
    self._build(self._filename, build_save=True, build_restore=True)
  File ".../miniconda3/envs/fair-tf/lib/python3.9/site-packages/tensorflow/python/training/saver.py", line 971, in _build
    raise ValueError("No variables to save")
ValueError: No variables to save

I believe it happens because of some updates in the frameworks. So could you please provide the exact versions of the packages you used to run the experiments?

Thank you!

Best,
Iryna

Dear Prof. Lu,

For the numerical experiment "Linear instability waves in high-speed boundary layers", in the arXiv version, the FNO is unstable to added noise, while in the published version, the FNO is stable to noise. I also have some numerical experiences that FNO is unstable to noise. Could you please explain why you have found that FNO is stable to noise in your published version? Thank you.

I am trying to run the code on Darcy_rectangular_pwc and getting following error. Please look into it. Thanks

code:

branch = tf.keras.Sequential(
      [
          tf.keras.layers.InputLayer(input_shape=(m,)),
          tf.keras.layers.Reshape((29, 29, 1)),
          tf.keras.layers.Conv2D(64, (5, 5), strides=2, activation=activation),
          tf.keras.layers.Conv2D(128, (5, 5), strides=2, activation=activation),
          tf.keras.layers.Flatten(),
          tf.keras.layers.Dense(128, activation=activation),
          tf.keras.layers.Dense(128),
      ]
  )
  branch.summary()
  net = dde.maps.DeepONetCartesianProd(
      [m, branch], [2, 128, 128, 128, 128], activation, "Glorot normal"
  )

error:

in SingleOutputStrategy.build(self, layer_sizes_branch, layer_sizes_trunk)
     29 def build(self, layer_sizes_branch, layer_sizes_trunk):
     30     if layer_sizes_branch[-1] != layer_sizes_trunk[-1]:
---> 31         raise AssertionError(
     32             "Output sizes of branch net and trunk net do not match."
     33         )
     34     branch = self.net.build_branch_net(layer_sizes_branch)
     35     trunk = self.net.build_trunk_net(layer_sizes_trunk)
AssertionError: Output sizes of branch net and trunk net do not match.**

RuntimeError: The size of tensor a (2) must match the size of tensor b (16) at non-singleton dimension 3;
After removes optimizer 's param weight_decay, the model run normally,but training abnormal
How should I modify it？

Hi Professor Lu, thanks for the well-written codes. I am reading the advection data generation code, case 3, where the input function is defined to be a square wave plus a continuous bump, the following are your parameter setting:

    c1 = 0.1 * rng.random() + 0.2  # [0.2, 0.3]
    w = 0.1 * rng.random() + 0.1  # [0.1, 0.2]
    h1 = 1.5 * rng.random() + 0.5  # [0.5, 2]
    c2 = 0.1 * rng.random() + 0.7  # [0.7, 0.8]
    a = 5 * rng.random() + 5  # 1 / [0.1, 0.2] = [5, 10]
    h2 = 1.5 * rng.random() + 0.5  # [0.5, 2]
    pde = Advection_v2(0, 1, c1, w, h1, c2, a, h2)

Thus, the c2 + a / h2 could be bigger than 1. In pde.solve code, seems like this case is never handled, and because of the periodic boundary condition assumption, the current implementation may have the following inputs:
Initial condition:

solution at t = 29/40

which means currently, the initial boundary will always be 0, which is not ideal (shall not be a big concern though). I am not sure if this is intended. Please let me know if there are some reasons behind this, thanks!

Hello, I'm sorry to bother you again. Could you please give me the script for synthesizing two datasets of Darcy? I haven't figured it out yet. in your “A comprehensive and fair comparison of two neural operators (with practical extensions) based on FAIR data” darcy_rectangular_pwc question

Hi Lu,

I am running your code for example of darcy_rectangular_pwc. I note that in your paper, you used hard constraints, as mentioned, "We use the coefficient 20 such that 20x(1−x)y(1−y) is of order 1 for x ∈ [0, 1] and y ∈ [0, 1]." I want to add it.

Then, I write the following code

def output_transform(inputs, output):
    x_loc = inputs[1]
    x1, x2 = x_loc[:, 0:1], x_loc[:, 1:2]
    final_output = x1 * x2 * (1 - x1) * (1 - x2) * output * 20
    return final_output

net.apply_output_transform(output_transform)

However, it does not work for me.

"File "/public/home/hpc214801033/DeepONet/Darcy/src/BVx_H/deeponet_BVx_H.py", line 82, in output_transform
final_output = x1 * x2 * (1 - x1) * (1 - x2) * (y * std + scaler.mean_.astype(np.float32))
File "/public/home/hpc214801033/.conda/envs/tensorflow2.91/lib/python3.10/site-packages/tensorflow/python/util/traceback_utils.py", line 150, in error_handler
return fn(*args, **kwargs)"

What happens to this?

Best regards,

Peng