💡 [Dynamic Graph] - Does nnsmith support dynamic graphs? about nnsmith HOT 3 CLOSED

Thank you so much for providing such thoughtful and detailed answers@lazycal @ganler. I learned a lot from it. I just read the paper of nnsmith and am currently trying to use it. It is an impressive piece of work, and I hope it continues to develop and improve even further:)

from nnsmith.

Hi, do you mean graphs with dynamic control flow that depends on the data, like graphs with tf.cond? Current NNSmith won't generate such graphs, but it may be possible to extend it a bit by adding a new operator and materialize it with true_outputs if cond else false_outputs. I am not sure if torch is able to convert this into a valid ONNX graph though, will need to take a look.

For RNN, it requires a loop structure I think, and that might be even harder to support (currently we assume dag structure when generating graphs).

I am not very familiar with Deformable Conv, but looking at this https://pytorch.org/vision/main/generated/torchvision.ops.deform_conv2d.html, it doens't seem to be dynamic, but just some normal operator like convolution (with one more input)?

from nnsmith.

The representation capability for DL Models of nnsmith is bounded by the internal IR (GraphIR) we use. Currently the shape of the IR is rather simple but still Very extensible. Basically it presents a model with a set of SSA-like instructions (calling it like is because I dislike to make use of Tuples and GetTupleItems for fetching and parsing multiple return values which makes the IR system complicated for just a few corner cases. So unlike most SSA impl it would do in place unpacking when there are multiple return values).

Each instruction follows a continuation pass style, that you can wrap it with any function/operator but just need specify a few high level type information. Also the function being called must be side effect free.

Going back to your question, can nnsmith support control flows? The answer is yes and no. To quickly support op such as RNN you can regard a whole RNN phase as a function being called in an instruction. It could work perfectly because we said as long as the callable is side effect free we can put it in the instruction. However, if you want to have a more fine-grained control to put the control flow over the instructions, it is not currently available. But it is not hard to support such cases where you can just add control flow instructions among multiple GraphIR. And actually that is how advanced DL compilers such as torch inductor supports such dynamism (they split a model into multiple subgraphs using graph breakpoint and do optimization at subgraph wise). Why does not NNSmith support that immediately? Because it seems that is still not a strong motivation in the current phase of Deep Learning Deployment. Many existing compilers still only optimized models of static shapes and graphs. On the other side, runtime dependent branches are neither widely optimized nor optimizable where ppl oftentimes just do graph break pointing and only optimize the static part (more optimizable given more information available at compile time).

Is that all what we can do now? We are still trying to improve the IR to tolerate more side effects such as in place modification. Like a.fold_ is not functional as it modifies the content of a. Such cases are commonly used in real world and we will support it sooner or later by performer a strong graph analysis at model construction time.

Hope these information can bring you a better understanding of the current NNSmith!

from nnsmith.