inconsistent behavior of optional input tensor binding about directml HOT 3 CLOSED

Sorry for the delayed response! Most of us are still on holiday vacation. :)

This looks like a bug in DirectMLX. When there is a single node in a graph, the dml::Graph::Compile function is trying to eliminate some overhead by compiling the node as a standalone operator. This optimization only works when the bindings for the graph are identical to the bindings for the sole node in the graph. In your example they are not, since the graph input bindings do not include the third (optional) bias tensor for the convolution node.

The key thing to remember is that you're binding for the graph; you're not binding for the first node and last nodes in the graph. When you create a graph, you're effectively defining a new operator with its own bindings. Each call to dml::InputTensor with a unique inputIndex is adding another input binding to the graph. Your code is explicitly creating two input bind points for the graph:

auto input = dml::InputTensor(graph, 0, ...);
auto filter = dml::InputTensor(graph, 1, ...);

However, you could have declared the input and filter tensors in the opposite order:

auto filter = dml::InputTensor(graph, 0, ...);
auto input = dml::InputTensor(graph, 1, ...);

Either way, the graph itself only has two inputs. The graph inputs are wired to the node inputs based on the expressions you pass in: since you call dml::Convolution(input, filter) it will wire the graph's input at whatever index the input and filter tensors have when you called dml::InputTensor. It's a little easier to explain with a figure:

Pinging @adtsai (our binding expert) for his thoughts when he's back from vacation, but I believe we should remove this optimization or somehow detect when it's possible to do safely. For now, you can comment out lines 630-638 to work around.

from directml.

Some one can comment this for me ? :)

from directml.

thanks. So the "graph compiler" will deal with the optional binding automatically.

from directml.