Proposed RFC Feature Specialization Constants about sig-graphics-audio HOT 8 CLOSED

The DXIL blob can't be patched inline if the LLVM encoding of integers is variable.
It will have to be copied, and patched sequentially from stream position 0 to end.
As we encounter integer values, the VBR encoding needs to be computed, take its size and copy the rest from the original stream+oldVBRsize.
repeat.
This blob patch can't be parallelized, or patched inline without copy unless it's proven that all new VBR values will have the same size as the old ones. Maybe a hack can be found were initial VBR values are large enough, and some leading zeros are still possible in this particular encoding, but I doubt it since the continuation bit concept is going backward from LSB to MSB.

It's honestly quite dreadful that the DX12 penchant of that RFC is so hacky, but I'm not emitting further judgement or conclusions based on this feeling, just expressing it.

About the -sc-options switch, can we not rather use a language modifier like [[specialize]] on option declaration lines?
Because if all options become spec constants, we lose the chance of Ahead Of Time optimizer going through program-wide opportunites when we do have the variant bytecode. Maybe keeping that possibility around for a small number of core options is desirable. Especially since we can't throw away all the code about variant preparations yet, so if it's there, might as well use it where it can be at its most powerful.

from sig-graphics-audio.

The DXIL blob can't be patched inline if the LLVM encoding of integers is variable. It will have to be copied, and patched sequentially from stream position 0 to end. As we encounter integer values, the VBR encoding needs to be computed, take its size and copy the rest from the original stream+oldVBRsize. repeat. This blob patch can't be parallelized, or patched inline without copy unless it's proven that all new VBR values will have the same size as the old ones. Maybe a hack can be found were initial VBR values are large enough, and some leading zeros are still possible in this particular encoding, but I doubt it since the continuation bit concept is going backward from LSB to MSB.

I had the same reservations about patching with a smaller number when using VBR, so I built a prototype to see if it was possible. In the prototype I was able to successfully patch a number that was using 5 bytes in VBR (1164413184) with a 1 (that also uses 5 bytes). I just need to set the continuation bit properly for each byte. The patching works and I was able to run the shader on O3DE without problems.

It's honestly quite dreadful that the DX12 penchant of that RFC is so hacky, but I'm not emitting further judgement or conclusions based on this feeling, just expressing it.

I agree that is a hack, but we don't have more options. If you think of another way of doing it, please feel free to suggest it. Something to keep in mind is that this approach is currently implemented in Godot, which has thousands of users. Worst case we can easily disable specialization constant for DX12 by changing just one file.

About the -sc-options switch, can we not rather use a language modifier like [[specialize]] on option declaration lines? Because if all options become spec constants, we lose the chance of Ahead Of Time optimizer going through program-wide opportunites when we do have the variant bytecode. Maybe keeping that possibility around for a small number of core options is desirable. Especially since we can't throw away all the code about variant preparations yet, so if it's there, might as well use it where it can be at its most powerful.

Adding an attribute per option is a good idea, but I would do it in reverse. Specialization constant on by default (or whatever the API decides), and can be overridden to disable it. Currently shader variants are a major pain in mid to big projects. It has ballooned assets processing times and it still produces inefficient variants (a lot of register pressure). Since our shader variant options are being used mostly for branching, I'm pretty confident that the driver can do the simple optimization. I don't see the need of the AOT optimizer of LLVM for those simple optimizations. But, if there's a more complex usage of a specific shader variant option, then I think the attribute to bake it makes sense. About the -sc-options, we still need a way to do global disable/enable, so I would change it to -force-sc-options=disable or something like that. A simple test that we could do is check the performance by enabling and disabling the specialization constants to see if the offline compilation produces better results.

from sig-graphics-audio.

Great design. We can keep Shader and ShaderVariant class as is. If the application calls Shader.GetShaderVariant(ShaderVariantId) then we now ha ve all the information for the ShaderVariant class to automatically run the Specialization code.
If the application calss Shader.GetRootVariant() then they'll get the root variant and no Specialization will be done, unless the application calls the new Specialization APIs on the returned Root Variant.

Can't wait to see this in action!

from sig-graphics-audio.

Great design. We can keep Shader and ShaderVariant class as is. If the application calls Shader.GetShaderVariant(ShaderVariantId) then we now ha ve all the information for the ShaderVariant class to automatically run the Specialization code.

Unfortunately we don't have all the information if we don't modify the ShaderVariant class. When calling Shader.GetShaderVariant(ShaderVariantId), you don't get a ShaderVariant with id ShaderVariantId. You may get the "closest" one, and in the case of the specialization constants, it would be the root. So the ShaderVariantId of the returned ShaderVariant would be 0, hence we just lost the values of the shader variant options. Only if you get the perfect match, the ShaderVariantId that was used for the GetShaderVariant function is the same as the id of the returned ShaderVariant.

If the application calss Shader.GetRootVariant() then they'll get the root variant and no Specialization will be done, unless the application calls the new Specialization APIs on the returned Root Variant.

The problem is that the application HAS to call the new specialization API. Since the shader was built with specialization constants, you need to specialize it in order to use it. There's no way of using the fallback system once the shader was built with specialization constants.

from sig-graphics-audio.

but I would do it in reverse

sounds ok :)

from sig-graphics-audio.

Like the design and looking forward to the perf gains from this feature. We could start with implementation for Vk and metal for mobile as that is where this feature will shine. DX12 can come last. I do have one question- With option 1 can we use shaders where some shader options are using the old way and other are using function constants? If yes then I like this approach. I am worried that we are going to find that using some shader options as function constants may cause long compilation times at runtime and if that is the case can we convert then to the current way of using shader options. Essentially allowing us to offline compile shader options if needed.

from sig-graphics-audio.

RFC is approved and green lit for implementation

from sig-graphics-audio.

Like the design and looking forward to the perf gains from this feature. We could start with implementation for Vk and metal for mobile as that is where this feature will shine. DX12 can come last. I do have one question- With option 1 can we use shaders where some shader options are using the old way and other are using function constants? If yes then I like this approach. I am worried that we are going to find that using some shader options as function constants may cause long compilation times at runtime and if that is the case can we convert then to the current way of using shader options. Essentially allowing us to offline compile shader options if needed.

With the proposed RFC you will be able to disable specialization constant per platform, per RHI, per Shader and per Shader Option.

from sig-graphics-audio.