Incorrect translation/mapping for D3D12_SUBOBJECT_TO_EXPORTS_ASSOCIATION about vortice.windows HOT 21 CLOSED

I marshalled most of the types, can you try and see if it works?
Do you have any example of raytracing?

from vortice.windows.

It doesn't seem work yet, although I'm not sure about the exact reason. It crashes in Vortice.Direct3D12.ID3D12Device5.CreateStateObject(Vortice.Direct3D12.StateObjectDescription) with another memory AccessViolationException. I believe this is a different one though, because the other one earlier today was certainly fixed.

I'm doing a line-by-line conversion of the tutorial code at https://github.com/NVIDIAGameWorks/DxrTutorials/ and I'm currently stuck at part 04: https://github.com/NVIDIAGameWorks/DxrTutorials/tree/master/Tutorials/04-RtPipelineState
Part 04 is almost entirely about StateSubObjects.
The previous parts worked fine up until 03, which initializes Direct3D12, sets up AccelerationStructures and clears the render target.
(Edit: The Cpp code works fine, so it's not an issue of the OS or SDK or anything)

My code currently doesn't live at a public repository, but I could send it over. Beware though, it's very messy, because my goal right now is simply to get it to work and then clean it up later (this means stuff like: Really large classes, lowercased public variables, ....)

from vortice.windows.

Problem is I don't have raytracing capable GPU and I'm not able to test the code

from vortice.windows.

I think the main issue is still within SubObjectToExportsAssociation and StateSubObject.

If you look at the code at https://github.com/NVIDIAGameWorks/DxrTutorials/tree/master/Tutorials/04-RtPipelineState the subobjects and their descriptions are cross-referenced everywhere.
I'm absolutely no expert on Marshalling and the magic behind it, but to me it looks like the C# code creates copies of everything (both of the references in the StateSubObjects (Description), as well as the StateSubObjects themselves, which are structs), which may cause problems depending on how the underlying native code expects the data to behave.

I did try to change StateSubObject to a class in the mapping.xml, which allowed me to make some minor progress, but there were more problems after this.

I'm hitting a wall here, because I don't know enough about neither SharpGenTools nor Marshalling to be much of help.

from vortice.windows.

On other side I don't have capable GPU :(
Will try to give better look.

from vortice.windows.

Thanks for your efforts by the way. It's really great that someone is working on the DX12 bindings in this way.
I'll think about it some more, maybe I can come up with something as well.

from vortice.windows.

Initial better interop landed, it currently handles only DxilLibraryDescription, will marshal next one.

Tell me if it works.

from vortice.windows.

The DxilLibraryDescription seems to work fine. However, a StateObjectDescription with only a DxilLibrary doesn't seem to be valid way to create a StateObject. The main culprit was the SubObjectToExportsAssociation anyway and for that I need at least one more class to establish the association to. Preferably the LocalRootSignature.

Some good news though. I wanted to continue with the implementation of the tutorial code, so I moved the createRtPipeline method of Tutorial04 (the one with all the StateSubObjects) to a native library. I'm only passing in the ID3DDevice5 as IntPtr and receive the created StateObject as an out IntPtr.
This allowed me to implement the rest of the tutorials in C# and I now have a shiny, raytraced triangle on my screen. Yay!
So it seems that only the StateSubObjects are causing issues right now, but everything else seems to work fine.

from vortice.windows.

I will try to fix it ASAP the you can continue with implementation and submit your examples somewhere

from vortice.windows.

Marshalled rest of types, can you try please?

from vortice.windows.

Well, the good news is, I've finally got it working without native code, but the bad news is, not directly with your code.

The problems is still within the StateSubObject references, or more specifically how the StateSubObjects are marshalled in the StateObjectDescription passed into "device.CreateStateObject()".
It's pretty obvious if you really look at what the C++ code does and what the CreateStateObject function actually expects.
I did not really pay close attention to how that works and for that I apologize!

So let's look at the C++ code:

std::array<D3D12_STATE_SUBOBJECT,16> subobjects;

// ... lots of code with subobjects omitted

D3D12_STATE_OBJECT_DESC desc;
desc.NumSubobjects = index; // 16
desc.pSubobjects = subobjects.data();
desc.Type = D3D12_STATE_OBJECT_TYPE_RAYTRACING_PIPELINE;

d3d_call(mpDevice->CreateStateObject(&desc, IID_PPV_ARGS(&mpPipelineState)));

It's pretty clear that the CreateStateObject function expects an array of structs (not pointers to structs).
With this in mind, let's look at another part of the code:

LocalRootSignature triHitRootSignature(mpDevice, createTriangleHitRootDesc().desc);
subobjects[index] = triHitRootSignature.subobject;

uint32_t triHitRootIndex = index++;
ExportAssociation triHitRootAssociation(&kTriangleChs, 1, 
        &(subobjects[triHitRootIndex])); // <--------- This is the important part!
subobjects[index++] = triHitRootAssociation.subobject;

The ExportAssociation is created with a reference to the already existing element in the original "subobjects" array from above.
This means that not only the ExportAssociation.pAssociatedSubObject is a reference to any SubObject, but an exact reference into the array. And this is what's missing right now in the C# code.
The marshalling code of StateObjectDescription creates a completely new array of StateSubObject.__Native, thus breaking the references again.

What I did now is to rework how the StateSubObject.__Native instances are created. They are no longer creted a result of marshalling, but are instead created in the StateObjectDescription as a backing store for StateSubObjects, who also no longer create new instances of StateSubObject.__Native. When a new StateSubObject is created it receives a reference to the already existing struct in memory.
So even with all the cross-referencing, there are, at any given time, exactly only those StateSubObject.__Native instances alive that will eventually be passed into CreateStateObject.

The same code as above in C#:

StateObjectDescription desc = 
    new StateObjectDescription(StateObjectType.RaytracingPipeline, 16);

// ... stuff

MyLocalRootSignature triHitRootSignature = 
    new MyLocalRootSignature(device, CreateTriangleHitRootDesc());
desc.AssignStateSubObject(index, triHitRootSignature.subObject); // <--------- Ensure that the original native instance is reused

int triHitRootIndex = index++;
ExportAssociation triHitRootAssociation = 
    new ExportAssociation(
        new string[] { TriangleClosestHitShader }, 
        desc.GetStateSubObject(triHitRootIndex)); // <--------- Get the reference
desc.AssignStateSubObject(index++, triHitRootAssociation.SubObject);

// ... stuff

pipelineState = device.CreateStateObject(desc);

If you only have a flat list of StateSubObjects, without any references, the original marshalling code in StateObjectDescription works fine, but with the references it doesn't. To summarize, neither the original code could have ever worked, nor the changed code that we produced over the last few days, but we now have a solution.

I could send you the changes as a patch to give you an idea of what I did (it's not much). The code is not directly usable in production, because it basically does everything wrong (memory leaks, makes stuff public that shouldn't be, etc.), it's just a hack to make stuff work.

Let me know what you think

from vortice.windows.

By the way, I guess it would be possible to keep the original flow intact (which is more C#-like). That would mean to completely rebuild the structure along with any native references inside the MarshalTo of StateObjectDescription.cs.

Edit: Now that I've thought about it, this is likely the much better solution, because it limits the lifetime of unmanaged objects to the __MarshalTo/__MarshalFree method pair, instead of keeping them around for a much longer time.
It does mean, that the same StateSubObjects cannot be reused for multiple calls into CreateStateObject, but I'm not sure if that is a requirement anyway.

from vortice.windows.

So I created a variant of the marshalling code for StateObjectDescription where the public interface and workflow remains the same and the only changes are internal. It works in my sample code.

Here's the patch:
https://gist.github.com/CAMongrel/c0530211e552303b7ed6b650184f1460

The number of changed files is larger than in the other version, but that's only because of a change in a method signature in IStateSubObjectDescriptionMarshal.

Summary:
The patch creates a lookup table for StateSubObjects to their native pointers in StateObjectDescription.MarshalTo(), passing this into the marshalling method of SubObjectToExportsAssociation, which will check if the referenced StateSubObject is part of the lookup table and uses the existing address if that's the case (instead of creating a new instance).
If not, it throws an exception, although I'm not sure if there aren't valid cases where the SubObjectToExportsAssociation has a reference to a StateSubObject which is not part of the StateObjectDescription.SubObjects array.

Please review.

from vortice.windows.

Does raytracing work now? I can review and apply patch if yes

from vortice.windows.

Yes it does.

from vortice.windows.

Just pushed merged changes.

from vortice.windows.

Send me link with raytracing example when you put them somewhere so I tweet and add it in Readme please

from vortice.windows.

When will you update the NuGet package? I think it'd be helpful to correctly reference the updated NuGet packages in the sample instead of custom compiled binaries. Makes everything much easier.

from vortice.windows.

1.3.0 was just release and official 1.3.0 packages are on nuget.

from vortice.windows.

Sample code is now public at:
https://github.com/CAMongrel/D3D12SampleRaytracerSharp

from vortice.windows.

Just tweeted here https://twitter.com/AmerKoleci/status/1175693767363432450?s=19

from vortice.windows.