autodesk / arnold-usd Goto Github PK
View Code? Open in Web Editor NEWArnold components for USD
Home Page: https://www.arnoldrenderer.com/
License: Other
Arnold components for USD
Home Page: https://www.arnoldrenderer.com/
License: Other
maketx is part of the contrib folder, but it's not used for anything. Just clean it. If we need it in the future, the Arnold SDK does ship one.
Newer USD expanded the HdxCompositor to allow for non-8 bit int buffers for the color. We should take advantage of this and use f32 for beauty.
Related to: #8
Describe the bug
Currently, our schemas generation does some exceptions for a few attributes, whose name (or values) causes problems for USD. This happens because usdGenSchema
creates a file tokens.h
with all the attribute names and enum tokens, and compiles it. But some of the names can't compile.
Here's the current list :
//////////////////////////////////////////////
The attribute namespace
(from procedural
), will introduce this in tokens.h
:
/// UsdArnoldAlembic
const TfToken nameprefix;
/// \brief "namespace"
and then fails to compile :
tokens.h(2992): error C2059: syntax error: 'namespace'
The attribute default
(from user_data_*
) seems to be a problem for usdGenSchema
as it adds an underscore :
///
/// UsdArnoldUserDataString, UsdArnoldUserDataRgb, UsdArnoldUserDataRgba, UsdArnoldUserDataInt, UsdArnoldUserDataFloat
const TfToken default_;
and then fails to compile :
build\windows_x86_64\msvc_opt\usd-0.19.5_arnold-6.0.1.0\schemas\source\arnoldUse
rDataRgba.cpp(125): error C2039: 'default': is not a member of 'arnoldPxrInterna
l__pxrReserved__::UsdArnoldTokensType'
We have a similar problem with enum values, since they're also added as tokens in this same file.
Value auto
(from polymesh.subdiv_adaptive_metric
) creates this:
/// \brief "auto"
///
/// Possible value for UsdArnoldPolymeshAPI::GetArnoldSubdiv_adaptive_metricAttr(), Default value for UsdArnoldPolymeshAPI::GetArnoldSubdiv_adaptive_metricAttr(), Possible value for UsdArnoldPolymesh::GetSubdiv_adaptive_metricAttr(), Default value for UsdArnoldPolymesh::GetSubdiv_adaptive_metricAttr()
const TfToken auto;
which obviously fails to compile
0\schemas\source\.//tokens.h(684): error C2628: 'arnoldPxrInternal__pxrReserved_
_::TfToken' followed by 'auto' is illegal (did you forget a ';'?)
Our shader attribute compare.test
has enum values like ==
, <
, >
, etc...
and this is a problem for usdGenSchema
:
ERROR: string index out of range
Finally, we have problems with tokens upper/lower casing. Arnold's utility
shader has a parameter color_mode
with possible values p
, n
, ng
, ns
... but state_vector.Ro
has values P
, N
, Ng
, Ns
.... This isn't allowed in schema tokens, and fails with :
ERROR: Token identifiers must map to exactly one token value. One-to-Many mappin
g encountered: ng maps to "ng" and "Ng"
Currently, the functions for pushing primvar data to Arnold nodes will return early if a user parameter cannot be declared. This prevents dirty primvars from propagating to Arnold after they are initially declared.
Describe the bug
The #include <ai_map.h> in arnold-usd\translator\reader\registry.h fails when compiling against Arnold 5. I assume ai_map is only part of Arnold 6.
To Reproduce
Steps to reproduce the behavior:
Just try compiling.
Used Software Versions
Additional context
Add any other context about the problem here.
In Arnold 6, procedurals can define a new function procedural_viewport that determines what is displayed in the viewport. This ticket is about implementing this function in an efficient way, by just loading what is needed
UsdMaterialBindingAPI supports the use of material binding caches, which make computing material bindings more efficient.
Describe the solution you'd like
The procedural should use a material binding cache to compute its material bindings.
Additional context
This ticket is the continuation of #59 .
The procedural checks if a mesh has subdivision by invoking the function UsdGeomMesh::GetSubdivisionSchemeAttr()
, which returns catmull-clark by default.
The problem is that Arnold has polymesh.subdiv_iteration=1
by default. The combination of these 2 defaults causes all meshes to always appear with a 1-level catmull-clark subdivision when rendered with the procedural, which should be changed
Using Hydra and UsdImaging for a procedural is an alternative approach to directly accessing and traversing a Usd Stage. This is a similar approach to what the HdPrman plugin does and it offers significant code reuse between the Render Delegate and the Procedural. Besides this, Hydra also takes care of parallel access to the stage automatically.
Creating a procedural prototype should be straightforward, but the most important thing is to make sure performance is similar to a handwritten procedural
The default smoothing follows the Arnold default (off). We should invert by default like in all plugins.
Hi !
Amazing to see this repo go live !
I've been successfully able to compile against Houdini's shipped libraries with a few modifications and was wondering if these could become part of the main build system.
First, USD libs in Houdini have the libpxr_ prefix, so it would be nice to have a build option like BOOST_LIB_NAME for USD.
Second, it seems like the USD_HAS_UPDATED_COMPOSITOR doesn't set correctly when building against Houdini's USD libs. Setting it up manually has fixed it in my experience.
Thanks !
Add support for hydra curves.
We can use hdPrman/basisCurves.cpp as an example implementation.
USD allows writing powerful custom file format plugins. An ass plugin would allow us to do the following:
The feature is best implemented following the example of the
USD Alembic example plugin .
Refactor the render buffer system:
Add support for Hydra's coordSys
feature that allows shaders to define coordinate systems for transformations.
We should add a set of Schema APIs that simplify using USD stages in an Arnold focused pipeline. Due to USD's great python support, this would allow us to give TDs and Developers a great C++/Python toolset that's easy to use.
Some of the features we could add:
Example:
usd-arnold's UsdAi API
GitHub allows the use of templates for issues and pull-requests, this allows setting the value of the description box (and others) to guide the users in creating issues. This is done via storing markdown files in the .github subfolder of the repository.
The render delegate does not support per face material assignments at this time. We need to investigate what are the best options to do so.
Primvars are usually converted to arnold user data.
But as it is currently done in the render delegate, primvars in the arnold namespace, e.g. "primvars:arnold:max_bounces" should be converted to arnold builtin attributes.
This behaviour is currently helpful in the render delegate for the DCC / plugins integration, so we should mirror it in the procedural for consistency.
Add support for reading the vertex normals from the Scene Delegate. This feature is useful if the polymesh is not subdivided and the user supplies custom vertex normals.
This requires adding HdChangeTracker::DirtyNormals
to the initial dirty bits and query HdTokens->normals
from the scene delegate.
A good example is : hdEmbree/mesh.cpp
USD allows renderers to define shader and node registry plugins. This lets users and different tools to access node/shader parameter types and default values using USD tools.
Katana 3.2 introduced support for Hydra render delegates, and we should make sure the render delegate works with Katana 3.2 and later releases.
Document naming conventions for the various Arnold entities, via a markdown file and add it to doxygen.
Add tests for the Render Delegate.
We can use usdrecord from the newer USD versions to render any USD scene via an offscreen buffer. The result can be compared to a reference image, similarly to the current tests for the Procedural.
Describe the bug
When I open the PointInstancedMedCity example file with usdView and choose Arnold as the renderer, then the whole usdView app crashes. It renders nicely with OGL and Embree.
To Reproduce
Used Software Versions
Luma Pictures' usd-arnold provides a Maya Shading Mode Exporter plugin, which can be used to export Arnold shader definitions from usdMaya. Even though this repository might not be the final place for that exporter, we should allow early adopters to build the plugin and use it in its current form.
On linux, running usdview with the arnold render delegate crashes. The current workaround for this is to ld-preload the arnold library.
So we need to run
LD_PRELOAD=/path_to_arnold/bin/libai.so usdview scene.usda
and this prevents the crashes
Core ticket: #8869
Add support for the Hydra camera primitives and the physical camera attributes.
References:
PixarAnimationStudios/OpenUSD@c77e2e4
PixarAnimationStudios/OpenUSD@0b84f3a
To make the render delegate/procedural more flexible for users and allow them to modify its behavior, we should support writing plugins for the render delegate/procedural.
There are several ways we could allow users to customize behavior.
1, define custom material remapping functions, similar to the ones we use in the render delegate. The interface is already well defined, we just have to expose the material context class and provide a few example callbacks.
For example, someone could use this to translate a custom set of OpenGL shaders to Arnold shaders.
2, custom USD primitive to Arnold node writer for the procedural. The procedural works by defining writers for each supported USD primitive type, we can expose this base class and allow rendering of custom primitives this way.
For example, Luma has a custom primitive type for image planes in USD, and this interface should allow them to convert that to a background image.
3, custom Hydra to Arnold node. The render delegate creates Hd classes to translate between the Hydra primitives and the Arnold nodes, we should expose a plugin registry that allows users to register their own classes for custom Hydra types.
The USD ecosystem has a so-called plugin registry system. There are utility macros that let everybody easily define a custom plugin system for any sort of functionality. A good example is the Maya shading mode system.
Example classes:
This deals with registering and returning registered shading mode writers. Just a simple singleton class, that uses the TfType system to call into dynamic libraries that can register their own writer functions.
The display color shading mode
The following code showcases how to register a new shading mode writer.
TF_REGISTRY_FUNCTION_WITH_TAG(UsdMayaShadingModeExportContext, displayColor)
{
UsdMayaShadingModeRegistry::GetInstance().RegisterExporter(
"displayColor",
[]() -> UsdMayaShadingModeExporterPtr {
return UsdMayaShadingModeExporterPtr(
static_cast<UsdMayaShadingModeExporter*>(
new DisplayColorShadingModeExporter()));
}
);
}
The classes are already set up for external use in the render delegate, and headers are installed as part of the package. The same is not yet done for the procedural.
Arnold encounters error when loading a usdz file that has embedded textures.
To Reproduce
usdview --renderer Arnold simpleMaterial.usdz
ERROR | [texturesys] Image "C:\Users\treborc\Documents\usdme\pxr\usd\lib\usdUtils\testenv\testUsdUtilsCreateNewUsdzPackage\nestedUsdz\simpleMaterial.usdz[0/USDLogoLrg.png]" does not exist. Also, it is not the name of an image format that OpenImageIO recognizes.
Expected behavior
Able to open the usdz file without errors and load texture
Used Software Versions
Describe the feature you'd like
The ability to specify a full path to the gcc compiler, rather than restricting it to gcc
only
Describe the solution you'd like
Able to set COMPILER="/path/to/bin/gcc
in the custom.py
file
Describe alternatives you've considered
Tried to set it, but ALLOWED_COMPILERS
does not permit this. Also tried setting CXX
and CC
in the custom.py
file with no luck
The Procedural supports using a static USD build, but that limits extensibility and the option to run custom plugins. We need to investigate what's the best option to load plugins in a procedural that's using a static build of USD.
This most likely requires shipping the USD headers, pluginInfos, and exposing the USD symbols in the procedural.
Currently the USD writer only converts Arnold to arnold-specific schemas (ArnoldStandardSurface, ArnoldPolymesh, ArnoldSkydomeLight, etc....).
We should allow, at least optionally if not by default, to export to USD-native schemas when it makes sense.
For example an skydome_light could exported as a Dome primitive, and we'd add the additional arnold attributes in the arnold: scope.
At the moment the Render Delegate is using the P AOV to calculate the OpenGL depth and the ID AOV to differentiate between the background and foreground pixels.
The Z AOV should be better suited for both problems, as the P and ID AOV's behavior is inconsistent when using the GPU backend or mixing volumes and other objects.
Support the Hydra point instancer for primitives.
Initial support was already added for instancing meshes see render_delegate/instancer.cpp and related files.
A few features are still missing:
We can use hdEmbree/instancer.cpp as an example implementation.
The houdini python library has a -mt suffix on windows.
Add tests for the Writer command.
A good approach is to execute python scripts as part of the testsuite and compare the resulting USD file to a reference file.
Currently, performing a blocking Hydra render to completion requires a pattern similar to the following:
auto tasks = taskController->GetRenderingTasks();
do {
engine.Execute(renderIndex, &tasks);
}
while (!taskController->IsConverged());
Right now, if the render performed by HdArnoldRenderPass
aborts for any reason, the render pass is not marked as converged, and thus the above code will block indefinitely.
I think the most "correct" thing to do in the case of an Arnold render error is to mark the pass as converged and zero the render pass' output buffers.
The build system requires setting up the USD_HAS_PYTHON_SUPPORT to know if USD was built with python support or not. This is tracked in the installed headers, so we should be able to check for this during parsing the USD version.
Describe the bug
Compiling fails to build against USD 19.11 with this error:
render_delegate\render_pass.cpp(177): error C2660: 'pxrInternal_v0_19__pxrReserved__::HdxCompositor::UpdateColor': function does not take 3 arguments
D:\dev\usd\build\include\pxr/imaging/hdx/compositor.h(66): note: see declaration of 'pxrInternal_v0_19__pxrReserved__::HdxCompositor::UpdateColor'
The built usd version is: 0.19.11
. Which according to the merge release of 19.11 contains the updated UpdateColor
arguments including HdFormat format
.
As such this falsely enters the else
statement due to USD_HAS_UPDATED_COMPOSITOR
only being defined for USD 19.12 and up.
To Reproduce
Steps to reproduce the behavior:
arnold-usd
against it.Expected behavior
It should compile correctly.
It continues compiling when this line is changed to:
#if USED_USD_VERSION_GREATER_EQ(19, 11)
Used Software Versions
5.4.0.2
Neither the Procedural nor the Translators make use of the standard TfDebug toolset, which allows fine-grained control over the printed debug messages. Adding debug output would greatly simplify any future debugging.
The material adapter and the Render Delegate debug codes are good examples to follow.
The build fails with the latest USD version, due to the removal of UsdShadeMaterial::GetBoundMaterial
.
To Reproduce
Steps to reproduce the behavior:
Used Software Versions
We should handle arnold metadata and store additional information in the ndr plugins.
This include:
Describe the solution you'd like
We need to extend ndr/utils.cpp
to deal with metadata. Add additional USD metadata to store the raw arnold type on the primitives in the Usd Stage returned by NdrArnoldGetShaderDefs
. And pass metadata to Usd in the Discovery and Parser plugins.
Add a usdKatana scenegraph location decorator plugin that allows converting Arnold specific attributes to KtoA compatible attributes. This also includes supporting remapping of namespaced primvars to built-in Arnold parameters, a feature that is supported by the Render Delegate.
example:
usd-arnold Katana plugin
There should be a render setting to turn on the Optix denoiser for the beauty AOV.
The difficulty here is that the composition, responsible for differentiating between background and foreground pixels, happens at the moment per bucket, and the Optix denoised beauty is available at a different rate than the non-denoised ID and P AOV. We need to remove the requirement of having all 3 buffers available at the same time for each bucket and update how composition behaves.
The Render Delegate should utilize the TfDebug utilities and print information about receiving invalid data and extend it's currently used debug codes.
The material adapter already makes use of TfDebug, and we should follow its example.
Currently the objects happen to compute a matrix obtained by calling UsdGeomXformable::GetLocalTransformation
, since there's no function in UsdGeomXformable
to get a world matrix. Objects are obviously not in the right position if their parent was transformed.
In order to get the right matrix, we should use a XformCache
that efficiently caches the hierarchy matrices for multiple uses.
Describe the feature you'd like
At this stage it's unclear how to actually use the arnold-usd
repository and how to adopt or even test the functionality.
Describe the solution you'd like
It would be good to briefly elaborate what the correct environment should be after building, e.g. what should be added to PATH
or maybe to some Arnold-related environment variables to have the procedurals/plug-ins found. It would be good to have a simple example (potentially per OS).
With arnold_to_usd
as a helper utility it could be good to have a Getting Started tutorial that describes taking an example Arnold .ass
file and take it through the executable.
For example:
:: Psuedocode (random example)
set PATH={root}/bin;%PATH%
set PATH={root}/lib;%PATH%
set PATH={root}/procedural;%PATH%
set ARNOLD_PLUGIN_PATH={root}/plugin
Run arnold_to_usd
to convert it to a usd stage.
arnold_to_usd /path/to/input.ass /path/to/output.usd
Add the procedural to an Arnold scene and load the created USD scene. (or can we render it directly?)
(Optional) Show how one could layer a change into the .usd
file and influence the Arnold render using a secondary .usd
file.
#usda 1.0
over "Camera"
{
float[] arnold:focus_distance = [50.0]
}
Let's move the husdshadertranslators python module, and the HdArnoldRendererPlugin*.ds files to HtoA, and so remove them from the arnold_usd repository.
The Ndr Discovery and Parser plugins should register all the other node types available in arnold.
Describe the solution you'd like
All we need to do is to change the Node Entry iterator and manually handle the options node in ndr/utils.h
.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.