luisagroup / luisacompute Goto Github PK
View Code? Open in Web Editor NEWHigh-Performance Rendering Framework on Stream Architectures
License: BSD 3-Clause "New" or "Revised" License
High-Performance Rendering Framework on Stream Architectures
License: BSD 3-Clause "New" or "Revised" License
Describe the bug
After reordering, submitting the should-be-independent commands to a single CUDA stream in the REVERSED order triggers illegal memory access on the device.
To Reproduce
As aforementioned; when rendering the Glass-of-Water scene.
Expected behavior
The correct rendering.
Screenshots
N/A
Desktop (please complete the following information):
Additional context
TODO: construct a minimal reproducible example...
Bad.
Input:
import luisa
@luisa.kernel
def fill(x: int):
b.write(dispatch_id().x, x)
Report:
Process: Python [11639]
Path: /usr/local/Cellar/[email protected]/3.9.10/Frameworks/Python.framework/Versions/3.9/Resources/Python.app/Contents/MacOS/Python
Identifier: Python
Version: 3.9.10 (3.9.10)
Code Type: X86-64 (Native)
Parent Process: Python [11577]
Responsible: Terminal [684]
User ID: 501
Date/Time: 2022-04-20 14:40:32.793 +0800
OS Version: Mac OS X 10.15.7 (19H2)
Report Version: 12
Bridge OS Version: 5.4 (18P4663)
Anonymous UUID: 4A973336-E1A5-5C2F-38CB-EF3E8B8A3302
Sleep/Wake UUID: CBE6D37B-50D4-4C27-ABF1-00A08BA1B74F
Time Awake Since Boot: 25000 seconds
Time Since Wake: 2700 seconds
System Integrity Protection: enabled
Crashed Thread: 0 Dispatch queue: com.apple.main-thread
Exception Type: EXC_CRASH (SIGABRT)
Exception Codes: 0x0000000000000000, 0x0000000000000000
Exception Note: EXC_CORPSE_NOTIFY
Application Specific Information:
terminating with uncaught exception of type pybind11::error_already_set: Exception: error when building AST
At:
/Users/bilibili/code/LuisaCompute/build_debug/bin/luisa/astbuilder.py(26): __call__
/Users/bilibili/code/LuisaCompute/build_debug/bin/luisa/__init__.py(87): astgen
/Users/bilibili/code/LuisaCompute/build_debug/bin/luisa/__init__.py(92): __init__
/var/folders/9m/jl3vmk4d4cg5k2wjnfc7kzwm0000gn/T/ipykernel_11639/2580251621.py(2): <module>
/usr/local/lib/python3.9/site-packages/IPython/core/interactiveshell.py(3457): run_code
/usr/local/lib/python3.9/site-packages/IPython/core/interactiveshell.py(3377): run_ast_nodes
/usr/local/lib/python3.9/site-packages/IPython/core/interactiveshell.py(3185): run_cell_async
/usr/local/lib/python3.9/site-packages/IPython/core/async_helpers.py(78): _pseudo_sync_runner
/usr/local/lib/python3.9/site-packages/IPython/core/interactiveshell.py(2960): _run_cell
/usr/local/lib/python3.9/site-packages/IPython/core/interactiveshell.py(2914): run_cell
/usr/local/lib/python3.9/site-packages/ipykernel/zmqshell.py(533): run_cell
/usr/local/lib/python3.9/site-packages/ipykernel/ipkernel.py(353): do_execute
/usr/local/lib/python3.9/site-packages/ipykernel/kernelbase.py(648): execute_request
/usr/local/lib/python3.9/site-packages/ipykernel/kernelbase.py(353): dispatch_shell
/usr/local/lib/python3.9/site-packages/ipykernel/kernelbase.py(446): process_one
/usr/local/lib/python3.9/site-packages/ipykernel/kernelbase.py(457): dispatch_queue
/usr/local/Cellar/[email protected]/3.9.10/Frameworks/Python.framework/Versions/3.9/lib/python3.9/asyncio/events.py(80): _run
/usr/local/Cellar/[email protected]/3.9.10/Frameworks/Python.framework/Versions/3.9/lib/python3.9/asyncio/base_events.py(1890): _run_once
/usr/local/Cellar/[email protected]/3.9.10/Frameworks/Python.framework/Versions/3.9/lib/python3.9/asyncio/base_events.py(596): run_forever
/usr/local/lib/python3.9/site-packages/tornado/platform/asyncio.py(199): start
/usr/local/lib/python3.9/site-packages/ipykernel/kernelapp.py(677): start
/usr/local/lib/python3.9/site-packages/traitlets/config/application.py(846): launch_instance
/usr/local/lib/python3.9/site-packages/ipykernel_launcher.py(16): <module>
/usr/local/Cellar/[email protected]/3.9.10/Frameworks/Python.framework/Versions/3.9/lib/python3.9/runpy.py(87): _run_code
/usr/local/Cellar/[email protected]/3.9.10/Frameworks/Python.framework/Versions/3.9/lib/python3.9/runpy.py(197): _run_module_as_main
abort() called
Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0 libsystem_kernel.dylib 0x00007fff681cf33a __pthread_kill + 10
1 libsystem_pthread.dylib 0x00007fff6828be60 pthread_kill + 430
2 libsystem_c.dylib 0x00007fff68156808 abort + 120
3 libc++abi.dylib 0x00007fff653b5458 abort_message + 231
4 libc++abi.dylib 0x00007fff653a68a7 demangling_terminate_handler() + 238
5 libobjc.A.dylib 0x00007fff66ee15b1 _objc_terminate() + 104
6 libc++abi.dylib 0x00007fff653b4887 std::__terminate(void (*)()) + 8
7 libc++abi.dylib 0x00007fff653b4829 std::terminate() + 41
8 lcapi.cpython-39-darwin.so 0x0000000110cb5f48 auto luisa::compute::detail::FunctionBuilder::_define<std::__1::function<void ()> const&>(luisa::compute::Function::Tag, std::__1::function<void ()> const&) + 120 (function_builder.h:230)
9 lcapi.cpython-39-darwin.so 0x0000000110c88962 auto luisa::compute::detail::FunctionBuilder::define_kernel<std::__1::function<void ()> const&>(std::__1::function<void ()> const&) + 50 (function_builder.h:290)
10 lcapi.cpython-39-darwin.so 0x0000000110cb5d31 eastl::shared_ptr<luisa::compute::detail::FunctionBuilder const> pybind11::detail::argument_loader<std::__1::function<void ()> const&>::call_impl<eastl::shared_ptr<luisa::compute::detail::FunctionBuilder const>, eastl::shared_ptr<luisa::compute::detail::FunctionBuilder const> (*&)(std::__1::function<void ()> const&), 0ul, pybind11::detail::void_type>(eastl::shared_ptr<luisa::compute::detail::FunctionBuilder const> (*&)(std::__1::function<void ()> const&), std::__1::integer_sequence<unsigned long, 0ul>, pybind11::detail::void_type&&) && + 97 (cast.h:1207)
11 lcapi.cpython-39-darwin.so 0x0000000110cb1059 std::__1::enable_if<!(std::is_void<eastl::shared_ptr<luisa::compute::detail::FunctionBuilder const> >::value), eastl::shared_ptr<luisa::compute::detail::FunctionBuilder const> >::type pybind11::detail::argument_loader<std::__1::function<void ()> const&>::call<eastl::shared_ptr<luisa::compute::detail::FunctionBuilder const>, pybind11::detail::void_type, eastl::shared_ptr<luisa::compute::detail::FunctionBuilder const> (*&)(std::__1::function<void ()> const&)>(eastl::shared_ptr<luisa::compute::detail::FunctionBuilder const> (*&)(std::__1::function<void ()> const&)) && + 73 (cast.h:1179)
12 lcapi.cpython-39-darwin.so 0x0000000110cb0e4f void pybind11::cpp_function::initialize<eastl::shared_ptr<luisa::compute::detail::FunctionBuilder const> (*&)(std::__1::function<void ()> const&), eastl::shared_ptr<luisa::compute::detail::FunctionBuilder const>, std::__1::function<void ()> const&, pybind11::name, pybind11::is_method, pybind11::sibling>(eastl::shared_ptr<luisa::compute::detail::FunctionBuilder const> (*&)(std::__1::function<void ()> const&), eastl::shared_ptr<luisa::compute::detail::FunctionBuilder const> (*)(std::__1::function<void ()> const&), pybind11::name const&, pybind11::is_method const&, pybind11::sibling const&)::'lambda'(pybind11::detail::function_call&)::operator()(pybind11::detail::function_call&) const + 239 (pybind11.h:233)
13 lcapi.cpython-39-darwin.so 0x0000000110cb0d45 void pybind11::cpp_function::initialize<eastl::shared_ptr<luisa::compute::detail::FunctionBuilder const> (*&)(std::__1::function<void ()> const&), eastl::shared_ptr<luisa::compute::detail::FunctionBuilder const>, std::__1::function<void ()> const&, pybind11::name, pybind11::is_method, pybind11::sibling>(eastl::shared_ptr<luisa::compute::detail::FunctionBuilder const> (*&)(std::__1::function<void ()> const&), eastl::shared_ptr<luisa::compute::detail::FunctionBuilder const> (*)(std::__1::function<void ()> const&), pybind11::name const&, pybind11::is_method const&, pybind11::sibling const&)::'lambda'(pybind11::detail::function_call&)::__invoke(pybind11::detail::function_call&) + 21 (pybind11.h:210)
14 lcapi.cpython-39-darwin.so 0x0000000110c6e41d pybind11::cpp_function::dispatcher(_object*, _object*, _object*) + 4941 (pybind11.h:835)
...
See this comment.
Maybe use file locks to avoid concurrent writes to the .cache
folder.
>>> import inspect
>>> inspect.getsourcelines(f)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/Cellar/[email protected]/3.9.10/Frameworks/Python.framework/Versions/3.9/lib/python3.9/inspect.py", line 1006, in getsourcelines
lines, lnum = findsource(object)
File "/usr/local/Cellar/[email protected]/3.9.10/Frameworks/Python.framework/Versions/3.9/lib/python3.9/inspect.py", line 835, in findsource
raise OSError('could not get source code')
OSError: could not get source code
88e29e8 运行 build/bin/test_texture
,其中
会在输出一部分123之后段错误
i.e., support the synchronize_block()
built-in
Currently, BUFFER_READ
and BUFFER_WRITE
are used for atomic load/store, which might not honor the correct memory barrier semantics.
Maybe we have to explicitly support built-in ATOMIC_LOAD
and ATOMIC_STORE
.
e.g. luisa.struct
in kernel should translate to luisa.builtin.struct
Bad.
需求
arr = luisa.ArrayType(5, int)
sph = luisa.StructType(center=float3, radius=float)
geo = luisa.StructType(offset=float3, geometry=sph)
buf = luisa.Buffer(100000, dtype=geo)
buf.copy_from(...)
buf.copy_to(...)
@luisa.kernel
def f(a: arr, b: sph):
a1 = arr()
a2 = arr((1,2,3,4,5))
a3 = arr([1,2,3,4,5])
b1 = sph()
b2 = sph(make_float3(1,2,3), 4)
b3 = sph(center=make_float3(1,2,3), radius=4)
# Note: 4 should be implicitly converted to float
b0 = buf.read(dispatch_id().x)
b0.geometry.radius += 1
buf.write(dispatch_id().x, b0)
a = arr(...)
b = sph(...)
g = geo(...)
f(a,b, dispatch_size=...)
待定:Buffer上传下载对应的host数据类型
Probably, support kernel launching in the following style:
stream << kernel(device, args...).dispatch(dim);
Hi folks, I'm building with clang++15, ubuntu 20.04.
I encounter a compilation error that is related with EASTL
Err log:
In file included from /home/dev/LuisaCompute/src/ext/EASTL/include/EASTL/algorithm.h:243:
/home/dev/LuisaCompute/src/ext/EASTL/include/EASTL/internal/copy_help.h:135:90: error: no member named 'contiguous_iterator_tag' in namespace 'std'
(eastl::is_pointer<InputIterator>::value || eastl::is_same<IIC, EASTL_ITC_NS::contiguous_iterator_tag>::value) &&
~~~~~~~~~~~~~~^
/home/dev/LuisaCompute/src/ext/EASTL/include/EASTL/internal/copy_help.h:136:90: error: no member named 'contiguous_iterator_tag' in namespace 'std'
(eastl::is_pointer<OutputIterator>::value || eastl::is_same<OIC, EASTL_ITC_NS::contiguous_iterator_tag>::value);
~~~~~~~~~~~~~~^
/home/dev/LuisaCompute/src/ext/EASTL/include/EASTL/internal/copy_help.h:138:51: error: non-type template argument is not a constant expression
return eastl::move_and_copy_helper<IIC, isMove, canBeMemmoved>::move_or_copy(first, last, result); // Need to chose based on the input iterator tag and not the output iterator tag, because containers accept input ranges of iterator types different than self.
^~~~~~~~~~~~~
This seems the same error with electronicarts/EASTL#482
EASTL commit hash b6f329dc9c09ffaf68a979c6591e66a12cf6c2b9
Do you have any suggestions about this?
Probably use OpenGL for display? Maybe we need to re-design the related interfaces before that...
spdlog output (output from c++ side) is not real-time & has no color in jupyter.
related: rapidsai/cuml#2604
consider a<f()<c
where f has side effects. The expression will be interpreted as a<f() and f()<c
, causing f to be potentially executed twice.
when to_lctype
is called on unsupported (non-data) types. e.g:
xx1 = dispatch_id.xy
error:
(kernel)f:5:10: Error: Exception: to_lctype(<class 'luisa.types.BuiltinFuncType'>): unrecognized type
xx1 = dispatch_id.xy
~~~~~~~~~~~~~~~
I am having problems with LLVM backend.
I am using LLVM 15.0.5 on Linux and I get the following runtime error:
Failed to load dynamic module '/tmp/LuisaCompute/build/bin/libluisa-compute-backend-llvm.so', reason: /tmp/LuisaCompute/build/bin/libluisa-compute-backend-llvm.so: undefined symbol: _ZN4absl12lts_2022062318container_internal11kEmptyGroupE. [/tmp/LuisaCompute/src/core/platform.cpp:165]
Any ideas where the missing abseil symbol went?
When setting device via "--device 0", it will actually use device 1.
当贴图尺寸不符合 2ⁿ×2ⁿ 时,调用 bindless_texture_sample2d_level
可能返回不准确的结果
Use -Wa,-mbig-obj
flag in GNU and /bigobj
flag in MSVC
Bad.
def f(self, a:int):
self.k += a
return self.k
strt = luisa.StructType(k=int, f=f)
@luisa.kernel
f(s: strt):
s.f(4)
>>> float2x2(1,2,3,4)
float2x2([1,2], [3,4])
>>> float2x2(1,2,3,4)[0]
float2(9.731018e-35,1e-45)
Planned modifications:
uint64_t Device::Interface::create_accel(AccelBuildQuality quality, bool allow_update) noexcept;
uint64_t Device::Interface::create_mesh(
uint64_t v_buffer, size_t v_offset, size_t v_stride, size_t v_count,
uint64_t t_buffer, size_t t_offset, size_t t_count,
AccelBuildQuality quality, bool allow_update) noexcept;
where
enum struct AccelBuildQuality {
HIGH, // high-quality build, fast trace, typically with compaction
DEFAULT, // backend decided, balance between build time and trace performance
LOW // fast build, possibly lower trace performance, no compaction
};
Describe the bug
Some meshes disappear after the top-level acceleration structure compaction.
To Reproduce
Steps to reproduce the behavior:
Enable accel compaction and render the Staircase scene.
Expected behavior
All meshes rendered.
Screenshots
None.
Desktop (please complete the following information):
Additional context
Temporally disabling the compaction. May result in some increasing in VRAM usage. Will retry compaction some days later.
Describe the bug
After following the build guide for Python, I cannot import luisa
or run test files.
in build_release/bin
I see lcapi.cpython-310-darwin.so
but no .so to import luisa. I'm assuming this is a PyBind11 thing that the lcapi .so will allow importing luisa but it doesn't see to work:
import lcapi
dir(lcapi)
['Accel', 'AccelBuildCommand', 'AccelBuildRequest', 'AccelModification', 'AccelUsageHint', 'AccessExpr', 'BinaryExpr', 'BinaryOp', 'BindlessArrayUpdateCommand', 'BufferCopyCommand', 'BufferDownloadCommand', 'BufferToTextureCopyCommand', 'BufferUploadCommand', 'CallExpr', 'CallOp', 'CastExpr', 'CastOp', 'Command', 'Context', 'Device', 'DeviceInterface', 'Expression', 'ForStmt', 'FsPath', 'Function', 'FunctionBuilder', 'IfStmt', 'LiteralExpr', 'LoopStmt', 'MemberExpr', 'MeshBuildCommand', 'PixelFormat', 'PixelStorage', 'RefExpr', 'Sampler', 'ScopeStmt', 'ShaderDispatchCommand', 'Stream', 'TextureCopyCommand', 'TextureDownloadCommand', 'TextureToBufferCopyCommand', 'TextureUploadCommand', 'Type', 'UnaryExpr', 'UnaryOp', 'doc', 'file', 'loader', 'name', 'package', 'spec', '_vectorstorage_bool2', '_vectorstorage_bool3', '_vectorstorage_bool4', '_vectorstorage_float2', '_vectorstorage_float3', '_vectorstorage_float4', '_vectorstorage_int2', '_vectorstorage_int3', '_vectorstorage_int4', '_vectorstorage_uint2', '_vectorstorage_uint3', '_vectorstorage_uint4', 'abs', 'acos', 'asin', 'atan', 'atan2', 'bool2', 'bool3', 'bool4', 'builder', 'ceil', 'clamp', 'cos', 'cross', 'degrees', 'determinant', 'distance', 'dot', 'exp', 'float2', 'float2x2', 'float3', 'float3x3', 'float4', 'float4x4', 'floor', 'int2', 'int3', 'int4', 'inverse', 'length', 'lerp', 'log', 'log10', 'log2', 'log_level_error', 'log_level_info', 'log_level_verbose', 'log_level_warning', 'make_bool2', 'make_bool3', 'make_bool4', 'make_float2', 'make_float2x2', 'make_float3', 'make_float3x3', 'make_float4', 'make_float4x4', 'make_int2', 'make_int3', 'make_int4', 'make_uint2', 'make_uint3', 'make_uint4', 'max', 'min', 'normalize', 'pixel_storage_channel_count', 'pixel_storage_size', 'pixel_storage_to_format_float', 'pixel_storage_to_format_int', 'pow', 'radians', 'rotation', 'round', 'scaling', 'select', 'sin', 'sqrt', 'tan', 'to_bytes', 'translation', 'transpose', 'uint2', 'uint3', 'uint4']
import luisa
Traceback (most recent call last):
File "", line 1, in
ModuleNotFoundError: No module named 'luisa'
To Reproduce
Steps to reproduce the behavior:
following https://github.com/LuisaGroup/LuisaCompute/blob/master/README_Python_en.md
python3 src/py/tests/test_path_tracing.py
Traceback (most recent call last):
File "/Users/bsavery/Code/LuisaCompute/src/py/tests/test_path_tracing.py", line 6, in
import luisa
ModuleNotFoundError: No module named 'luisa'
Desktop (please complete the following information):
a = b = c
a or b or c
a < b < c
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.