yshui / index_camera_passthrough Goto Github PK
View Code? Open in Web Editor NEWExperimental Valve Index camera passthrough for Linux
License: MIT License
Experimental Valve Index camera passthrough for Linux
License: MIT License
The build fails with:
error[E0583]: file not found for module `events`
--> src/main.rs:4:1
|
4 | mod events;
| ^^^^^^^^^^^
|
= help: to create the module `events`, create file "src/events.rs" or "src/events/mod.rs"
error: couldn't read /home/chris/Projects/index_camera_passthrough/splash.png: No such file or directory (os error 2)
--> src/main.rs:50:30
|
50 | static SPLASH_IMAGE: &[u8] = include_bytes!(concat!(env!("CARGO_MANIFEST_DIR"), "/splash.png"));
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
= note: this error originates in the macro `include_bytes` (in Nightly builds, run with -Z macro-backtrace for more info)
error[E0433]: failed to resolve: could not find `Action` in `events`
--> src/main.rs:522:21
|
522 | events::Action::ShowOverlay => vroverlay
| ^^^^^^ could not find `Action` in `events`
error[E0433]: failed to resolve: could not find `Action` in `events`
--> src/main.rs:526:21
|
526 | events::Action::HideOverlay => vroverlay
| ^^^^^^ could not find `Action` in `events`
error[E0433]: failed to resolve: could not find `State` in `events`
--> src/main.rs:367:29
|
367 | let mut state = events::State::new(cfg.toggle_button, cfg.open_delay);
| ^^^^^ not found in `events`
|
help: consider importing this enum
|
10 | use serde_json::ser::State;
|
It seems b868aa5 didn't bring in the required package file
I built from a current checkout (cloned right before I built).
I used rust nightly to compile.
I'm using Arch Linux, I did have some packages out of date, but not ones relating to steam or graphics.
I have a NVIDIA GTX 1070 using proprietary drivers and I'm running steam with the native runtime.
I tried the menu button binding and the trigger binding. I have camera disabled in the SteamVR settings.
Steam and SteamVR were up to date. SteamVR version is 1.22.6 Beta. I couldn't find a version number for Steam.
I was using the "Stereo" display mode.
When running in release, it panics after showing a few frames from the camera.
Found matching device: AMD Radeon RX 6800 XT (RADV NAVI21)
thread 'main' panicked at /home/crispypin/.cargo/git/checkouts/vulkano-0df1d54ea6cf1ce4/d828801/vulkano/src/command_buffer/allocator.rs:440:41:
called `Option::unwrap()` on a `None` value
stack backtrace:
0: rust_begin_unwind
at /rustc/2831701757eb7b3105eda26a306c2f3a97e2664b/library/std/src/panicking.rs:645:5
1: core::panicking::panic_fmt
at /rustc/2831701757eb7b3105eda26a306c2f3a97e2664b/library/core/src/panicking.rs:72:14
2: core::panicking::panic
at /rustc/2831701757eb7b3105eda26a306c2f3a97e2664b/library/core/src/panicking.rs:127:5
3: vulkano::command_buffer::allocator::Entry::allocate
4: <vulkano::command_buffer::allocator::StandardCommandBufferAllocator as vulkano::command_buffer::allocator::CommandBufferAllocator>::allocate
5: vulkano::command_buffer::auto::builder::AutoCommandBufferBuilder<L>::begin
6: index_camera_passthrough::pipeline::submit_cpu_image
7: index_camera_passthrough::pipeline::Pipeline::run
8: index_camera_passthrough::main
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
Hi, I was mostly doing my own research on this and I thought you might be interested.
Vulkano's default StandardMemoryAllocator allocates around 576MB of memory even if you don't upload anything. This is not great for an overlay app that just wants to display a 720p image.
A typical AMD card is known to expose 3 heaps; 1 DEVICE_LOCAL, 1 HOST_VISIBLE and usually 1 smaller (<1GB) DEVICE_LOCAL | HOST_VISIBLE.
StandardMemoryAllocator::default_new()
will allocate 256MB per large (>1GB) heap and 64MB for the small heap, making it seem like your app is using 576MB memory (+ openxr swapchains, as they are not managed by vulkano).
The answer for me was to explicitly set the heap sizes instead of using new_default()
.
Built from source, throws this when i try to run:
[2024-03-15T01:47:27Z INFO index_camera_passthrough] width : 1920
height : 960
fourcc : YUYV
field : progressive
stride : 3840
size : 3686400
colorspace : sRGB
quantization : default
transfer : Rec. 709 transfer function
[2024-03-15T01:47:27Z INFO index_camera_passthrough] OpenVR
Found matching device: NVIDIA GeForce RTX 4090
[2024-03-15T01:47:28Z WARN index_camera_passthrough] No camera parameters found
[2024-03-15T01:47:28Z INFO index_camera_passthrough::vrapi] Dropping overlay handle
[2024-03-15T01:47:28Z INFO index_camera_passthrough::openvr] Shutdown OpenVR
Error: a non-validation error occurred
Caused by:
0: allocating memory for the image failed
1: finding a suitable memory type failed
Edit:Missed the last two lines.
For some reason the allocation of yuv_texture
in pipeline.rs
fails:
Error: a non-validation error occurred
Caused by:
0: allocating memory for the image failed
1: finding a suitable memory type failed
The Vulkan wrapper isn't finding the right memory type to create the image. I might be overlooking a flag but it seems like a suitable memory type is present though:
$ vulkaninfo
...
memoryHeaps: count = 3
memoryHeaps[0]:
size = 8589934592 (0x200000000) (8.00 GiB)
budget = 4746379264 (0x11ae80000) (4.42 GiB)
usage = 0 (0x00000000) (0.00 B)
flags: count = 1
MEMORY_HEAP_DEVICE_LOCAL_BIT
memoryHeaps[1]:
size = 25176649728 (0x5dca53000) (23.45 GiB)
budget = 25176649728 (0x5dca53000) (23.45 GiB)
usage = 0 (0x00000000) (0.00 B)
flags:
None
memoryHeaps[2]:
size = 257949696 (0x0f600000) (246.00 MiB)
budget = 198508544 (0x0bd50000) (189.31 MiB)
usage = 59441152 (0x038b0000) (56.69 MiB)
flags: count = 1
MEMORY_HEAP_DEVICE_LOCAL_BIT
memoryTypes: count = 11
# ... more memory types ...
memoryTypes[8]:
heapIndex = 1
propertyFlags = 0x0006: count = 2
MEMORY_PROPERTY_HOST_VISIBLE_BIT
MEMORY_PROPERTY_HOST_COHERENT_BIT
usable for:
IMAGE_TILING_OPTIMAL:
None
IMAGE_TILING_LINEAR:
color images
(non-sparse, non-transient)
memoryTypes[9]:
heapIndex = 1
propertyFlags = 0x000e: count = 3
MEMORY_PROPERTY_HOST_VISIBLE_BIT
MEMORY_PROPERTY_HOST_COHERENT_BIT
MEMORY_PROPERTY_HOST_CACHED_BIT
usable for:
IMAGE_TILING_OPTIMAL:
None
IMAGE_TILING_LINEAR:
color images
(non-sparse, non-transient)
memoryTypes[10]:
heapIndex = 2
propertyFlags = 0x0007: count = 3
MEMORY_PROPERTY_DEVICE_LOCAL_BIT
MEMORY_PROPERTY_HOST_VISIBLE_BIT
MEMORY_PROPERTY_HOST_COHERENT_BIT
usable for:
IMAGE_TILING_OPTIMAL:
None
IMAGE_TILING_LINEAR:
color images
(non-sparse, non-transient)
The overlay does work if I just change the memory type filter for this one allocation to force the allocation to run:
diff --git a/src/pipeline.rs b/src/pipeline.rs
index c632b4b..b6b17bd 100644
--- a/src/pipeline.rs
+++ b/src/pipeline.rs
@@ -226,8 +226,7 @@ impl Pipeline {
..Default::default()
},
AllocationCreateInfo {
- memory_type_filter: MemoryTypeFilter::HOST_SEQUENTIAL_WRITE
- | MemoryTypeFilter::PREFER_DEVICE,
+ memory_type_filter: MemoryTypeFilter::PREFER_DEVICE,
..Default::default()
},
)?;
System information for reference:
bones@rog ~/Programs/git/xr-overlays/index_camera_passthrough $ RUST_LOG=debug cargo run --release
Finished release [optimized] target(s) in 0.12s
Running `target/release/index_camera_passthrough`
[2024-01-30T07:09:18Z INFO index_camera_passthrough] width : 1920
height : 960
fourcc : YUYV
field : progressive
stride : 3840
size : 3686400
colorspace : sRGB
quantization : default
transfer : Rec. 709 transfer function
[2024-01-30T07:09:18Z DEBUG index_camera_passthrough] loading splash
[2024-01-30T07:09:18Z DEBUG index_camera_passthrough] splash loaded
[2024-01-30T07:09:18Z INFO index_camera_passthrough] OpenXR
LOG in xrCreateInstance: Instance created
createInfo->applicationInfo.applicationName: Camera
createInfo->applicationInfo.applicationVersion: 0
createInfo->applicationInfo.engineName: engine
createInfo->applicationInfo.engineVersion: 0
appinfo.detected.engine.name: (null)
appinfo.detected.engine.version: 0.0.0
quirks.disable_vulkan_format_depth_stencil: false
quirks.no_validation_error_in_create_ref_space: true
LOG in xrCreateInstance: Selected devices
Head: 'Index HMD'
Eyes: '<none>'
Left: 'Valve Index Controller'
Right: 'Valve Index Controller'
Gamepad: '<none>'
Hand-Tracking Left: 'Valve Index Controller'
Hand-Tracking Right: 'Valve Index Controller'
LOG in xrCreateVulkanInstanceKHR: Creation of VkInstance:
result: VK_ERROR_LAYER_NOT_PRESENT
vulkanInstance: 0x0
extensions:
VK_KHR_external_fence_capabilities
VK_KHR_external_memory_capabilities
VK_KHR_external_semaphore_capabilities
VK_KHR_get_physical_device_properties2
VK_EXT_debug_utils
VK_KHR_device_group_creation
VK_KHR_display
VK_KHR_get_display_properties2
VK_KHR_get_surface_capabilities2
VK_KHR_portability_enumeration
VK_KHR_surface
VK_KHR_surface_protected_capabilities
VK_KHR_wayland_surface
VK_KHR_xcb_surface
VK_KHR_xlib_surface
VK_EXT_acquire_drm_display
VK_EXT_acquire_xlib_display
VK_EXT_debug_report
VK_EXT_direct_mode_display
VK_EXT_display_surface_counter
VK_EXT_surface_maintenance1
VK_EXT_swapchain_colorspace
VK_LUNARG_direct_driver_loading
Error: vulkan error: Layer specified does not exist
Caused by:
Layer specified does not exist
Hello, I'm sorry if the solution to this issue is obvious. I've looked up the error, but I couldn't find anything.
OS: Arch Linux
Rust Nightly
Installed via rustup from the official source
I have openvr, and nvidia-utils installed
I'm getting this error when trying to build index_camera_passthrough
error: failed to run custom build command for v4l2-sys-mit v0.2.0
Caused by:
process didn't exit successfully: /tmp/cargo-installSET54P/release/build/v4l2-sys-mit-057ded0b20d221c6/build-script-build (exit status: 101)
--- stderr
thread 'main' panicked at /home/matrix/.cargo/registry/src/index.crates.io-6f17d22bba15001f/proc-macro2-1.0.69/src/fallback.rs:817:9:
"v4l2_pix_format_union_(anonymous_at_/usr/include/linux/videodev2_h_500_2)" is not a valid Ident
note: run with RUST_BACKTRACE=1 environment variable to display a backtrace
warning: build failed, waiting for other jobs to finish...
error: failed to compile index_camera_passthrough v0.1.0, intermediate artifacts can be found at /tmp/cargo-installSET54P.
It seems a file named splash.png
should be included in the source, but is not.
error: couldn't read /home/claire/GIT/index_camera_passthrough/splash.png: No such file or directory (os error 2)
--> src/main.rs:50:30
|
50 | static SPLASH_IMAGE: &[u8] = include_bytes!(concat!(env!("CARGO_MANIFEST_DIR"), "/splash.png"));
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
= note: this error originates in the macro `include_bytes` (in Nightly builds, run with -Z macro-backtrace for more info)
error: could not compile `index_camera_passthrough` due to previous error
Creating a blank 1920ร960 png works fine.
If the v4l2 device expected in the config is not found this is the result
Error: No such file or directory (os error 2)
Using the OG Vive (single camera) by manually setting the camera path in config causes a crash in either VR backend.
[2024-03-05T15:55:16Z INFO index_camera_passthrough] width : 612
height : 460
fourcc : YUYV
field : progressive
stride : 1224
size : 563040
colorspace : sRGB
quantization : default
transfer : Rec. 709 transfer function
...
[2024-03-05T15:55:16Z WARN index_camera_passthrough] No camera parameters found
[2024-03-05T15:55:16Z INFO index_camera_passthrough::pipeline] Adjusted FOV: [[1.19, 1.19], [1.19, 1.19]]
Error: copy_buffer_to_image_info: `regions[0].buffer_offset` plus the number of bytes being copied is greater than `src_buffer.size()` (Vulkan VUIDs: VUID-VkCopyBufferToImageInfo2-pRegions-00171)
Backtrace (backtrace.log):
...
[2024-03-05T15:25:41Z TRACE index_camera_passthrough] state: Capturing
[2024-03-05T15:25:41Z TRACE index_camera_passthrough] getting camera frame
[2024-03-05T15:25:41Z TRACE index_camera_passthrough] got camera frame Instant { tv_sec: 7818, tv_nsec: 405975511 }
[2024-03-05T15:25:41Z TRACE index_camera_passthrough] state: Capturing
[2024-03-05T15:25:41Z TRACE index_camera_passthrough] getting camera frame
Error: copy_buffer_to_image_info: `regions[0].buffer_offset` plus the number of bytes being copied is greater than `src_buffer.size()`
Stack backtrace:
0: anyhow::error::<impl core::convert::From<E> for anyhow::Error>::from
at /home/lea/.cargo/registry/src/index.crates.io-6f17d22bba15001f/anyhow-1.0.75/src/error.rs:551:25
1: <core::result::Result<T,F> as core::ops::try_trait::FromResidual<core::result::Result<core::convert::Infallible,E>>>::from_residual
at /rustc/d18480b84fdbf1efc34f62070951334aa833d761/library/core/src/result.rs:1964:27
2: index_camera_passthrough::pipeline::Pipeline::submit_cpu_image
at ./src/pipeline.rs:188:9
3: index_camera_passthrough::pipeline::Pipeline::run
at ./src/pipeline.rs:323:26
4: index_camera_passthrough::main
at ./src/main.rs:415:38
[2024-03-05T15:25:41Z TRACE index_camera_passthrough] got camera frame Instant { tv_sec: 7818, tv_nsec: 422727511 }
[2024-03-05T15:25:41Z TRACE index_camera_passthrough] state: Capturing
[2024-03-05T15:25:41Z TRACE index_camera_passthrough] getting camera frame
Changed config values:
backend = "OpenXR"
camera_device = "/dev/video0"
With a standard rust install, building gives the error
error[E0554]: `#![feature]` may not be used on the stable release channel
--> openvr_sys/src/lib.rs:1:1
|
1 | #![feature(untagged_unions)]
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
For more information about this error, try `rustc --explain E0554`.
The readme should note the requirement of pre-release rust.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.