aevyrie / bevy_mod_raycast Goto Github PK
View Code? Open in Web Editor NEWA little mesh raycasting plugin for Bevy
Home Page: https://crates.io/crates/bevy_mod_raycast
License: MIT License
A little mesh raycasting plugin for Bevy
Home Page: https://crates.io/crates/bevy_mod_raycast
License: MIT License
It would be nice if a hit was detected on all visible faces. For the moment object with a material with a cull_mode
of None
(no backface culling) are not hit.
mouse_picking
example on a high DPI displayRay starts from where the mouse cursor hovers the ball
Ray isn't drawn when hovering the ball. It seemingly unpredictably is drawn when mouse cursor is positioned to the bottom-right from the ball.
This effect only occurs when the camera viewport is offset.
I have a ray cast system that spawns new meshes when the ray hits certain objects. But the Raycast accesses the meshes (with Res<Assets>), so it is not possible to use ResMut<Assets> in the same system because it would cause error B0002 (see https://bevyengine.org/learn/errors/#b0002)
I have a system that looks like this:
pub fn chunk_deform_system(
mut proc_entities: ResMut<ProcEntities>,
camera: Query<&mut Transform, With<CameraComponent>>,
mut raycast: Raycast, // meshes: &mut Assets<Mesh>,
mut commands: Commands,
mut meshes: ResMut<Assets<Mesh>>,
mut materials: ResMut<Assets<StandardMaterial>>,
mut time: ResMut<Time>,
mut buttons: ResMut<Input<MouseButton>>,
) {
The access with mut meshes: ResMut<Assets> causes B0002 but I'm probably missing a better way of doing this
I'm using bevy_mod_raycast in my project. When upgrading from 0.8 to 0.9 along with bevy 0.11 upgrade I started getting a panic. Start of the backtrace that's relevant to the issue:
thread 'Compute Task Pool (15)' panicked at 'ArrayVec: capacity exceeded in extend/from_iter', /home/luke-biel/.cargo/registry/src/index.crates.io-6f17d22bba15001f/arrayvec-0.7.4/src/arrayvec.rs:1133:15
stack backtrace:
0: std::panicking::begin_panic
at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225/library/std/src/panicking.rs:625:12
1: arrayvec::arrayvec::extend_panic
at /home/luke-biel/.cargo/registry/src/index.crates.io-6f17d22bba15001f/arrayvec-0.7.4/src/arrayvec.rs:1057:5
2: arrayvec::arrayvec::ArrayVec<T,_>::extend_from_iter
at /home/luke-biel/.cargo/registry/src/index.crates.io-6f17d22bba15001f/arrayvec-0.7.4/src/arrayvec.rs:1089:46
3: <arrayvec::arrayvec::ArrayVec<T,_> as core::iter::traits::collect::Extend<T>>::extend
at /home/luke-biel/.cargo/registry/src/index.crates.io-6f17d22bba15001f/arrayvec-0.7.4/src/arrayvec.rs:1048:13
4: <arrayvec::arrayvec::ArrayVec<T,_> as core::iter::traits::collect::FromIterator<T>>::from_iter
at /home/luke-biel/.cargo/registry/src/index.crates.io-6f17d22bba15001f/arrayvec-0.7.4/src/arrayvec.rs:1133:15
5: core::iter::traits::iterator::Iterator::collect
at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225/library/core/src/iter/traits/iterator.rs:1895:9
6: bevy_mod_raycast::update_raycast
at /home/luke-biel/.cargo/registry/src/index.crates.io-6f17d22bba15001f/bevy_mod_raycast-0.9.0/src/lib.rs:496:82
When debugging the array vec
I found that the amount of pick points seem to be greater than 4 defined by the array vec as a capacity.
I'm looking for pointers what can be wrong with my setup (maybe model has invalid geometry, I'm also drawing the scene to an egui texture first rather than having it drawn directly to the window), or how in general fix this issue?
For model I'm using https://blendswap.com/blend/18206# converted to gltf through blender exporter. My setup is to iterate over all meshes once a scene is loaded and adding RaycastMesh
to each of them.
Create tests to check common cases that are easy to verify, e.g. fire a ray from each side of a box and make sure the intersection and normal are correct.
It's possible to get into a situation where the raycaster throws an error when the window is closed. This doesn't affect gameplay, of course, but I'm seeing it in the logs and it's not ideal.
To reproduce, this happens every time I run the mouse_picking_deferred
example, and close the window:
This is probably quite minor, but it would be good if this didn't happen!
Hi,
I upgraded to 0.16 from an earlier version where I was using DefaultRaycastingPlugin
, and nothing worked, the RaycastSource
ray was always None
. Turns out build_rays()
was not called because I needed instead DeferredRaycastingPlugin::<T>
. But this doesn't seem to be explained anywhere, and DefaultRaycastingPlugin
is not documented either.
Switching to DeferredRaycastingPlugin::<T>
instead solved everything. Maybe there should be some mention in the docs?
How do you detect multiple overlapping collisions along a raycast? I see that the raycast_source has a function called intersect_list()
. I would have expected that to have multiple collisions for overlapping objects, but it only appears to be detecting 1 object. You will see that in this example project where there is a large transparent sphere and a smaller sphere inside of it. Both have the same RayCastMesh
component.
And here is the code. I extended off of the mouse_picking.rs example:
use bevy::prelude::*;
use bevy_mod_raycast::{DefaultRaycastingPlugin, PluginState, RayCastMesh, RayCastMethod, RayCastSource, RaycastSystem, build_rays, update_debug_cursor, update_raycast};
// This example will show you how to use your mouse cursor as a ray casting source, cast into the
// scene, intersect a mesh, and mark the intersection with the built in debug cursor. If you are
// looking for a more fully-featured mouse picking plugin, try out bevy_mod_picking.
fn main() {
App::build()
.insert_resource(WindowDescriptor {
vsync: false, // We'll turn off vsync for this example, as it's a source of input lag.
..Default::default()
})
.add_plugins(DefaultPlugins)
.init_resource::<PluginState<Gizmoable>>()
.add_system_to_stage(
CoreStage::PreUpdate,
build_rays::<Gizmoable>.system().label(RaycastSystem::BuildRays),
)
.add_system_to_stage(
CoreStage::PreUpdate,
update_raycast::<Gizmoable>
.system()
.label(RaycastSystem::UpdateRaycast)
.after(RaycastSystem::BuildRays),
)
// Draw the debug rays
.add_system_to_stage(
CoreStage::PreUpdate,
update_debug_cursor::<Gizmoable>
.system()
.label(RaycastSystem::UpdateDebugCursor)
.after(RaycastSystem::UpdateRaycast),
)
// You will need to pay attention to what order you add systems! Putting them in the wrong
// order can result in multiple frames of latency. Ray casting should probably happen after
// the positions of your meshes have been updated in the UPDATE stage.
.add_system_to_stage(
CoreStage::PreUpdate,
update_raycast_with_cursor
.system()
.before(RaycastSystem::BuildRays),
)
.add_startup_system(setup.system())
.add_system(detect_selected.system())
.run();
}
// This is a unit struct we will use to mark our generic `RayCastMesh`s and `RayCastSource` as part
// of the same group, or "RayCastSet". For more complex use cases, you might use this to associate
// some meshes with one ray casting source, and other meshes with a different ray casting source."
struct Gizmoable;
struct TranslationGizmo;
struct RotationGizmo;
struct ScaleGizmo;
// Update our `RayCastSource` with the current cursor position every frame.
fn update_raycast_with_cursor(
mut cursor: EventReader<CursorMoved>,
mut query: Query<&mut RayCastSource<Gizmoable>>,
) {
for mut pick_source in &mut query.iter_mut() {
// Grab the most recent cursor event if it exists:
if let Some(cursor_latest) = cursor.iter().last() {
pick_source.cast_method = RayCastMethod::Screenspace(cursor_latest.position);
}
}
}
fn detect_selected(raycast_source: Query<&RayCastSource<Gizmoable>>) {
if let Ok(raycast_source) = raycast_source.single() {
let count = raycast_source.intersect_list().iter().count();
// I would have expected count to equal 2.
println!("{}", count)
}
}
// Set up a simple 3D scene
fn setup(
mut commands: Commands,
mut meshes: ResMut<Assets<Mesh>>,
mut materials: ResMut<Assets<StandardMaterial>>,
) {
commands
.spawn_bundle(PerspectiveCameraBundle::default())
.insert(RayCastSource::<Gizmoable>::new()); // Designate the camera as our source
commands
.spawn_bundle(PbrBundle {
mesh: meshes.add(Mesh::from(shape::Icosphere::default())),
material: materials.add(Color::rgb(1.0, 1.0, 1.0).into()),
transform: Transform::from_translation(Vec3::new(0.0, 0.0, -5.0)),
..Default::default()
})
.insert(RayCastMesh::<Gizmoable>::default()); // Make this mesh ray cast-able
commands
.spawn_bundle(PbrBundle {
mesh: meshes.add(Mesh::from(shape::Icosphere {
radius : 2.0,
subdivisions: 5
})),
material: materials.add(Color::rgba(1.0, 1.0, 1.0, 0.5).into()),
transform: Transform::from_translation(Vec3::new(1.0, 0.0, -5.0)),
..Default::default()
})
.insert(RayCastMesh::<Gizmoable>::default()); // Make this mesh ray cast-able
commands.spawn_bundle(PointLightBundle {
transform: Transform::from_translation(Vec3::new(4.0, 8.0, 4.0)),
..Default::default()
});
}
This is a great crate so far but what I'm personally missing is a way to see how close a ray was to intersecting when it didn't quite intersect.
The use case I'm imagining is picking meshes with some amount of leniency in precision. For example, in RTS games like Warcraft and Starcraft, you don't have to click exactly on the mesh of a unit to select it. Getting within a certain distance is good enough. This makes the experience for players much better because selecting units is much easier (because you don't have to be super precise).
I don't really see a way to implement this myself downstream, as this crate only provides intersections (do tell me if you think there is a way!). So I guess implementing directly in this crate is the only option.
I thought that perhaps RayCastSource
could be extended with the closest approaches like this (names could use work):
pub struct RayCastSource<T> {
pub cast_method: RayCastMethod,
ray: Option<Ray3d>,
intersections: Vec<(Entity, Intersection)>,
closest: HashMap<Entity, ClosestPoint>,
_marker: PhantomData<T>,
}
where the closest
map holds for each entity, the single closest point from the mesh to the ray. (side note: Why is the type of intersections
not HashMap<Entity, Vec<Intersection>>
?). ClosestPoint
(could be named better) would simply be something along the lines of this:
/// The two points that are the closest points between the ray and the triangle.
struct ClosestPoint {
ray_point: Vec3,
triangle_point: Vec3,
origin: Vec3,
}
impl ClosestPoint {
pub fn new(ray_point: Vec3, triangle_point: Vec3, origin: Vec3) -> Self {
Self {
ray_point,
triangle_point,
origin
}
}
/// The point on the ray that is closest to the triangle.
pub fn ray_point(&self) -> Vec3 {
self.ray_point
}
/// The point on the triangle that is closest to the ray.
pub fn triangle_point(&self) -> Vec3 {
self.triangle_point
}
/// The origin of the ray.
pub fn origin(&self) -> Vec3 {
self.origin
}
}
Then one could easily check if the ray is "close enough" to select it with some leniency.
Do you see another way of implementing this idea of picking leniency? Maybe I'm missing something obvious. Also is this the right crate to implement this kind of thing in, or should this be in a downstream crate (i.e. bevy_mod_picking)? Otherwise, if you think this is a good idea, I wouldn't mind trying my hand at implementing it and doing a PR.
Hi!
In 0.2.2 it seems like no matter what I do, the debug cursor stays one frame after my camera update.
My code is virtually identical to your example - though I noted you changed from PostUpdate to PreUpdate on master. Neither works for me, and I tried running my transform-updating before and after respectively..
Could this be related to that it should be PreUpdate?
Thankful for a fix!
Great lib! Cheers! <3
How do I query entities that are raycasted?
I.e. in minimal example I can see debug cursor on entity, but I also expect it to be in shooting_query.
fn event_gun_shot(
mut commands: Commands,
mut shooting_query: Query<(Entity), With<RayCastSource<MyRaycastSet>>>,
) {
for e in shooting_query.iter_mut() {
// commands.entity(e).despawn();
println!("{:?}", e);
}
}
}
How to do so?
I'm creating an editor in bevy using egui_dock and want to add my own mouse picking. My camera renders to an image which is displayed in a widget. How should I make this work with bevy_mod_raycasting? I have tried scaling the mouse position so that 0,0 is the top left of the viewport and the window resolution is the bottom right of the viewport then sending that scaled mouse position to bevy_mod_raycast, but it's off by some scale factor and I can't work out why. I have checked the numbers and the maths is all correct. Is there something I'm missing or a better way of doing this? Thanks
When raycasting against and indexed mesh I am unable to differentiate between two vertices when both vertices start at the same index.
I'm trying to use triangle_index
to lookup some attribute data but it looks like, for indexed meshes, triangle_index
contains the index after its already been mapped meaning I can't derive the index of the other verts in the tri.
Extra, problem specific, details in case the context is helpful.
The second log is my (incorrectly) derived indices to lookup the attribute with.
The final log is my attempt at calculating the position attribute at the hit location using my (incorrectly) derived indices, the position attribute buffer and the barycentric coordinates. It should be equal to the
Not 100% certain but it seems like the solution is we'll need to return all 3 triangle indices as [usize; 3]
, growing the struct by 2 usize
. I'm happy to PR if this is suitable but I worry I might be missing something as well. Can you let me know your thoughts? And thank you for maintaining this great library :)
When the raycast source isn't a camera, and you mess up where it's pointing, it would be useful to have a debug cursor on that source that points in the direction that it's casting.
Just a small point: The plugin state should not be named 'DefaultPluginState'. I suggest to use 'DefaultRaycastPluginState' instead.
I am using the latest commit of the bevy_editor_pls
crate. When I add bevy_rapier's RapierDebugRenderPlugin
to my app my console gets spammed with the following error.
ERROR raycast: bevy_mod_raycast: Invalid intersection check: `TriangleList` is the only supported `PrimitiveTopology`
I understand the bevy_rapier's debug lines aren't using triangle lists but why does it have to spam the console, can't it just skip it?
Further accelerate ray casting performance with some BVH/space partitioning
Using bounding volumes, place entities into an octree. Maybe instead of a bounding sphere, find the oriented bounding box (OBB). This seems much better suited because we can simply apply the entity's GlobalTransform
to the bounding box, and it will be exactly correct. We only need to recompute the OBB if the mesh changes. Checking if an OBB fits into an octree cell is trivial, we can just test each of the 8 vertices of the OBB to verify that they are within the 6 planes (faces) that bound a cell.
When casting a ray, now there will be two options - get all intersections, or just the topmost. If we are only finding the topmost intersection, we can speed up a ray cast by:
I use the immediate mode to cast rays in a system and it works well if the system is in Update
. However I needed to move this system to PostUpdate
(for reasons unrelated to this mod but related to egui) and, with this change, the ray cast never returns any hit. Is this a known/normal limitation (but undocumented) or is this a bug?
I don't see
use bevy_mod_raycast::prelude::*;
I check git which work on main. But on crate it broke I think. Was using the example to test it normal until it missing prelude.
when I update to new version. this happened and not cast with 'Set'? how to fix it
2d: Circle, Box, Capsule
3d: Sphere, Cube/box, Capsule
Additionally, it would be nice if they could rotate and stuff, and if it would be possible for these shapes to merge. This would allow for a basic collision system to be formed.
bevy_mod_raycast version: 0.3
Bevy version: 0.6
OS & graphic stack: ArchLinux kernel 5.16.2
This is a copy of an upstream bug reported in bevyengine/bevy#4294.
When I update an entity's mesh, either by (1) changing it's Handle<Mesh>
or (2) updating the mesh itself in the Assets<Mesh>
, the plugin doesn't account for the change in mesh value.
This is because bevy never updates the Aabb
of an entity after the initial spawn of a mesh.
As a user of this library, manually update the Aabb
after updating the mesh, by setting the Aabb
component of you rmesh entity to the result of the Mesh::compute_aabb
method.
It looks like the plugin currently always raycasts against the static mesh data, which is typically a T or V pose for rigged meshes. It would be useful if it respected the current bone poses to accurately cast rays against deformed meshes.
Is this even possible with the current implementation? Do I even need the first ray to accomplish this?
I've a attached a little demo of my project to show what I currently have.
https://user-images.githubusercontent.com/34474922/117439102-b2ec7a80-af32-11eb-9686-bd134a2c2921.mp4
Essentially I'd like to shoot a ray from the player at where the screenspace ray intersects, but the ray's direction seems to not alter on the Y-axis as desired.
My code is here: https://github.com/AugustHolst/shooter/blob/974aeb2e239b203508f20e130bc7f8c78f94b5c2/src/main.rs
I instantiate my two ray cast sources on line 28 and 69 and try to set the desired ray's transform on line 121.
Also sorry if this is misuse of the 'Issues' system, it's my first time using it.
thread 'main' panicked at '$state_name<bevy_mod_raycast::bounding::BoundVol> conflicts with a previous access in this query. Shared access cannot coincide with exclusive access.'
I believe this bevy commit introduced the problem: bevyengine/bevy@9657f58
From investigating aevyrie/bevy_mod_picking#341, I've discovered that:
bevy_mod_raycast/src/primitives.rs
Lines 165 to 204 in dbc5ef3
intersects_aabb
returns [NaN, NaN]
because of a divide by 0 when the intersection check is right on the edge of a mesh, thus failing the filter predicate far > 0.0
here:
bevy_mod_raycast/src/immediate.rs
Line 260 in dbc5ef3
It was not apparent to me what the right approach to prevent intersects_aabb
from returning NaN
. It's possible I'm being naive, and there's a more obvious fix, however, this diff seems to solve the symptom of that issue. (I have no idea if this introduces other side effects, but all the tests seem to pass?)
Index: src/immediate.rs
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/src/immediate.rs b/src/immediate.rs
--- a/src/immediate.rs (revision Staged)
+++ b/src/immediate.rs (date 1720831909681)
@@ -257,7 +257,7 @@
};
if should_raycast {
if let Some([near, _]) = intersects_aabb(ray, aabb, &transform.compute_matrix())
- .filter(|[_, far]| *far >= 0.0)
+ .filter(|[_, far]| *far >= 0.0 || far.is_nan())
{
aabb_hits_tx.send((FloatOrd(near), entity)).ok();
}
After 0.9 Bevy have this functionality built-in.
Following our chat in issue #114 I added this code into my app:
fn main() {
let mut app = App::new();
app.configure_schedules(ScheduleBuildSettings {
ambiguity_detection: LogLevel::Warn,
..default()
});
...
}
to detect system ambiguities. It did not help at the time (I am still looking for my mistake causing #114, but thanks a lot for you help) however, after migrating to Bevy 0.14, this now create the following warning:
2024-07-30T17:07:34.811700Z WARN bevy_ecs::schedule::schedule: Schedule First has ambiguities.
2 pairs of systems with conflicting data access have indeterminate execution order. Consider adding `before`, `after`, or `ambiguous_with` relationships between these:
-- update_cursor_ray and event_update_system (in set EventUpdates)
conflict on: bevy_ecs::world::World
-- time_system (in set TimeSystem) and event_update_system (in set EventUpdates)
conflict on: bevy_ecs::world::World
The second pair of systems causing an ambiguity is internal to Bevy and reported in bevyengine/bevy#14524. But the first pair, regarding update_cursor_ray
and event_update_system
is internal to this mod.
As of today Bevy is 2 open issues behind the 0.9 milestone. See: https://github.com/bevyengine/bevy/milestone/7 and this crate should support it asap ;)
Bevy Migration
spawn_bundle()
and use spawn()
insteadI have a scene that I want to serialize/deserialize, containing meshes that I'd like marked with RayCastMesh
. However this doesn't seem to be possible unless RayCastMesh
is registered with the type system. Would it be possible to add this? RayCastSource
is probably another good candidate too.
When a mesh without indices set is in the scene, bevy_mod_raycast
panics at No index matrix found in mesh WeakHandle<Mesh>
.
With the debug plugin it shows an arrow on top and on the sides: https://i.imgur.com/NuWP61u.png (and returns Some) on the bottom it doesn't seem to: https://i.imgur.com/DUajQG4.png
I did some investigating and rotating the mesh (in this case a Bevy cube) even slightly fixes this.
It doesn't seem to be compared to epsilon, but rather if rotation == 0.
For example, things work as expected if I rotate by Quat::from_axis_y(0.000001)
.
This should be reproducible by:
RayCastMethod::CameraScreenSpace(Vec2::zero())
, not really sure what I'm doing tbh)Apply the linked patch that translates the camera on mouse_picking example.
It seems picking does not work anymore.
https://gist.github.com/pjalaber/f43979121cad518cbcb95e960a70678a
Hi! Thank you for maintaining this crate.
I'm using bevy_mod_raycast
in my WASM pet project, and after updating to 0.3 my project stopped working.
panicked at 'The global thread pool has not been initialized.: ThreadPoolBuildError { kind: IOError(Error { kind: Unsupported, message: "operation not supported on this platform" }) }'
I believe rayon
dependency should be optional and enabled only for non-WASM targets.
Problem description
Currently intersection inserted into a separate entity:
Lines 517 to 519 in 5949fc7
But this behavior is not very expected. In a game with menu, users will need to remove such entities manually when exiting a game session. And since the Intersection
component is added to a new entity, users do not have access to information about which entity was intersected (they have to use RayCastSource::intersect_list()
), which makes this component not very useful.
Suggestion
I would suggest to just remove Intersection
component and its system at all since this information already available in RayCastSource
to avoid needless synchronization and duplication.
Hi, are there plans to merge this code into bevy?
Compiling bevy_mod_raycast v0.4.0 (/home/kemper/rustprojects/bevy_mod_raycast)
error[E0432]: unresolved import `bevy::core::FloatOrd`
--> src/lib.rs:8:5
|
8 | core::FloatOrd,
| ^^^^^^^^^^^^^^ no `FloatOrd` in `core`
For more information about this error, try `rustc --explain E0432`.
error: could not compile `bevy_mod_raycast` due to previous error
Using a perspective camera in the mouse_picking example, the debug cursor points towards the camera when the sphere is hovered over. Using an orthographic camera (OrthographicCameraBundle::new_3d()
), the cursor is instead placed on the side facing away from the camera. Here's a screenshot of what this looks like:
This issue also seems to affect bevy_mod_picking.
I'm in the process of implementing a 3d translation gizmo, and looking to use bevy_mod_raycast
to compute the translation delta for when a gizmo axis is dragged.
Could support be added to intersect a mouse position (or any screenspace position) with a fixed set of geometry or meshes, and return the point of intersection? In my case, I would like to intersect screenspace with a plane aligned to the axis being dragged.
Use events to trigger ray casts.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.