Using https://xirsys.com/ is kinda shit for two reasons:

• They are another point of potential failure
• Heavily limited bandwith

Self hosting the TURN server seems not only feasible but also like the much better chose. I should give that a try at some point. Should probably use https://github.com/coturn/coturn for that.

It would be very useful to see network stats ingame like the ping of the players. This could be achieved with a toggle like TAB.

In general, the dodge will trigger a desync as it's rotation does not always reset to the same value (zero).

Split effects, create new module and have a file for each effect.

thread 'main' panicked at 'Send failed: TrySendError { kind: Disconnected }', /home/rancic/.cargo/registry/src/index.crates.io-6f17d22bba15001f/matchbox_socket-0.7.0/src/webrtc_socket/socket.rs:351:48
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Encountered a panic in exclusive system `bevy_ggrs::ggrs_stage::GgrsStage<ace_of_the_heavens::network::GgrsConfig>::run`!
Encountered a panic in system `bevy_app::main_schedule::Main::run_main`!

If we are in game and the server we are connected to shuts down we get the panic above. Not sure what the best solution is, I suppose to throw the player back into the networking screen?

It's weird because I thought the server is only used to establish the first peer to peer connection and is not needed after that point. However that is obviously not the case.

When play testing I noticed that the player trail was not synced to the player. Not in a desync way (though that may or may not be the cause), but in the sense that the trail was just randomly on the map and not on the players wings. Unfortunately this only happened ones and I don't have footage of it.

I may create some sort of milestones or "ideal states" of the game to get a better view on what I should and could focus on.

Supporting Android would be very cool for letting other people play anywhere. It would also make playing with more people possible. I am a little concerned with how much I would need to adjust UI and controls to make it work but I would only support Android once the game is in a fairly advanced state anyways.

To begin with, it might not even be possible to use ggrs in Android builds. I will have to test to see that.

For some reason every once in a while the RNG doesn't match. One instance were this happend I checked the seeds, however both players had the seed 1697191983. I unfortunately didn't see the seed on the connecting screen (and I also didn't record at the start). I doubt that this is a local issue.

For some reason, when one player is at least doing this: Forward, Sideways, Rocket (maybe rocket is enough) and then dying (by flying in a wall for instance), then there is a chance that if that round was the last round (match point), then one of the peers could instantly restart, without any waiting for a rematch (note that perhaps both peers this could happen to perhaps).

Support multiple rounds that can be played at the same time.

On Linux

To set a ping of 300ms, use

sudo tc qdisc add dev enp3s0 root netem delay 300ms

to reset it run

sudo tc qdisc del dev enp3s0 root

This uses iproute2, make sure that package is installed.

If one peer already played some rounds and then gets back to the Main Menu and plays again with some peer, the music will be desynced between peers. Is this even a problem though? I don't quite know why this is, as the music should be synced, at least I thought that.

Split functions into multiple smaller ones.

So I implemented a very boilerplate heavy solution for sending seeds between peers, which currently still doesn't work, but then I realized that you could just use the start time of the game (GameState::InGame) as a seed. Given that the game must be deterministic, the game should start at the same second. There is one catch to this however: The game could start at the end/start of a second, for instance

Peer1: 15.990
Peer2: 16.020

If we just take the seconds we will get different seeds, and that will result in instant desyncs. However we can simply round to the nearest integer and the problem is solved. Unless the peers don't have a enomers delay this will work pretty well. Even in that case, we could simply disregard the last digit of the second and round to 10 seconds, this will make sure that even peers with far distances can at least have the same seed (even if the playing experience will be less then ideal).

game_logic.rs is doing a lot right now.

Also move ui::MAX_SCORE into game_logic.

It would be very nice to be able to simple press a button and then see all the collision boxes. I think this could be achieved by rendering sprites for all collision boxes on a separate camera and then simply overlaying the collision debug camera on the main one.

I don't know how to properly clear the world of all entities (except Window Components) and reset all custom Resource types. I am doing it manually, which seems to be the smallest Evil but is obviously not maintainable.

Display some general info of the game, how to move and play, objectives, obstacles etc.

Not sure what was the cause, but there definitely are still desyncs happening. When play testing I managed to get BLUE to win on one peer and ORANGE to win on the other. I suspect this to be an issue with the rockets, but it's hard to tell. I will try to record both sessions next time, this proofed to be very effective.

Currently, if I forget to add add_rollback to the spawn command then the sound will play but will not be rolled back. This could be super frustrating to debug. So I should check every sound effect if it has the rollback component. Not sure how to do this though. Perhaps I can use With<>? Though I first have to figure out what the rollback component is (should be fairly easy to figure out by just turtling down the add_rollback command).

Adding Plugins that add systems to the GgrsSchedule does NOT work.
The fix that I came up with here is to first add the DefaultPlugin,
then add the GgrsPlugin (this adds the GgrsSchedule and all
the things necessary), and finally we add our own plugins like
GameUiPlugin that will potentially add systems to the GgrsSchedule.

As far as I am concerned this shouldn't really cause any issues.
In fact it seems like you could add the GgrsPlugin before
the DefaultPlugin. Though I decided against it because it seems
like it might cause issues.

The issue is the unwrap here. When the peer we are sending to closed their connection we panic at this unwrap. This however seems to be fixed in the most recent version, see here. I will leave this like that for now, this is more like a memo to check this ones a new release of matchbox is out.

thread 'IO Task Pool (1)' panicked at 'called  on an  value: MessagingError(TrySendError { kind: Disconnected })',
/home/rancic/.cargo/registry/src/index.crates.io-6f17d22bba15001f/matchbox_socket-0.7.0/src/webrtc_socket/mod.rs:217:51note:

'main' panicked at 'Send failed: TrySendError { kind: Disconnected }',
/home/rancic/.cargo/registry/src/index.crates.io-6f17d22bba15001f/matchbox_socket-0.7.0/src/webrtc_socket/socket.rs:351:48
Encountered a panic in exclusive system !Encountered a panic in system !

While playtesting I had huge syncing errors, to the point where one player was full on dead while he was alive in the other session. The thing is, even IF I forgot a rollback etc, there is essentially zero latency, and the fact that I happend to get these errors the very first time I tested for it is also very low.

I believe this error is actually because of scaling up the screen. I realized some issues with the positioning when rescaling and switching screens. Though it's hard to tell for sure.

I must test this further, though not on a single machine.

The aim of this report is to figure out where and how desyncs happen.
To this end I am first going to see what the actual latest release build
is in terms of non desyncs (if one even exists!).

To start, I check the commit 3d2d49b,
it desyncs when played with a ping of 50ms. The issue as far as I can tell,
is that if one player spams Left-Right (triggering a bunch of rollbacks because
the other peer is constantly mispredicting), the bullets get rolledback
(as they should) but if they despawn and get rolledback after that,
they will NOT get respawned (EDIT: I believe the bulelts are working correctly, see below comments, and the actual issue I am refering to here is probably #20). This is exactly what I observed
with the rockets on the current main (c631d5b).

The desync detection issue of comparing the wrong frames has been resolved in the main branch. Switching might break things but I should at least try. It would be hugely beneficial to have proper desync detection.

Currently, we update every single DebugTransform manually whenever the corresponding Transform is changed.
We could instead have one single system that updates all DebugTransforms.
The DebugTransforms would need to store a reference to their Transform, but that should be doable.
If nothing else you could maybe reference the Entity of the Transform.

Having Singleplayer would be quite handy, and if I were to put the game on Steam pretty much a requirement. I would also need Local multiplayer and controller support and more, but we will see.

Some promising resources:

• https://dev.to/akshayballal/deep-neural-network-from-scratch-in-rust-part-1-basics-of-neural-network-1pgg
• https://www.youtube.com/watch?v=FI-8L-hobDY

I will 100% have to study and research this topic first before even trying anything on my own.

This is an issue of how we "restart" rounds and should be restructred anyways. Still something to keep in mind. I assume this happens because the sprite gets disabled the same frame the round restarts.

Should UI input be handled in ui module or somewhere separate?

Current idea
Have the UI module consistent of different parts: visual and logic or something similar.

Alternative
Have many small modules like game_over, round_start and so on and have logic split there (you could even split them like visual, logic in there too).

The current maps aren't ideal for the game. I do like some of them, but others are really bad. I also need more data on this and get some feedback from other players. It's probably best to design a bunch of different ones and then playtest them with other players.

Things like

pub struct Score(pub usize, pub usize, pub Option<usize>);

are really confusing and shouldn't be used. Instead use

pub struct BgmStage {
    stage: usize,
}

Currently, the spawn_players spawns ALL of the player related entities. This is really bad for many reasons. The way I want to restructure this is to spawn the health bar, reload bar, etc. separately. This can be done easily because we use handles to identify which player the corresponding entity belongs to. For things that need to be set on spawn like the position we can simply access the constats.

Stress test the following

• Number of Bullets (this will test both moving objects and collision detection system)
• Number of peers that can connect until the game starts to get unplayable
• Bullet Damage, how many bullets can hit the player at the same time without causing any issues
• Max ping (input delay) until game becomes unplayable

This is most likely the source of #23 and #17. #20 is also partially affected.

The GameState and RollbackState are extremely incompatible.
Or rather, the way I am using GameState is incompatible.
For instance, the RNG and UI are using GameState, but these must
be rollback-able, otherwise the RNG breaks and the UI will show the wrong things.

I will most likely have to restructure the whole thing and figure how to make this work.
This should be one of the highest priorities.

If the player dies and we rollback his death then the sprite is not respawned (the player is effectively invisible). I believe this also happens to the healthbar, not sure about reloadbar.

It's very bad that I am significantly better then my peers when playing. That simply results from the fact that I have made the game and therefore I played it a lot and know the ins and outs. Still, it does mean that I absolutely demolish new players, and even ones that have a bit of experience don't stand a chance.

Some ideas for nerfs:

• reduced bullet damage
• increased rocket timer duration
• increased dodge timer duration
• decreased invicibility time during dodge
• reduced health
• reduced max speed?

overall just some number changes which should be very easy to implement, but that could already be enough to make up for the differnce in skill. This will eventually not be an issue anymore, but it should ease in the experience for my peers. I am not quite sure how exactly to implement this though. Given that we only have numbers we need to change, it might be tempting to use something like a console to set my "level" of nerf, to dynamically adjust it (if my peer is catching up in skill for instance).

Not quite sure how to reproduce, but I think one player has to be dead and disconnected and then the other player also dies. The game will try to respawn both in that case but only one is connected, at least I think that's what's happening. One solution might be to go into a different state altogether when one player disconnects, like maybe go back to matchmaking or something like that

thread 'Compute Task Pool (0)' panicked at 'assertion failed: `(left == right)`
  left: `33v1`,
 right: `6v0`: Malformed hierarchy. This probably means that your hierarchy has been improperly maintained, or contains a cycle', /home/rancic/.cargo/registry/src/index.crates.io-6f17d22bba15001f/bevy_render-0.11.2/src/view/visibility/mod.rs:331:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
thread '<unnamed>thread 'Compute Task Pool (1)' panicked at 'assertion failed: `(left == right)`
  left: `33v2`,
 right: `11v0`: Malformed hierarchy. This probably means that your hierarchy has been improperly maintained, or contains a cycle' panicked at '', /home/rancic/.cargo/registry/src/index.crates.io-6f17d22bba15001f/bevy_transform-0.11.2/src/systems.rs:assertion failed: `(left == right)`
  left: `33v1`,
 right: `6v0`: Malformed hierarchy. This probably means that your hierarchy has been improperly maintained, or contains a cycle160:9
', /home/rancic/.cargo/registry/src/index.crates.io-6f17d22bba15001f/bevy_transform-0.11.2/src/systems.rs:160:9
Encountered a panic in system `bevy_render::view::visibility::visibility_propagate_system`!
Encountered a panic in system `bevy_transform::systems::propagate_transforms`!
Encountered a panic in system `bevy_app::main_schedule::Main::run_main`!

When shooting rockets with a high ping (100ms is enough) and spamming Left-Right you will get infinite desyncs that don't reset (even after a rematch). I suspect something with either despawn of rocket, spawn of explosion or something with the timer.

When the player dodges and his x axis gets rotated the spawned projectiles have the same rotation (which results in weird looking projectiles).

The rematch state in the game over screen can desync. I was able to trigger this twice in a row, but then not anymore. It seemed like it happend around the third or forth when rematching. The issue is that one player sends the rematch, and as soon as he does the other player is already playing, without even having pressed the rematch button.

Currently, the connecting screen just waits an arbitrary amount (one second) to fetch the seed from our peer. This is bad in many ways, and the proper way to do this is to use ACK's or Acknowledgments to broadcast to our peer(s) that we received the seed and are ready to go in-game.

Only issue is that I have no clue how to do that 🙃

The Score in game_logic.rs is NOT getting rolled back (because it is not registered in the GgrsPlugin). The UI is also not getting rolled back, resulting in UI telling that a player won, even though we rolled back the win.

• Decrease bullet damage a bit
• Visual (and perhaps auditory) feedback for when dodge is ready
• Visual (and perhaps auditory) feedback for when rocket is ready
• Map Redesigns (also add more)

The reason I didn't use that right away was because the rotation gave me nightmares. However this post explained to just use

transform.rotation = parent_transform.rotation.inverse();