I was running commit 3421722 with a SystemRandom seed of 342932250. The first jump has a noticeable seam. Here's an idea for a technique for investigating a better alignment:

• Set up the game to do a 2 second loop around the first jump and try different values for shiftTimestampsSeconds. Maybe such a loop would enable a rapid testing of jump quality and could be a good technique to find the right alignment for a song.

Adding a song requires 3 pieces of information:

• The song ID
• The Spotify analysis file for the song
• ShiftTimestampsSeconds. The number of seconds by which to shift the Spotify analysis timestamps so that they're aligned with those of the Beat Saber song.

Potential solutions

• To add a song, create a JSON file called <song-id.json> in InfiniteBeatSaber/songs. It has these keys:
  • shiftTimestampsSeconds
  • spotifyAnalysis: The value should be the contents of the Spotify analysis file (for example Spotify analysis files, see SpotifyAnalyses/)

Other notes

• Modifying a song's Info.dat changes its hash and, consequently, ID. This is used by Beat Saver when submitting scores to the leaderboard.
• I'm not sure if adding new files to a song's folder would have any unintentional effects. I don't think it affects the song's hash/ID. It would be good to double check.

See also

• #16

The major constraint with Spotify’s audio analysis web API is that the data is only available for Spotify’s audio files. Implications:

• Songs don’t work with Infinite Beat Saber when:
  • They aren’t on Spotify
  • They are on Spotify but the audio file we have is too different from Spotify’s (e.g. OneHope)
• Even when a song is on Spotify and is nearly identical to our audio file, it still takes 10 minutes or so to align the audio files (i.e. determine shiftTimestampsSeconds in RemixableSongs.cs)

For these reasons, it’d be great to have a replacement for Spotify’s audio analysis that we can use with any audio file we want. It would enable Infinite Beat Saber to work with any song and it’d remove some of the manual effort required to add a song to Infinite Beat Saber.

Resources

These resources may be helpful. I found them in spotify/web-api#209 (comment).

• Amen. A tool for algorithmic remixing.
• Lower level feature extractors:
  • librosa
  • essentia
  • meyda

For the list of ideas, see the Idea Pool wiki page.

BeatmapRemixer's job is to generate a remix of the beatmap. This involves joining beatmap sections that weren’t originally joined. This can result in boundaries of beatmap sections that are nearly impossible for the player to hit. Examples:

• Two notes of the same color that appear on sequential beats (or close to it) and are on completely different parts of the screen. For example, imagine trying to hit a red note in the top left corner on one beat and a red note in the bottom right corner on the next beat.
• A note that is blocked by a wall (because the note is on the beat immediately after the wall (or close to it)).
• Two notes of the same color that appear on sequential beats (or close to it) that are in the same spot and have the same cut direction. For example, imagine having to swing down to hit a red note on one beat and having to swing down again to hit a red note on the next beat.
• A note sequence that results in a handclap (the player smashing their controllers together). For example, a red note pointing right and, on the next beat, a blue note pointing left.
• See the “Pattern Best Practices” section of the Beat Saber Modding Group Wiki’s Basic Mapping page for more ideas on the kinds of things BeatmapRemixer should avoid.

Possible solutions:

• Here’s a straightforward solution. When joining two beatmap sections, ensure the boundary doesn’t have any beatmap items that are within X seconds of each other (X to be determined). If there are, delete some of these beatmap items so that there’s at least X seconds of free space at the boundary. This would have to be tested to see how good the resulting beatmap remixes feel.
• A more complicated solution would be to manually implement heuristics for eliminating undesirable beatmap patterns at section boundaries. For example, there could be a heuristic that checks for notes that are immediately behind walls. If there are, the wall is removed.

I was running commit 1513025 with a SystemRandom seed of 348098031.

The song has a bunch of branches near the beginning and it kept taking those branches back to just a few seconds ago so the remix was a little repetitive in the beginning. Not sure if it’s really a problem. Something to think about.

When mod support becomes available for Beat Saber 1.32.0 we'll need to:

• Update the manifest's gameVersion
• Potentially update the versions of other dependencies we're using.
• Test Infinite Beat Saber running in Beat Saber 1.32.0.
• Submit to BeatMods.

This level is currently unsupported because the Beat Saber & Spotify songs are too different. Notes:

• The Spotify song (4:01) is significantly longer than the Beat Saber song (2:52) so there will be differences.
• The 2nd instrument starts at the same timestamp in the Spotify & Beat Saber songs (~3.3).
• But at 2:30 Spotify has a horn solo & Beat Saber has singing.
• So the Spotify & Beat Saber songs might be too different to use Spotify's audio analysis data.

If more time is spent comparing the Beat Saber & Spotify songs, we might discover that the songs are sufficiently similar that we can make use of Spotify's audio analysis data.

These songs have jumps that are noticeable:

• SpinEternally
• EscapeRemix
• CycleHit

Is there an easy fix like using a better value for shiftTimestampsSeconds (in RemixableSongs.cs)? Or does the remixing implementation need to be improved (see #11 Explore improving the remixing implementation)?

These are the BPMs:

• Beat Saber: 128
• Spotify: 192.076 (as seen in the Spotify analysis file)

Spotify’s BPM is 1.5x that of the Beat Saber one. It’s not an integer multiple. Questions:

• Does it make sense that the BPM could be interpreted to be 192.076? Or is it a bug in Spotify’s analysis data?
• When we generate a remix, does this result in noticeable seams when the audio jumps from one beat to another?
  • InfiniteRemix.BeatAlignedObject shifts slices such that every slice starts and ends on a beat (using Beat Saber’s BPM). Is this problematic? Does it result in noticeable seams when the audio jumps from one beat to another? Should we be using Spotify’s BPM instead? But then could we end up with weird results when using Spotify’s BPM to slice the beatmap (which the author designed with Beat Saber’s BPM, not Spotify’s)?

I went with the easiest implementation. It seems like these events set the color & rotation states of the lights.

With the current implementation, I suspect that a song section’s lights can look differently depending on what sections were played before it. So when playing a section in Infinite Beat Saber mode, the lights may not look as intended by the level designer.

I went with the most straightforward implementation. It just omits any chains that don't fully fit into the song slice.

Improvement ideas:

• If a chain doesn't fully fit into a slice, it could be truncated to the portion that does fit into the slice. Would this benefit be worth the implementation cost?

• Whenever we omit a SliderData that represents a chain (i.e. its sliderType is Burst) then we should also omit the item representing the head of the chain (the head of the chain is a NoteData with type: BurstSliderHead).

  For now, we leave the head of the chain alone because it results in the simplest implementation. Leaving it in doesn't result in any undesirable side effects in the game, as far as I know.

  A good question is: is there any noticeable impact of our current implementation of omitting the SliderData but not NoteData, type: BurstSliderHead? If not, perhaps it’s not worth the implementation cost of doing anything fancier than we’re currently doing.

This level is currently unsupported because aligning the Beat Saber & Spotify songs requires more investigation. I tried 3 reference points but they all had quite different shifts between the Beat Saber & Spotify songs:

• Initial drum (shiftTimestampsSeconds: 1.6 secs)
  • Spotify: 0:01.2
  • Beat Saber: 0:02.8
• Vocals (shiftTimestampsSeconds: 2.9 secs)
  • Spotify: 4:01.2
  • Beat Saber: 4:04.1
• Final clap (shiftTimestampsSeconds: 3.7 secs)
  • Spotify: 6:02.9
  • Beat Saber: 6:06.6

Did I make any mistakes?

Is the Spotify version of the song too different from the Beat Saber one for the Spotify audio analysis data to be usable?

Currently Infinite Beat Saber only looks for Spotify analysis data in its own DLL. In addition, we should allow maps to ship with Spotify analysis data. Thoughts:

• The map’s Info.plist can contain custom keys for:
  • The path to the Spotify analysis data file
  • The value of shiftTimestampsSeconds (name taken from RemixableSongs.cs)
• The map can include the Spotify analysis data file somewhere in its directory (like it includes a beatmap file for each difficulty it supports).

Notes

• Modifying a song's Info.dat changes its hash and, consequently, ID. This is used by Beat Saver when submitting scores to the leaderboard.
• I'm not sure if adding new files to a song's folder would have any unintentional effects. I don't think it affects the song's hash/ID. It would be good to double check.
• Maybe it'd be better to have an infiniteBeatSaber.json file in the song's directory if it's the case that this information could be added and updated without affecting the song's hash/ID.

See also

• #19

Some songs have Spotify analysis files with beats that have a length other than 1 beat. Here’s what I’m wondering:

• Why isn’t each beat’s duration 1 beat? Is it a bug in Spotify’s analysis data?
• When we generate a remix, does this result in noticeable seams when the audio jumps from one beat to another?
  • InfiniteRemix.BeatAlignedObject shifts slices such that every slice starts and ends on a beat. Is this problematic? Does it result in noticeable seams when the audio jumps from one beat to another?
• When loading a Spotify analysis file, should we consider filtering out the beats that have a duration that isn’t 1?

RumNBass

Below is the list of beats in RumNBass that have a duration that isn’t 1. It was generated by EvalProgram.GenerateSongCatalogCompatibilityReport.

AngelVoices

Below is the list of beats in AngelVoices that have a duration that isn’t 1. It was generated by EvalProgram.GenerateSongCatalogCompatibilityReport:

I went with the easiest implementation. My understanding is that ColorBoostBeatmapEventData enables the map to switch between two different color schemes. I'm not sure exactly what colors are affected. Maybe just lights & lasers? This video has some info: Boost Colors | Beat Saber Explained - YouTube

Currently, for each map segment, there's no guarantee that we preserve that segment's original boost setting. We should consider preserving it. The problem is that the boost color setting is stateful so a segment receives the boost color setting of the most recent ColorBoostBeatmapEventData event.

This issue is about supporting levels that change their BPM during the song (via the BPMChangeBeatmapEventData event).

This issue affects these levels:

• IWannaBeAMachine

For example, IWannaBeAMachine has these BPM changes via the event BPMChangeBeatmapEventData:

• We could run EvalProgram.GenerateSongCatalogCompatibilityReport or some subset of it.
• Would we launch Beat Saber?
  • Or just link against its DLL?
  • Or test stuff that doesn’t require Beat Saber (e.g. validating Spotify analysis files)?
  • Or load Beat Saber data from disk rather than via runtime APIs?

Jumps in SpinEternally often seem more noticeable than those in other songs. So SpinEternally seems like a good song to study to identify weaknesses in the remix implementation and how to improve it. Things to explore:

• Is shiftTimestampsSeconds (in RemixableSongs.cs) close enough?
• Is QueueBasedAudioRemixer.cs doing a good job of scheduling the audio?
• Is InfiniteRemix.cs producing song slices with bad timestamps (e.g. maybe BeatAlignedObject is flawed and is messing up the timestamps).
• Is InfiniteJukeboxAlgorithm/ doing a good job of identifying similar beats in the song?
  • Maybe its implementation or its parameter values should be tweaked to improve SpinEternally.
  • Maybe it’s being too lenient on SpinEternally and suggesting jumps between beats that aren’t similar enough.
• Are there any issues with the analysis data provided by Spotify's audio analysis web API?

Related

• #12