maxton / libforge Goto Github PK

Maxton thank you for your hard work first of all, There are a few songs after convertion that freeze the game completely also I notice some other songs that cause the game to start and it freezes after 2 seconds in these songs are easy to identify because on the library I hover them and the preview track stops abruptly. I dont know what to do.

This is causing cases where songs show up as another song. EG - Metallica - One conversion has the ShortName One which is the same as U2 - One which has the ShortName one.

Proposal: add a cu_ prefix to the shortname

Ideally LibForge would support making textures in addition to just reading them. Maybe get a NuGet package that does DXT1/DXT5 compression for us? At the very least we should be able to create lossless textures with mipmaps.

Somewhat dependent on #2 and an improved understanding of the unknown header bytes...

The most important thing when it comes to customs. This is already underway, but I will itemize the specific things that are working/not working here.

Unlike previous games, in RB4 and RBVR the midi file is preprocessed (probably to help loading times). The original midi data is kept around, but in order to create an RBMid we have to do nearly everything the previous engines did under the hood ahead of time (this happens in RBMidConverter.cs):

• Overdrive / solo section markers
• Vocal phrase markers, type 1 (partially done but not correctly, esp. for HARM tracks)
  • These are autogenerated somehow and don't match the authored phrases.
  • Black betty percussion break: phrase split is missed
• UNKSTRUCT1 - this usually only has one value in one track for any given song, no clue what it's for.
• Vocal tacets? These structs are created during long breaks in the vocal track. (partially done)
  • These are OK in e.g. bestdayever, but broken in e.g. abc
• Vocal range markers - this might be done already?
• Unknown uints
• Unknown floats
• Other toplevel unknowns
• Vocal phrase markers, type 2
  • These seem to match the authored phrase markers
• Lyrics (one struct for each instrument track, but only containing lyrics for vocal tracks)
• Drum fills (used for fills and BREs)
• Pro drum tom markers (both in the gem track and in their own array)
• Roll/tremolo markers
• Trill markers
• Drum mix markers
• Gems
• HOPOs
• Vocal notes and portamentos (completed, except for the parts dealing with phrases)
• Vocal percussion
• Guitar / bass hand maps
• Guitar / bass hand positions
• Guitar / bass strum maps
• Tempos
• Time signatures
• Beats
• Midi track data
• Preview start/end

RB4-only features

• MarkupSoloNotes - notes found during solo sections on the MARKUP track
• SoloLoops - something to do with freestyle solos, from the MARKUP track
• MarkupChord - freestyle vocal permitted pitches

Not supported in RB4?

• Keys animations (partially done, but there are still some unknowns)

RBSong has all the venue animation information in it (and in newer songs it also has lipsync data embedded, like the milo file used to have). I think we can skip the lipsync embedding and just use external lipsync files, but we need at the very least to be able to generate an “empty” rbsong template with no animations, and work from there.

This might require more reverse engineering work as I don’t believe the current RBSong structure is completely correct.

Right now the Forge Browser has a tab for each object type, which is always visible. It would be more useful to have just one tab for each open object, to be able to quickly compare structures, textures, etc.

• Rename File->Open to File->Open Package, File->Close to File->Close package
• Add File->Open to allow the user to open a file into a tab
• Add File->Close tab to close the selected tab
• User double-clicks a file (or hits enter while selected) on the left to open a tab
• User middle-clicks or hits file->close tab to close a tab

After #7 is done, figure out how to fill in the venue data

Apparently the game REALLY doesn't like shortnames with capital letters? At least that's what I discovered in a recent test. Change shortnames to lowercase.

I am unsure if this is a conversion error, or RB4's engine being more finicky than RB3's and/or the author not following a guideline. I have gathered 3 example that crash on PS4, but work fine on PS3 from the C3 Custom library.

In LibForge the conversion go without error on 2 of them, and throws an error on 1 of them, which I bypassed with a try/catch but this is most likely unrelated, as other songs where I bypassed the same error work fine, and the tick position related to the error is much earlier in the song. (Just included this in case someone wants to try the same song, but gets an conversion error)

I get in-game crash on some converted songs, only when vocals are present (wont crash when guitar only, will crash on Vox + Guitar or Vox alone). The song will keep playing, but visual stop updating and button press dont do anything (other than PS button to close the game).

In the 3 case, the vocal pitch line seems to double back on itself for a small bit, which doesn't happen on the same song in RB3.

For all pictures, on the PS4 side, the game is crashed. On the PS3 side, the song is paused at the same place where the PS4 crash.

Song 1, this is the most obvious one. I only had suspicion before this, but that one is so blatant.
Natural Born Killer - Avenged Sevenfold

PS4

PS3

Song 2
Not Ready to Die - Avenged Sevenfold

PS4

PS3

Song 3 - This one is the one I bypassed the crash in the conversion process, but the midi tick that cause the crash was at 87 second, while the crash happens near the end of the song.
Trashed and Scattered - Avenged Sevenfold

PS4

PS3

Hey Maxton,

Don't know if this is the appropriate place for this but thought it might be useful to share my experience with ForgeTool v0.1.143. I have attempted to convert 316 customs and have had 246 successes. Thats almost a 78% success rate. Thank you so very much for this wonderful tool - myself, my 2 children and my wife have LOVED playing some of these new tracks in RB4.

Of the other 70 customs that did not convert properly: 25 erred during conversion, 41 converted fine and installed as packages fine on my PS4 but crash RB4 when attempting to play and 4 more play fine but are mislabeled as the wrong artists. I created a quick batch file to test conversions of customs quickly and log results to better understand the issues. I thought you might want a copy of these customs and log files as it might help you track down any remaining issues with ForgeTool.

You can access the customs that error, logs and text of successful conversions here:
http://tribewalker.com/forgetool/rock_band_customs_errors_with_forgetool_0.1.143.zip

Again, just wanted to thank you for what you are doing with the PS4 and Rock Band 4 in particular. If this is an inappropriate place to post like this, please delete at your discretion. Merry Christmas and have a happy new year!

• Elrondolio

There's a new texture format in the version of forge used by RB4-PS4-v2.14

Right now we are guessing the texture format based on the dimensions versus the data size. This works to distinguish DXT3/5 from DXT1 and lossless, but there seem to be a whole lot more texture formats supported by the engine.

The format is definitely being saved in one of the header bytes, so all you have to do is figure out which one to look at.

For simplicity the texture converter only supports 512x512 and 256x256 textures as both can just have their image data copied over to a 256x256 Forge texture, but it would be a good idea to support other texture sizes.

First, make invalid texture a warning and just leave the texture out of the custom. Then fix this.

There should definitely be a way to do this from the UI not just the command line. I don't know how it would look. At least a menu option, File->Convert Custom or something for now

This could allow people to interactively review the song data before making a package.

Hello,

I have compiled the most recent version of this tool. It states it supports png_ps4 to png but with Amplitude 2016 PS4 it errors on attempt to open or convert to PNG.

Is PS4 Amp2016 not supported? Do you have any tips or advice.

Error shown in command window:

G:\Amplitude\Assest_Tools\3_ForgeTool>ForgeTool.exe tex2png warning.png_ps4 warning1.png

Unhandled Exception: System.Exception: Invalid array size FFFFFFFF encountered at CC8. File is corrupt or not understood.
at LibForge.Util.BinReader.FixedArr[T](Func1 constructor, UInt32 size) in C:\forge\libforge\LibForge\LibForge\LibForge\Util\BinReader.cs:line 81 at LibForge.Util.BinReader.Arr[T](Func1 constructor, UInt32 maxSize) in C:\forge\libforge\LibForge\LibForge\LibForge\Util\BinReader.cs:line 93
at LibForge.Texture.TextureReader.Read() in C:\forge\libforge\LibForge\LibForge\LibForge\Texture\TextureReader.cs:line 37
at LibForge.Texture.TextureReader.ReadStream(Stream s) in C:\forge\libforge\LibForge\LibForge\LibForge\Texture\TextureReader.cs:line 14
at ForgeTool.Program.<>c.

G:\Amplitude\Assest_Tools\3_ForgeTool>

Currently only the point and face data are being loaded from mesh files. It would be great to also have UV coordinates and whatever other data we are skipping (vertex colors maybe?).

• UV coordinates
• Other stuff

After hitting the entitlement limit on PS4 for RB4 (2500, so 2500 pkg limit), decided to try again and make multi-song pkg. After testing, it seems the game has a size limit on what it can load via a single pkg file. That limit is around 100mb. (Around 100mb of CON file before conversion seems to work fine, even if the resulting pkg is a bit bigger)

If you go over the limit, the game will get stuck forever on "finding additional content".

If you manage to get just one song above the size limit, the game will load the content, but won't be able to load the song to play, the data seems to be out of range of what the game can read.

Ex: Make a 2 pack with a 90mb song and a 70 mb song, it will find the songs, let you play whichever was packed first, but the 2nd one will not play the preview and will crash when you start it.

Not quite an issue, but I couldn't find another spot to report this.

In the drum track don't throw an exception when encountering unknown notes

Help please
Thanks

Found in http://customscreators.com/index.php?/page/index.html/_/money-money-money-r252
Reported by Synderella

There's some bad stuff that authors can do which shouldn't make the conversion fail because RB3 tolerates it, but there might as well be a mechanism to alert the user to this and other non-fatal stuff.

hello i want to try but it's impossible to fine .exe files.

Found in http://customscreators.com/index.php?/page/index.html/_/night-drive-r5826
Reported by Synderella

Might be a good idea to leave the LibForge version somewhere in the generated PKGs

First of all thank you for awesome tool. I have been using LibForge for converting RB3 files to RB4.
I encountered a problem when im converting CON to PKG with multiple songs, LibForge crashing and showing this error: System.Exception: Element at index 0 is not an integer.

At first i thought it had problem with song names but i found out few songs causing this error. I found the bad songs by pain stackingly checking each one of them on CON to PKG conversion.

My question is, is there any way to findout which song is causing the problem in batch?. i mean is there any log file for check?.
Thank you very much.

I thought I had fixed this one, but it turns out that I only fixed it in the GUI because of copy-pasted code. Avoid copy-pasting code!

This bug only shows up on songs that have short ids

Found in http://customscreators.com/index.php?/page/index.html/_/whyd-you-only-call-me-r15343
Reported by Synderella

We should be able to load lipsync data from an RB3 custom and spit out an RB4 lipsync.

Any Multitrack song (both Guitar Hero conversion and Original C3 release), the sound track are messed up.

Playing drum:
Missing the Bass Pedal note mutes the whole drum track, and it won't come back until you hit another bass pedal note.

Missing Red/Snare mutes the Bass track.

Missing Toms or Cymbals (Yellow, Blue, Green), mutes the guitar track.

This is using single converted song via ForgeToolGUI-0.1.163

Tested with a bunch of Guitar Hero: Metallica conversion and one of the latest Multi-Track C3 release (Thin Lizzy - Jailbreak)

While this is not too bad once the drum kicks in and you don't miss too many notes, intro of song that have no drums are mostly silent.

Edit: Tried so many way to mess with the mogg.dta track and shuffling them around but to no avail. While I was able to get the guitar to mute some other parts (drum), I wasn't able to come up with a combination that worked properly. And if you omit a track (let say, cut the drum short to (0 1 2 3) instead of (0 1 2 3 4 5), the songs will start, but hang a few seconds in. The audio preview will reflect the change and omit some sound channel, but will also hang after a few seconds.

For making my large clone hero setlist playable in rock band 4/3/2, I have used the program Onyx (Onyxite Customs) to convert the clone hero song folders into RB3CON format for use in RB3 and to convert to RB4 .pkgs via libforge. In RB3, these songs show correct timings (same as clone hero) when activating overdrive, but when those songs are converted to RB4, they end up having the overdrive chip down in odd increments and causing the time in overdrive to be much shorter/nearly halved compared to in RB3.

this is the same overdrive phrases on both games
RB4:

RB3:

https://drive.google.com/file/d/1iJAh0wtf_ZhDdic0s04zftuEJGdF8L7o/view?usp=sharing (con file to help debug it)

Strings and symbols in RBSongs and DTBs are stored as what some call "pascal strings" - that is, first the length of the string is written as an unsigned int, then the string itself is written. This is in contrast to "C strings" which do not have the length, but rather are terminated with a single null byte after the string.

However, this is not entirely true, evidently: sometimes, strings in RBSongs are both length-prefixed AND null-terminated! Through trial and error I devised some code that seems to work for reading these strings and skipping the null-terminator when needed (https://github.com/maxton/LibForge/blob/master/LibForge/LibForge/RBSong/RBSongReader.cs#L164) but the code is basically "guessing" if the next byte is a null terminator or not.

I think the only way to sort this out is to find the RBSong reading routines (possibly the writing ones, in case they are in the game binary and not just the editor) in either RBVR's or RB4's binary and see what the actual logic behind this is. What a pain!

When you load RBMids, RBSongs, etc into a tree view in ForgeToolGUI, it's really slow because it's adding all the nodes at once, even though you probably aren't going to look at all of them.

The object inspector should be refactored to only load nodes into the tree view when a node is expanded.

The cause of errors on many customs.

e.g. "Back in Black - ACDC", "Hells Bells - ACDC", "Black Smoke Rising - Greta Van Fleet" to name a few