I've been trying convert TF2 sprays into bmp files, but the images end up all warped and I'm unable to pinpoint the issue.

This is the original image:

And this is what I end up with:

Here's a snipet of my code:

Perhaps I misinterpreted how the pixels in the byte array are organized? I couldn't find much information about it.
The spray is in the Dxt1 format.

When writing a wad file the texture lumps are written without padding them to a 4 byte boundary like qlumpy does: https://github.com/ValveSoftware/halflife/blob/c7240b965743a53a29491dd49320c88eecf6257b/utils/qlumpy/qlumpy.c#L197-L201

The resulting wad files can't be read properly by Wally because the workaround for an old Wally bug happens to trigger here: https://github.com/Ty-Matthews-VisualStudio/Wally/blob/e8394c574e8efd63da3b56d2ea8f27e13ff3af32/Source/Wally/WADList.cpp#L104-L121

Additionally the padding byte for lump info is written as the value 2 instead of 0 which shows up when doing a binary diff.

There can be dictionaries in properties:
color(color255) { enabled={ variable="colormode" value="0" } }
fade_size_start(float) { group="Render" min="0.0" max="1.0" enabled={ variable="directlight" values=["2", "3"] } }

Adding this code after Source 2 metadata for properties: works:

                /* Source 2 metadata for properties:
                color(color255) { enabled={ variable="colormode" value="0" } } : "Color" : "255 255 255"
                luminaire_shape(choices) { group="Luminaire" }: "Area Light" : "0" : "Shape of the area light"
                 */
                if (it.Current?.Is(TokenType.Symbol, Symbols.OpenBrace) == true)
                {
                    var dict = new GameDataDictionary(string.Empty);
                    ParseGameDataDictionary(it, dict);
                }

However ParseGameDataDictionary also needs to be updated to support arrays: values=["2", "3"].

Arrays can also appear in class metadata:

	metadata
	{
		model_archetypes = [ "generic_actor_model" ]
	}

There's also property type that has a slash and ampersand in it which fails to parse:
light_style( vdata_choice:scripts/light_styles.vdata )
item_1( vdata_choice:scripts/grenades.vdata&scripts/misc.vdata&scripts/npc_abilities.vdata )

I don't know if you want to add / and & in Tokeniser.TokenName.

There's three new variable types:

        VDataChoice, // Source 2
        SubclassChoice, // Source 2
        NPCAbilityName, // Source 2
        PathNodeClass, // Source 2
        Color255Alpha, // Source 2
        ParticleCfg, // Source 2
        ModelClothVertexMap, // Source 2

Files:
https://github.com/SteamDatabase/GameTracking-Dota2/blob/master/game/core/lights2.fgd
https://github.com/SteamDatabase/GameTracking-Dota2/blob/master/game/core/ai_defaultnpc.fgd

There's also @VData and @VDataDerived commands here: https://github.com/SteamDatabase/GameTracking-Dota2/blob/master/game/core/vdata_base.fgd

There's @helpinfo command:

@helpinfo( "ai_basenpc", "tools/help/fgd/ai_basenpc.txt" )

The easiest way to reproduce is to try and parse steam\userdata\<user_id>\7\remote\sharedconfig.vdf, which has such property at UserLocalConfigStore>Software>Valve>Steam>FriendsUI>FriendsUIJSON.

Here's an excerpt:

"FriendsUI"
{
	"FriendsUIJSON"		"{\"bNotifications_ShowIngame\":false,\"bNotifications_ShowOnline\":false,\"bNotifications_ShowMessage\":true,\"bNotifications_EventsAndAnnouncements\":true,\"bSounds_PlayIngame\":false,\"bSounds_PlayOnline\":false,\"bSounds_PlayMessage\":true,\"bSounds_EventsAndAnnouncements\":false,\"bAlwaysNewChatWindow\":false,\"bForceAlphabeticFriendSorting\":false,\"nChatFlashMode\":1,\"bRememberOpenChats\":false,\"bCompactQuickAccess\":true,\"bCompactFriendsList\":true,\"bNotifications_ShowChatRoomNotification\":false,\"bSounds_PlayChatRoomNotification\":false,\"bHideOfflineFriendsInTagGroups\":false,\"bHideCategorizedFriends\":true,\"bCategorizeInGameFriendsByGame\":false,\"nChatFontSize\":2,\"b24HourClock\":true,\"bDoNotDisturbMode\":true,\"bDisableEmbedInlining\":false,\"bSignIntoFriends\":true,\"bDisableSpellcheck\":false,\"bDisableRoomEffects\":false,\"bAnimatedAvatars\":true,\"featuresEnabled\":{\"DoNotDisturb\":1,\"LoaderWindowSynchronization\":1,\"NonFriendMessageHandling\":1,\"NewVoiceHotKeyState\":1,\"PersonaNotifications\":1,\"ServerVirtualizedMemberLists\":1,\"SteamworksChatAPI\":1},\"bAutoSignIntoFriends\":true,\"bShowTimeInChatLogCheck\":true,\"bSingleWindowMode\":false}"
}

Hello it's me again :D

When parsing valve formats as output by TrenchBroom (e.g. "Quake 2 (Valve)", "Quake 3 (Valve)"), if any surface has a surface or content flag the QuakeMapFormat will crash with the following error:

System.Exception: Parsing error (line 10, column 83): Unexpected token value Whitespace( )
   at Sledge.Formats.Tokens.TokenParsing.Expect(IEnumerator`1 it, TokenType type, Predicate`1 valueChecker) in E:\Projects\C#\sledge-formats\Sledge.Formats\Tokens\TokenParsing.cs:line 29
   at Sledge.Formats.Map.Formats.QuakeMapFormat.ReadSolid(IEnumerator`1 it) in E:\Projects\C#\sledge-formats\Sledge.Formats.Map\Formats\QuakeMapFormat.cs:line 180
   at Sledge.Formats.Map.Formats.QuakeMapFormat.ReadEntity(IEnumerator`1 it) in E:\Projects\C#\sledge-formats\Sledge.Formats.Map\Formats\QuakeMapFormat.cs:line 155
   at Sledge.Formats.Map.Formats.QuakeMapFormat.Read(Stream stream) in E:\Projects\C#\sledge-formats\Sledge.Formats.Map\Formats\QuakeMapFormat.cs:line 111

I assume this is because it's expecting a new line but got the contentflags number instead. Here is a map file that can reproduce this error:

// Format: Quake2 (Valve)
// entity 0
{
"mapversion" "220"
"classname" "worldspawn"
"_tb_textures" "textures"
// brush 0
{
( -64 -64 -16 ) ( -64 -63 -16 ) ( -64 -64 -15 ) __TB_empty [ 0 -1 0 0 ] [ 0 0 -1 0 ] 0 1 1 1 0 0
( -64 -64 -16 ) ( -64 -64 -15 ) ( -63 -64 -16 ) __TB_empty [ 1 0 0 0 ] [ 0 0 -1 0 ] 0 1 1 1 0 0
( -64 -64 -16 ) ( -63 -64 -16 ) ( -64 -63 -16 ) __TB_empty [ -1 0 0 0 ] [ 0 -1 0 0 ] 0 1 1 1 0 0
( 64 64 16 ) ( 64 65 16 ) ( 65 64 16 ) __TB_empty [ 1 0 0 0 ] [ 0 -1 0 0 ] 0 1 1 1 0 0
( 64 64 16 ) ( 65 64 16 ) ( 64 64 17 ) __TB_empty [ -1 0 0 0 ] [ 0 0 -1 0 ] 0 1 1 1 0 0
( 64 64 16 ) ( 64 64 17 ) ( 64 65 16 ) __TB_empty [ 0 1 0 0 ] [ 0 0 -1 0 ] 0 1 1 1 0 0
}
}

I think the "Valve" format is the same as the "Worldcraft" format but with the contentflags surfaceflags value values tacked on at the end like in "idTech3" format.

// Worldcraft:
( x y z ) ( x y z ) ( x y z ) texturename [ ux uy uz xshift ] [ vx vy vz yshift ] rotation xscale yscale
// Valve:
( x y z ) ( x y z ) ( x y z ) texturename [ ux uy uz xshift ] [ vx vy vz yshift ] rotation xscale yscale contentflags surfaceflags value

Some of the WADs I'm using use the old standard of prefixing certain textures with special characters to denote their type. One of these characters is {. The .map parser incorrectly drops this symbol and reads a face with texture name {stone_blck2_f1 as stone_blck2_f1.

Currently there are multiple different ways to create an object:

• SerialisedObject: SerialisedObjectFormatter.Deserialize(Stream)
• Liblist: Liblist(Stream)
• BspFile: BspFile(Stream)
• MapFile: WorldcraftRmfFormat.Read(Stream)
• IPackage: VpkPackage(string)
• VtfFile: VtfFile(Stream stream)
• WadFile: WadFile(Stream stream, bool = true)

Each way can be justified individually, but together they're all just inconsistent. At the very least, provide a Constructor(Stream) option for all type. VPK and possibly MDL and FGD (once implemented) would need to be a FileStream at least, as those need to hit the disk to load extra data. Investigate supporting a virtual file system interface to facilitate non-traditional loading environments for formats with dependencies.

The latest version of JACK has updated the JMF format with support for viewport background images.

https://twhl.info/wiki/page/Specification%3A_JMF#comment-105812

https://github.com/pwitvoet/mess/blob/1cbb161491c7e74581798aca8fa371e4505aae42/MESS/Formats/JmfFormat.cs#L31-L45

Wad file lumps currently do not allow default construction:
https://github.com/LogicAndTrick/sledge-formats/blob/b9b5f956a694a5faa9a307afb364c090573c2d05/Sledge.Formats.Texture/Wad/Lumps/MipTextureLump.cs

When creating new wad files this requires the lumps to be re-implemented to support this:

internal sealed class JobMipTextureLump : MipTexture, ILump
{
    public LumpType Type => LumpType.MipTexture;

    public int Write(BinaryWriter bw)
    {
        var pos = bw.BaseStream.Position;
        Write(bw, true, this);
        return (int)(bw.BaseStream.Position - pos);
    }
}

 WadFile wadFile = new(WadVersion.Wad3);

 foreach (var texture in bspFile.Textures.Where(t => t.NumMips > 0))
 {
     JobMipTextureLump lump = new()
     {
         Name = texture.Name,
         Width = texture.Width,
         Height = texture.Height,
         NumMips = texture.NumMips,
         MipData = texture.MipData,
         Palette = texture.Palette
     };

     wadFile.AddLump(texture.Name, lump);
 }

 using var stream = File.Open(wadFileName, FileMode.Create);

 wadFile.Write(stream);

Adding a public default constructor to the lumps should suffice.

When replacing the SortedProperties list on a Sledge.Formats.Map.Objects.Entity object the Properties dictionary still wraps the original list instead of the new list.

Either the setter needs to be removed or the setter should recreate the wrapper as well (which may leave dangling references to the old wrapper).

When a BSP file's entities lump is written it writes the classname and model keys twice. This is because it writes both all keyvalues and these keyvalues explicitly:

Both keyvalues are added to the list of keyvalues on load:

It should use one or the other to save these keyvalues. It might be better to remove the explicit keyvalue properties for classname and model and rely solely on the keyvalues dictionary, since users will likely have their own wrapper around this data.

I'm trying to make a nextbot generator but I couldn't figure out how to convert a png to vtf. I decided to post this here so that this can maybe could get added.

Very nice and easy to use, would like to see mdl supported, so far what I need is to read mdl infos, like material and texture names, so I can find them for this mdl automatically, don't know how to do that myself though.

When the Textures lump deserializes textures invalid offsets are ignored, but no texture is added:

Maps that have invalid offsets will thus have Texinfo objects that refer to incorrect textures and to textures with an index exceeding the size of the Textures lump.

The engine handles this differently. It allocates a list of nummiptex textures, zero initializes all of them and loads them one by one. Textures with invalid offsets are skipped, leaving the texture object as all zeroes.

See https://github.com/id-Software/Quake/blob/bf4ac424ce754894ac8f1dae6a3981954bc9852d/WinQuake/gl_model.c#L358-L364

The compiler initializes the offsets here:
https://github.com/ValveSoftware/halflife/blob/c7240b965743a53a29491dd49320c88eecf6257b/utils/qcsg/textures.c#L353-L368

The correct way to handle this case here would be to create a MipTexture object with an empty name and defaulted properties.

This is known to happen with cz_recoil, cz_stadium_cz and possibly Sven Co-op's grunts2.

When the Entities lump writes the entity data string to the stream it writes the string but it doesn't append a null terminator:

In most cases there will be a null byte right after the entities lump, but in some cases such as c1a2c this will not be the case. The engine will continue to parse data past the entity data string and will fail to read a { character, causing a Host_Error: ED_LoadFromFile: found � when expecting { error.

Although the engine does copy the string before parsing it, the lack of null terminator in the file itself seems to consistently affect parsing as well.

Comparing the bsp file before and after modification shows the entities lump is shorter by one byte.

Adding this after the string is written should fix this problem:

bw.Write((byte)0);

@PointClass
	base( Targetname, Parentname, CanBeClientOnly, Light2Inputs )
	omnilight()
	metadata
	{
		entity_tool_name = "Omni Light"
		entity_tool_tip = "Omni directional light source"
		entity_tool_group = "Lighting"
		entity_tool_sortname = "lights2"
		default_pitch = 90.0
	}
	iconsprite
	{
		image = "materials/editor/light_omni.vmat"
		tintKey = "color"
	}
= light_omni2 : "A sphere or tube shaped light"

In this example, class.Behaviour for iconsprite has no values, and the actual values are in class.Dictionaries, but as far as I can tell there is no way to know which dictionary to access for this behaviour since I don't know which dictionary index it is.

Currently it's using a lot of line substring stuff which isn't very nice

CS2 released their workshop tools, and the fgd has new stuff that does not parse:

@OverrideClass
	metadata
	{
		class_game_keys =
		[
			{ key = "clientSideEntity" value = 1 }
		]
	}
= light_environment
[
]

It trips up on the class_game_keys value:

Parsing error (line 6, column 4): Unexpected token value Symbol({)

https://github.com/SteamDatabase/GameTracking-CSGO/blob/6526f854b5b94d1c3103b2e77eaff0a7f2b968c8/game/csgo/csgo.fgd#L627-L630

When using the QuakeMapFormat class on this map file (see below) I get a System.FormatException: Input string was not in a correct format with the following stack trace:

Unhandled exception. System.FormatException: Input string was not in a correct format.
   at System.Number.ThrowOverflowOrFormatException(ParsingStatus status, TypeCode type)
   at System.Decimal.Parse(String s, NumberStyles style)
   at Sledge.Formats.Map.Formats.QuakeMapFormat.ParseDecimal(IEnumerator`1 it)
   at Sledge.Formats.Map.Formats.QuakeMapFormat.ReadTextureAxis(IEnumerator`1 it)
   at Sledge.Formats.Map.Formats.QuakeMapFormat.ReadFace(IEnumerator`1 it)
   at Sledge.Formats.Map.Formats.QuakeMapFormat.ReadSolid(IEnumerator`1 it)
   at Sledge.Formats.Map.Formats.QuakeMapFormat.ReadEntity(IEnumerator`1 it)
   at Sledge.Formats.Map.Formats.QuakeMapFormat.Read(Stream stream)

The map file is short and simple, it is the default map with the texture replaced. The texture is the "Icon.png" file in the "Generic" game mode folder (not that it matters... probably).

The map was created using TrenchBroom and is in the "Valve" format as specified here: https://trenchbroom.github.io/manual/latest/#compiling_maps:~:text=for%20more%20information.-,File%20Formats,-The%20file%20format

{
"mapversion" "220"
"classname" "worldspawn"
"_tb_textures" "textures"
// brush 0
{
( -64 -64 -16 ) ( -64 -63 -16 ) ( -64 -64 -15 ) Icon [ 0 -0.9999999999999848 -1.7484556036340138e-07 0 ] [ 0 1.7484556036340138e-07 -0.9999999999999848 0 ] 360 1 1
( -64 -64 -16 ) ( -64 -64 -15 ) ( -63 -64 -16 ) Icon [ 1 0 -0 55 ] [ 0 -0 -1 64 ] 0 1 1
( -64 -64 -16 ) ( -63 -64 -16 ) ( -64 -63 -16 ) Icon [ -0.9999999999999848 -1.7484556036340138e-07 0 0 ] [ 1.7484556036340138e-07 -0.9999999999999848 0 0 ] 360 1 1
( 64 64 16 ) ( 64 65 16 ) ( 65 64 16 ) Icon [ 0.9999999999999848 -1.7484556036340138e-07 0 0 ] [ -1.7484556036340138e-07 -0.9999999999999848 0 0 ] 360 1 1
( 64 64 16 ) ( 65 64 16 ) ( 64 64 17 ) Icon [ -0.9999999999999848 0 -1.7484556036340138e-07 0 ] [ 1.7484556036340138e-07 0 -0.9999999999999848 0 ] 360 1 1
( 64 64 16 ) ( 64 64 17 ) ( 64 65 16 ) Icon [ 0 0.9999999999999848 -1.7484556036340138e-07 0 ] [ 0 -1.7484556036340138e-07 -0.9999999999999848 0 ] 360 1 1
}
}

Package version: 1.0.4 (NuGet)

Right now SerializedObjectFormatter.CleanLine() truncates anything after // token even if it's inside a valid string (such as URL).
It should be only removed if it's outside the string or the sole content of the line.

Example:

"web"
{
	"WebFav0_URL"		"https://www.google.com/"
	"WebFav0_Name"		"Google"
}