Code Monkey home page Code Monkey logo

valvepak's Introduction

Source 2 Viewer

GitHub Workflow Status

* The library component of Source 2 Viewer is called ValveResourceFormat (VRF).

Valve's Source 2 resource file format parser, decompiler, and exporter. Source 2 files usually end with _c, for example .vmdl_c.

This repository is split into three components:

  • CLI Decompiler - File data viewer, decompiler and a playground for testing new formats and features.
  • GUI Viewer - A vpk archive viewer and extractor. Also supports viewing resources such as sounds, textures, models, maps, and much more.
  • Library - Provides public API to parse resource files and some helpers.

βš’ View the official website for downloads.

Join our Discord

Join our Discord

Eye catchy screenshots

What's supported?

  • VPK viewer which supports opening and exporting files
  • Creating new vpk archives
  • Model viewer and decompiler to glTF and modeldoc
  • Map viewer and decompiler to glTF and vmap
  • Material decompiler to vmat
  • Sound player
  • Binary KeyValues3 parser
  • NTRO support

Limitations

This tool is based entirely on a reverse engineered effort because Valve does not provide any documentation or Source 2 code (SDK or engine code), while the Source 1 SDK and leaked engine code are helpful, a lot of systems and formats have changed.

The code contained in this repository is based on countless hours of reverse engineering Source 2 games and not all intricate details have been figured out.

If you are interested in helping, take a look at the open issues and join our Discord.

Not all formats are 100% supported, some parameters are still unknown and not fully understood.

Supported resource types

Ext Name Support
vagrp Animation Group πŸ‘
vanim Animation πŸ‘
vanmgrph Animation Graph No
vcompmat Composite Material No
vcss Panorama Style πŸ‘
vdata Data πŸ‘
vents EntityLump πŸ‘
vjs Panorama Script πŸ‘
vmap Map πŸ‘
vmat Material πŸ‘
vmdl Model πŸ‘
vmesh Mesh πŸ‘
vmorf MorphSet πŸ‘
vpcf Particle System πŸ‘
vpdi Panorama Dynamic Images No
vphys Physics Collision Mesh πŸ‘
vpost Postprocessing Settings πŸ‘
vpsf Particle Snapshot No
vpulse Pulse Graph Definition No
vrman ResourceManifest πŸ‘
vrmap Resource Remap Table No
vrr Response rules πŸ‘
vseq Sequence Group No
vsmart Smart Prop Partially
vsnap Particle Snapshot πŸ‘
vsnd Sound πŸ‘
vsndevts Sound Event Script πŸ‘
vsndstck Sound Stack Script πŸ‘
vsurf Surface Properties No
vsvg Panorama Vector Graphic πŸ‘
vtex Compiled Texture πŸ‘
vts Panorama TypeScript πŸ‘
vvis WorldVisibility No
vwnod WorldNode πŸ‘
vwrld World πŸ‘
vxml Panorama Layout πŸ‘
Β  Β  Β 
vpk Pak (package) πŸ‘ Handled by ValvePak
vcs Compiled Shader πŸ‘ Handled by CompiledShader
vfont Bitmap Font πŸ‘ Decrypts VFONT1, supported in Source 1 and Source 2.
dat Closed Captions πŸ‘ Handled by ClosedCaptions
bin Tools Asset Info πŸ‘ Handled by ToolsAssetInfo
vdpn Dota Patch Notes πŸ‘
vdacdefs DAC Game Defs Data No
vfe Flex Scene File πŸ‘ Handled by FlexSceneFile
vcd VCD No
vcdlist VCD list πŸ‘

List of supported magics

Magic Description
0x03564B56 VKV\x03 - First binary keyvalues 3 encoding with custom block compression
0x4B563301 KV3\x01 - Binary keyvalues 3 (version 1)
0x4B563302 KV3\x02 - Binary keyvalues 3 (version 2)
0x4B563303 KV3\x03 - Binary keyvalues 3 (version 3)
0x4B563304 KV3\x04 - Binary keyvalues 3 (version 4)
0x564B4256 VBKV - binary keyvalues 1 (handled by ValveKeyvalue)
0x55AA1234 VPK - valve package (handled by ValvePak)
0x44434356 VCCD - closed captions
0xC4CCACE8 tools asset info
0xC4CCACE9 tools asset info (newer version)
0x32736376 vcs2 - compiled shader
0x31415926 murmurhash2 seed used in various places (like entity keys)
VFONT1 "encrypted" font file
0x00564645 VFE - flex scene file

Command-line options

Option Description
Input
--input (or -i) Input file to be processed. With no additional arguments, a summary of the input(s) will be displayed.
--recursive If specified and given input is a folder, all sub directories will be scanned too.
--recursive_vpk If specified along with --recursive, will also recurse into VPK archives.
--vpk_extensions (or -e) File extension(s) filter, example: "vcss_c,vjs_c,vxml_c".
--vpk_filepath (or -f) File path filter, example: "panorama\" or "scripts/items/items_game.txt".
--vpk_cache Use cached VPK manifest to keep track of updates. Only changed files will be written to disk.
--vpk_verify Verify checksums and signatures.
Output
--output (or -o) Output path to write to. If input is a folder (or a VPK), this should be a folder.
--all (or -a) Print the content of each resource block in the file.
--block (or -b) Print the content of a specific block, example: DATA, RERL, REDI, NTRO.
--vpk_decompile (or -d) Decompile supported resource files.
--vpk_list (or -l) Lists all resources in given VPK. File extension and path filters apply.
--vpk_dir Print a list of files in given VPK and information about them.
Type specific export
--gltf_export_format Exports meshes/models in given glTF format. Must be either 'gltf' (default) or 'glb'.
--gltf_export_materials Whether to export materials during glTF exports.
--gltf_textures_adapt Whether to perform any glTF spec adaptations on textures (e.g. split metallic map).
--gltf_export_extras Export additional Mesh properties into glTF extras
--tools_asset_info_short Whether to print only file paths for tools_asset_info files.
Other
--threads If higher than 1, files will be processed concurrently.
--version Show version information.
--help Show help information.

There are also --stats related options, but they are not listed here as they are not relevant to most users.

Examples:

# List all files in the vpk
# Use `--vpk_dir` to also print file metadata
.\Decompiler.exe -i "core/pak01_dir.vpk" --vpk_list

# Export the entire vpk as is
.\Decompiler.exe -i "core/pak01_dir.vpk" --output "pak01_exported"

# Export only the "panorama/layout" folder
.\Decompiler.exe -i "core/pak01_dir.vpk" --output "pak01_exported" --vpk_filepath "panorama/layout"

# Decompile and export all Panorama files to a folder named "exported"
.\Decompiler.exe -i "core/pak01_dir.vpk" -e "vjs_c,vxml_c,vcss_c" -o "exported" -d

# Print resource blocks for a specific file similar to resourceinfo.exe in Source 2
# Use `--block DATA` to only print a specific block
.\Decompiler.exe -i "file.vtex_c" --all

# Decompile a specific file on disk
.\Decompiler.exe -i "file.vtex_c" -o exported.png

License

Contents of this repository are available under MIT license, except for Tests/Files folder contains files which have likely come from Valve's games.

valvepak's People

Contributors

babelshift avatar dependabot[bot] avatar sinz163 avatar xpaw avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

valvepak's Issues

Encountered OtherMD5Section with size of 0 (should be 48)

The following stack trace is reported when attempting to read test_clock.vpk from this Workshop map:

System.IO.InvalidDataException: Encountered OtherMD5Section with size of 0 (should be 48)
   at SteamDatabase.ValvePak.Package.ReadOtherMD5Section() in /home/runner/work/ValvePak/ValvePak/ValvePak/ValvePak/Package.Read.cs:line 266
   at SteamDatabase.ValvePak.Package.Read(Stream input) in /home/runner/work/ValvePak/ValvePak/ValvePak/ValvePak/Package.Read.cs:line 84
   at GUI.Types.Viewers.Package.Create(VrfGuiContext vrfGuiContext, Byte[] input) in D:\a\ValveResourceFormat\ValveResourceFormat\GUI\Types\Viewers\Package.cs:line 37
   at GUI.MainForm.ProcessFile(VrfGuiContext vrfGuiContext, PackageEntry file) in D:\a\ValveResourceFormat\ValveResourceFormat\GUI\MainForm.cs:line 606
   at System.Threading.Tasks.Task`1.InnerInvoke()
   at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location ---
   at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot, Thread threadPoolThread)

CS2 is able to load this map and so is GCFScape.

The exact test_clock.vpk used when this error was encountered can be downloaded here: https://gist.github.com/roge/2493b95513b8f1b0914e1664d9cee983/raw/0d934400f71018a9547b288e0209802bf983b0b6/test_clock.vpk

Findings based on disassembly

  • The metadata that can be included after directory entries is never used as part of the file when reading it.
  • There are two special archive IDs: 0x7fff (current archive, for non-_dir archives), and 0xffff (which means "end of list")
  • Each VPK entry has one or more archive locations, terminated by 0xffff (which does not have an offset or size after it)
  • Archive locations are concatenated to create the file's actual content while reading

Bad VPK Version (4294967041)

Describe the bug

Whether I open the VPK using GCFScape or VRF, you will be prompted with the VPK version problem.
VRF displays "Bad VPK version. (4294967041)".
This is an L4D2 game package file. The game can read it, but the unpacking softwares cannot read the file

What Valve game does this happen in

Left 4 Dead 2

Reference file in a Source 2 game

(Not an official vpk fileοΌ‰

Expected behavior

System.IO.InvalidDataException: Bad VPK version. (4294967041)
at SteamDatabase.ValvePak.Package.Read(Stream input) in /home/runner/work/ValvePak/ValvePak/ValvePak/ValvePak/Package.cs:line 203
at SteamDatabase.ValvePak.Package.Read(String filename) in /home/runner/work/ValvePak/ValvePak/ValvePak/ValvePak/Package.cs:line 157
at GUI.Types.Viewers.Package.Create(VrfGuiContext vrfGuiContext, Byte[] input) in D:\a\ValveResourceFormat\ValveResourceFormat\GUI\Types\Viewers\Package.cs:line 35
at GUI.MainForm.ProcessFile(VrfGuiContext vrfGuiContext, PackageEntry file) in D:\a\ValveResourceFormat\ValveResourceFormat\GUI\MainForm.cs:line 538
at GUI.MainForm.<>c__DisplayClass22_0.b__0() in D:\a\ValveResourceFormat\ValveResourceFormat\GUI\MainForm.cs:line 456
at System.Threading.Tasks.Task`1.InnerInvoke()
at System.Threading.Tasks.Task.<>c.<.cctor>b__272_0(Object obj)
at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location ---
at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot, Thread threadPoolThread)
the vpk file.zip

What version of VRF are you using? On what platform?

the latest one,Windows

Remove `IsDirVPK` check

image

It's possible for chunked vpks to not have _dir suffix.

I think it should work just fine with this check removed:

if (!IsDirVPK)
{
throw new InvalidOperationException("Given VPK filename does not end in '_dir.vpk', but entry is referencing an external archive.");
}

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    πŸ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. πŸ“ŠπŸ“ˆπŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❀️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.