asheiduk / factorio-blueprint-decoder Goto Github PK

View Code? Open in Web Editor NEW

26.0 5.0 2.0 8.05 MB

Decode Factorio's binary `blueprint-storage.dat` file into JSON for backup, downgrading or further manipulation.

Python 91.80% Shell 1.23% Makefile 5.57% Lua 1.40%

factorio factorio-blueprints

factorio-blueprint-decoder's People

Contributors

Stargazers

Watchers

Forkers

flamesla phy6geniuxgh

factorio-blueprint-decoder's Issues

Fails to decode blueprint library containing trains that was stored while Krastorio 2 was active

Steps to reproduce (Windows 10)

Disable all mods
Activate Krastorio 2 mod
Create blueprint that contains a cargo wagon / fluid wagon / train locomotive and add it to the blueprint library
Exit game
Run python .\factorio-blueprint-decoder\decode <User Home>\AppData\Roaming\Factorio\blueprint-storage.dat

Alternate steps to reproduce (Windows 10)

Disable all mods
Create blueprint that contains a cargo wagon / fluid wagon / train locomotive and add it to the blueprint library
Activate Krastorio 2 mod
Make any change to the blueprint library
Exit game.
Run python .\factorio-blueprint-decoder\decode <User Home>\AppData\Roaming\Factorio\blueprint-storage.dat

Error message

Traceback (most recent call last):
  File "...\factorio-blueprint-decoder\decode", line 2879, in <module>
    library = parse_blueprint_library(PrimitiveStream(f))
  File "...\factorio-blueprint-decoder\decode", line 2550, in parse_blueprint_library
    parse_library_objects(stream, global_index, book_item, library_version)
  File "...\factorio-blueprint-decoder\decode", line 2175, in parse_library_objects
    handler_result = handler(stream, index, library_version)
  File "...\factorio-blueprint-decoder\decode", line 2636, in parse_blueprint
    parse_entities(stream, index, result)
  File "...\factorio-blueprint-decoder\decode", line 2300, in parse_entities
    handler(stream, index, entity)
  File "...\factorio-blueprint-decoder\decode", line 1308, in eh_cargo_wagon
    ep_v1_1_51_4_flag(stream, index, entity, 0x00)
  File "...\factorio-blueprint-decoder\decode", line 546, in ep_v1_1_51_4_flag
    stream.expect(expected_value)
  File "...\factorio-blueprint-decoder\decode", line 339, in expect
    raise ParseError(f"expected {expected:#04x} but got {actual:#04x} at position {position} ({position:#x})")
__main__.ParseError: expected 0x00 but got 0x01 at position <some number>

("..." are paths to the checkout dir replaced by me)

Additional Info

Since Krastorio 2 adds equipment grids to train wagons, the problem might be related to that.

Radar and turrents broken

Radars and turrets have a new byte -- purpose unknown.
This byte must have been introduced somewhere between 1.1.62 (exclusive) and 1.1.87 (inclusive).
TODOs

fix decoder
narrow down to exact version :-(
add tests

KeyError on first run

Hi, here is the stack trace. I am following your instructions from clone -> README.md -> ./decode

If you are not maintaining, I will take a look into it. Thanks in advance!

buddy@darkstationmk2:~/dev/factorio-blueprint-decoder$ ./decode blueprint-storage.dat 


Traceback (most recent call last):
  File "./decode", line 3114, in <module>
    library = parse_blueprint_library(PrimitiveStream(f))
  File "./decode", line 2785, in parse_blueprint_library
    parse_library_objects(stream, global_index, book_item, library_version)
  File "./decode", line 2427, in parse_library_objects
    handler_result = handler(stream, index, library_version)
  File "./decode", line 2930, in parse_blueprint_book
    parse_library_objects(stream, index, result, library_version)
  File "./decode", line 2427, in parse_library_objects
    handler_result = handler(stream, index, library_version)
  File "./decode", line 2879, in parse_blueprint
    fixup_entity_ids(result)
  File "./decode", line 2728, in fixup_entity_ids
    walk(blueprint)
  File "./decode", line 2722, in walk
    walk(value)
  File "./decode", line 2726, in walk
    walk(value)
  File "./decode", line 2714, in walk
    it["locomotives"] = [ id_to_entity[l]["entity_number"] for l in it["locomotives"]]
  File "./decode", line 2714, in <listcomp>
    it["locomotives"] = [ id_to_entity[l]["entity_number"] for l in it["locomotives"]]
KeyError: 0

License

What license is this project using?
Is it light OSI like BSD/MIT, or heavy like GPL, or even custom one?

Please add support for game version 1.1-Stable (1.1.21)

In Steam version of the game 1.1-Stable has already become 1.1.21.

We get the following error:

decode blueprint-storage.dat > my-bps1.json
Traceback (most recent call last):
  File "./decode", line 2405, in <module>
    library = parse_blueprint_library(PrimitiveStream(f))
  File "./decode", line 2144, in parse_blueprint_library
    stream.expect(0x00, 0x00)
  File "./decode", line 278, in expect
    raise ParseError(f"expected {expected:#04x} but got {actual:#04x} at position {position} ({position:#x})")
__main__.ParseError: expected 0x00 but got 0x2c at position 11034 (0x2b1a)

Here is the link of the forum post to someone else's blueprint-storage.dat file.
Our errors are similar.

ParseError: expected 0x00 but got 0x01 at position 47104 (0xb800)

Stacktrace

D:\git\gitlab\tools\factorio-blueprint-chef\.venv-312\Scripts\python.exe D:\git\gitlab\tools\factorio-blueprint-chef\src\factorio_blueprint_decoder\decode.py C:\Users\Hares\AppData\Roaming\Factorio\blueprint-storage.dat 
Traceback (most recent call last):
  File "D:\git\gitlab\tools\factorio-blueprint-chef\src\factorio_blueprint_decoder\decode.py", line 2905, in <module>
    library = parse_blueprint_library(PrimitiveStream(f))
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\git\gitlab\tools\factorio-blueprint-chef\src\factorio_blueprint_decoder\decode.py", line 2576, in parse_blueprint_library
    parse_library_objects(stream, global_index, book_item, library_version)
  File "D:\git\gitlab\tools\factorio-blueprint-chef\src\factorio_blueprint_decoder\decode.py", line 2201, in parse_library_objects
    handler_result = handler(stream, index, library_version)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\git\gitlab\tools\factorio-blueprint-chef\src\factorio_blueprint_decoder\decode.py", line 2721, in parse_blueprint_book
    parse_library_objects(stream, index, result, library_version)
  File "D:\git\gitlab\tools\factorio-blueprint-chef\src\factorio_blueprint_decoder\decode.py", line 2201, in parse_library_objects
    handler_result = handler(stream, index, library_version)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\git\gitlab\tools\factorio-blueprint-chef\src\factorio_blueprint_decoder\decode.py", line 2662, in parse_blueprint
    parse_entities(stream, index, result)
  File "D:\git\gitlab\tools\factorio-blueprint-chef\src\factorio_blueprint_decoder\decode.py", line 2326, in parse_entities
    handler(stream, index, entity)
  File "D:\git\gitlab\tools\factorio-blueprint-chef\src\factorio_blueprint_decoder\decode.py", line 867, in eh_logistic_container
    ep_v1_1_62_5_flag(stream, index, entity)
  File "D:\git\gitlab\tools\factorio-blueprint-chef\src\factorio_blueprint_decoder\decode.py", line 581, in ep_v1_1_62_5_flag
    stream.expect(0x00)
  File "D:\git\gitlab\tools\factorio-blueprint-chef\src\factorio_blueprint_decoder\decode.py", line 347, in expect
    raise ParseError(f"expected {expected:#04x} but got {actual:#04x} at position {position} ({position:#x})")
ParseError: expected 0x00 but got 0x01 at position 47104 (0xb800)

Blueprint Storage

blueprint-storage.zip

ParseError: expected 0x00 but got 0x02 at position 630668 (0x99f8c) in ep_railway_vehicle_common

When using ./decode, I get the following error.

Traceback (most recent call last):
  File "./decode", line 2648, in <module>
    library = parse_blueprint_library(PrimitiveStream(f))
  File "./decode", line 2336, in parse_blueprint_library
    parse_library_objects(stream, global_index, book_item, library_version)
  File "./decode", line 2008, in parse_library_objects
    handler_result = handler(stream, index, library_version)
  File "./decode", line 2416, in parse_blueprint
    parse_entities(stream, index, result)
  File "./decode", line 2130, in parse_entities
    handler(stream, index, entity)
  File "./decode", line 1163, in eh_cargo_wagon
    ep_railway_vehicle_common(stream, index, entity)
  File "./decode", line 548, in ep_railway_vehicle_common
    stream.expect(0x00, 0x00)
  File "./decode", line 314, in expect
    raise ParseError(f"expected {expected:#04x} but got {actual:#04x} at position {position} ({position:#x})")
__main__.ParseError: expected 0x00 but got 0x02 at position 630668 (0x99f8c)

This blueprint is from an 1.0.0 game which I had initially started as 0.18 (attached).
blueprint-storage.zip

Hello, having found nowhere, neither on your git, nor on your site, nor in your project a way to contact you (email/discord) I allow myself to send you a message here
I have some problems using your project.
Could you please contact me?

[email protected]
Dae#5125 (discord)

Error importing from the game version 1.1.21.

XXX@PC-XXX MSYS ~/factorio-blueprint-decoder
$ ./decode blueprint-storage.dat > my-bps1.json
Traceback (most recent call last):
  File "./decode", line 2404, in <module>
    library = parse_blueprint_library(PrimitiveStream(f))
  File "./decode", line 2141, in parse_blueprint_library
    global_index = parse_index(stream, result)
  File "./decode", line 1827, in parse_index
    index.add(name_id, prototype_name, name)
  File "./decode", line 174, in add
    raise KeyError(f"unknown prototype '{prototype}'")
KeyError: "unknown prototype 'copy-paste-tool'"

$ python3
Python 3.8.6 (default, Oct 23 2020, 14:59:35)
[GCC 9.3.0] on msys

and other machine:

# python3
Python 3.8.6 (default, Oct  9 2020, 20:21:12)
[Clang 10.0.0 ([email protected]:llvm/llvm-project.git llvmorg-10.0.0-0-gd32170dbd on freebsd12

I can put the blueprint-storage.dat file somewhere, but it's big - 125MB.

Angels Flarestacks direction is not retained

Great tool
I'm about to downgrade a factorio version and want to keep the changes, which I have done in the meantime

I was exporting the blueprint lib with angels/bobs entities and the flare stack direction of angels is not retained, meaning, that every instances with the fluid input are facing downwards after importing the string.

I guess, I can fix it manually, because its not too much, but I maybe it could be transferred too

edit:
same for clarifier

Unknown prototype: simple-entity-with-force

Blueprint Storage

Same blueprint storage as in #15

Workaround

Changed lines 231-234 to the following:

From

factorio-blueprint-decoder/decode

Lines 231 to 234 in d6cd156

    
           if prototype not in self._type_mapping: 
        
               raise KeyError(f"unknown prototype '{prototype}'") 
        
           type = self._type_mapping[prototype] 
        
           bucket = self._data[type]

To

        type: Index.Type
        if (prototype in self._type_mapping):
            type = self._type_mapping.get(prototype)
        elif ('entity' in prototype):
            type = Index.Type.ENTITY
        else:
            raise KeyError(f"unknown prototype '{prototype}'")

Encode support

It would be nice if importing directly to blueprint-storage.dat was possible as well, especially since pasting very large blueprints can freeze the game.

Got error while decoding blueprint-storage.dat

Factorio Version: 1.1.45
OS: macOS Big Sur 11.5.2
python: 3.7.0

factorio-blueprint-decoder git:(master) python3 decode ../blueprint-storage.dat > /tmp/bps-latest.json
Traceback (most recent call last):
File "decode", line 2690, in
library = parse_blueprint_library(PrimitiveStream(f))
File "decode", line 2368, in parse_blueprint_library
parse_library_objects(stream, global_index, book_item, library_version)
File "decode", line 2036, in parse_library_objects
handler_result = handler(stream, index, library_version)
File "decode", line 2507, in parse_blueprint_book
parse_library_objects(stream, index, result, library_version)
File "decode", line 2036, in parse_library_objects
handler_result = handler(stream, index, library_version)
File "decode", line 2507, in parse_blueprint_book
parse_library_objects(stream, index, result, library_version)
File "decode", line 2036, in parse_library_objects
handler_result = handler(stream, index, library_version)
File "decode", line 2507, in parse_blueprint_book
parse_library_objects(stream, index, result, library_version)
File "decode", line 2036, in parse_library_objects
handler_result = handler(stream, index, library_version)
File "decode", line 2448, in parse_blueprint
parse_entities(stream, index, result)
File "decode", line 2158, in parse_entities
handler(stream, index, entity)
File "decode", line 1695, in eh_ammo_turret
stream.expect(*[0x00]*9)
File "decode", line 317, in expect
raise ParseError(f"expected {expected:#04x} but got {actual:#04x} at position {position} ({position:#x})")
main.ParseError: expected 0x00 but got 0x01 at position 3695124 (0x386214)

blueprint-storage.dat.zip

Error decoding unicode char

Traceback (most recent call last):
  File "D:\tmp\factorio\decode.py", line 2647, in <module>
    json.dump(library, sys.stdout, indent=4, sort_keys=True, ensure_ascii=False)
  File "C:\Python39\lib\json\__init__.py", line 180, in dump
    fp.write(chunk)
  File "C:\Python39\lib\encodings\cp1251.py", line 19, in encode
    return codecs.charmap_encode(input,self.errors,encoding_table)[0]
UnicodeEncodeError: 'charmap' codec can't encode character '\u2260' in position 1: character maps to <undefined>

File "encode-export-string"

File "encode-export-string", line 4
jq -c . |
^
SyntaxError: invalid syntax

	if prototype not in self._type_mapping:
	raise KeyError(f"unknown prototype '{prototype}'")
	type = self._type_mapping[prototype]
	bucket = self._data[type]

asheiduk / factorio-blueprint-decoder Goto Github PK

factorio-blueprint-decoder's People

Contributors

Stargazers

Watchers

Forkers

factorio-blueprint-decoder's Issues

Steps to reproduce (Windows 10)

Alternate steps to reproduce (Windows 10)

Error message

Additional Info

Stacktrace

Blueprint Storage

Blueprint Storage

Workaround

From

To

Recommend Projects

Recommend Topics

Recommend Org