tibiamaps / tibia-maps-script Goto Github PK

View Code? Open in Web Editor NEW

17.0 5.0 8.0 260 KB

:wrench: A command-line utility to convert between binary Tibia maps and human-readable forms of the map data.

Home Page: https://tibiamaps.io/

License: MIT License

JavaScript 100.00%

tibia tibia-maps converter

tibia-maps-script's Introduction

`tibia-maps` CLI

tibia-maps is a command-line utility to convert between binary Tibia maps and human-readable forms of the map data.

Installation

Note: Use the expected Node.js version!

npm install -g tibia-maps

If you’re on macOS and you get an error about xcb-shm, try this instead:

export PKG_CONFIG_PATH="${PKG_CONFIG_PATH}:/opt/X11/lib/pkgconfig"; npm install -g tibia-maps

Usage

`minimap/` → `data/`

To generate PNGs for the maps + pathfinding visualization and JSON for the marker data based on the map files in the minimap directory, run:

tibia-maps --from-minimap=./minimap --output-dir=./data

The output is saved in the data directory.

`data/` → `minimap/`

To generate Tibia-compatible minimap/* files based on the PNGs and JSON files in the data directory, run:

tibia-maps --from-data=./data --output-dir=./minimap-new

The output is saved in the minimap-new directory.

Author


Mathias Bynens

tibia-maps-script's People

Contributors

Stargazers

Watchers

Forkers

davifo sleepyynet dsieradzki-stuff otbr umdez rmobis

tibia-maps-script's Issues

Figure out maximum marker description length

Marker descriptions are limited to 99 bytes. Since only symbols that can be represented in the windows-1252 encoding may be used, 99 bytes equals 99 symbols.

Markers with descriptions longer than 99 symbols/bytes are truncated when viewed in the client. The client then overwrites the map file with the truncated marker description.

Support optional markers/maps

As @LeeKunTibia said:

I'm having a hard time dealing with the Ank desert digging spots... There are so many markers that I can't click anywhere else around a tomb and it's really annoying, but I can see their usefulness of course. In similar fashion, there are also several markers around Thais of Pinatas which some players might be interested, but others certainly aren't.

We could add support for optional markers. These markers would live in a separate subdirectory (perhaps extra-markers?) and consist of a single JSON file per set of markers. Some example sets/categories:

achievements: contains highly-specific achievement-related markers, that are unlikely to be used generally (e.g. Ank desert digging spots)
events (or spawns?): e.g. pinata spawn points in Thais
- not sure if we want to name this spawns, since there are spawns like undead cavebear and midnight panther that we do want to include in the standard markers
blocklist: contains known "bad" markers that we explicitly do not want to include when importing markers from *.bin files. These include markers like "Commander Stone" which the client auto-inserts -- we don't want those, because we already have our own marker for that. This is kind of a separate feature but it could be based on the same implementation.
... more ideas?

Colors

I dont really know JS but a tip for you on the Colors for the map that will be future proof that i found with pure luck its a pattern in the color bytes this is php but you can probably easy make it js

$r = 0;
$g = 0;
$b = 0;

for ($i = 0; $i < 240; $i++) {
  $col[$i] = imagecolorallocate($im, $r, $g, $b);

  $b += 51;

  if ($b > 255) {
    $b = 0;
    $g += 51;
  }

  if ($g > 255) {
    $g = 0;
    $r += 51;
  }

  if ($r > 255) {
    $r = 50;
  }
}

Use absolute coordinates in marker JSON

These coordinates are compatible with *.exp files and the online map viewer.

.exp → .png + *.json

With Tibia 11 getting newer updates and some features not being implemented on the flash client, I suspect more people will make the switch and want to bring along their map files.

That's the case for me, only I'm not familiar enough with Node / JS to attempt it myself and the byte handling went over my head.

I did workout a quick shell command to extract the coordinates from the .exp file and copy over the corresponding .png files from a full map download and for the most part that worked okay, but I lost the data on areas explored. The whole section is uncovered even though I might have only explored a small part of it.

One thing I do like about having everything uncovered is the ability to see the water between islands instead of it being blanked out. I'm wondering if it'd be possible to detect the large bodies of water in a given map file and configure it so water gets shown.

Node.js v6: use new `Buffer` APIs

https://medium.com/@jasnell/node-js-buffer-api-changes-3c21f1048f97

Unknown color ID while trying to convert from maps on Linux

Hi,

I've been trying to convert my *.map files to *.png, but I'm getting the error Unknown color ID. Here's the full log:

tibia-maps --from-maps=./Automap --output-dir=./data

/data/bounds.json created successfully.
Rendering floor 00…

assert.js:81
  throw new assert.AssertionError({
  ^
AssertionError: Unknown color ID: 40
    at Console.assert (console.js:95:23)
    at renderMap (/usr/lib/node_modules/tibia-maps/src/from-maps.js:74:12)
    at /usr/lib/node_modules/tibia-maps/src/from-maps.js:196:25
    at FSReqWrap.readFileAfterClose [as oncomplete] (fs.js:438:3)

Do you know how can I fix it?

Thanks!

Too little files

I'm not sure this is working… I run this script on the data downloaded from tibia-map-data repo, and this is the list of files that the script generated:

27.09.2020  23:56           111 369 minimapmarkers.bin
27.09.2020  23:56             4 499 Minimap_Color_81152_113408_9.png
27.09.2020  23:56             9 019 Minimap_Color_81152_115968_9.png
27.09.2020  23:56            10 927 Minimap_Color_81152_118528_9.png
27.09.2020  23:56            11 091 Minimap_Color_81152_121088_9.png
27.09.2020  23:56            16 828 Minimap_Color_81152_123648_9.png
27.09.2020  23:56            15 591 Minimap_Color_81152_126208_9.png
27.09.2020  23:56             8 478 Minimap_Color_81152_128768_9.png
27.09.2020  23:56            11 068 Minimap_Color_81152_131328_9.png
27.09.2020  23:56             8 403 Minimap_Color_81152_133888_9.png
27.09.2020  23:56             1 365 Minimap_Color_81152_136448_9.png
27.09.2020  23:56             4 044 Minimap_WaypointCost_81152_113408_9.png
27.09.2020  23:56             8 799 Minimap_WaypointCost_81152_115968_9.png
27.09.2020  23:56            10 525 Minimap_WaypointCost_81152_118528_9.png
27.09.2020  23:56            10 640 Minimap_WaypointCost_81152_121088_9.png
27.09.2020  23:56            14 247 Minimap_WaypointCost_81152_123648_9.png
27.09.2020  23:56            10 517 Minimap_WaypointCost_81152_126208_9.png
27.09.2020  23:56             6 519 Minimap_WaypointCost_81152_128768_9.png
27.09.2020  23:56             9 362 Minimap_WaypointCost_81152_131328_9.png
27.09.2020  23:56             6 964 Minimap_WaypointCost_81152_133888_9.png
27.09.2020  23:56             1 367 Minimap_WaypointCost_81152_136448_9.png

Marker format is different for certain markers

There’s something about the minimapmarkers.bin format I don’t quite understand yet. For the most part (99.99%), it seems to follow the format described in our guide.

However, some (not all!) very specific markers that are automatically added by quest NPCs seem to follow a different format.

For example, in the attached file, this marker is fine:

$ hexdump -s 0xEC6C -n 78 -C minimap/minimapmarkers.bin
0000ec6c  0a 24 0a 0a 08 a5 86 02  10 f1 f3 01 18 03 10 03  |.$..............|
0000ec7c  1a 12 49 6e 73 65 63 74  6f 69 64 20 6d 6f 6e 6f  |..Insectoid mono|
0000ec8c  6c 69 74 68 20 00 0a 12  0a 0a 08 a6 86 02 10 89  |lith ...........|
0000ec9c  f4 01 18 0e 10 04 1a 00  20 00 0a 12 0a 0a 08 a6  |........ .......|
0000ecac  86 02 10 8d f4 01 18 0e  10 04 1a 00 20 00        |............ .|
0000ecba

But the next one (a marker added by talking to NPC Sholley during the Dark Trails Quest), isn’t:

$ hexdump -s 0xECBA -n 65 -C minimap/minimapmarkers.bin
0000ecba  0a 24 0a 0a 08 a6 86 02  10 d1 f9 01 18 07 10 02  |.$..............|
0000ecca  1a 0d 51 75 61 6e 64 6f  6e 73 20 48 6f 6d 65 20  |..Quandons Home |
0000ecda  ce 95 ea be f6 2d 0a 19  0a 0a 08 a7 86 02 10 a0  |.....-..........|
0000ecea  f4 01 18 07 10 03 1a 07  41 6e 74 65 6e 6e 61 20  |........Antenna |
0000ecfa  00                                                |.|
0000ecfb

(Note: the above example shows the bytes for the marker following it as well.)

Unlike other markers, this particular marker doesn’t end with 20 00 — instead, it ends with 20 ce 95 ea be f6 2d. Why is that? What does it mean?

To allow importing this marker, we can patch tibia-maps-script as follows. Replace these lines: https://github.com/tibiamaps/tibia-maps-script/blob/master/src/from-minimap.js#L92-L94

// The byte sequence 0x20 0x00 marks the end of the marker.
console.assert(buffer[index++] === 0x20);
console.assert(buffer[index++] === 0x00);

…with…

while (buffer[index] !== undefined && buffer[index] !== 0x0A) {
  index++;
}

allow union/additive mode for markers

Working on a few PRs on tibiamaps/tibia-maps-data I thought it would be really helpful to have a way to import markers additively, so I started working on this proposal.

Idea

The idea is to have a way to add new markers to an already existing minimapmarkers.bin or markers.json file, without overwriting old ones, by importing them from another file. We would add a new flag (--union, --keep-markers, etc.) that would signal the script that if the destination file already exists, it should not be overwritten but rather added to.

This can come in handy when importing new POIs made on old data, if I want to import all TibiaMaps markers but also keep my own or if a friend has some markers for some bestiary bombs and I want those imported along with mine.

Example

Below is a representation of files containing given markers, before and after running the following command with the new flag: tibia-maps --union --from-minimap=./minimap --output-dir=./extra/points-of-interest:

./minimap/minimapmarkers.bin

marker-a
marker-b

./extra/points-of-interest/markers.json (before running the script)

marker-c
marker-d

./extra/points-of-interest/markers.json (after running the script)

marker-a
marker-b
marker-c
marker-d

On Marker Duplication

We obviously want to avoid having the same marker multiple times, as that can even become a size issue with time. A simple equality check of all the marker properties should be enough to identify duplicities and is what I would recommend with, but we could look into other strategies later, such as checking only for the coordinates equality and allowing a single marker per sqm, if we find we need to. Can we even have more than 1 marker per SQM, technically?

There's probably a bit more to take into consideration, but I would love to hear what you think about the idea. I'd be willing to work on a PR, if you find it could be useful too.

Avoid empty rectangles on ground floor

See tibiamaps/tibia-map-data@d1b3dc6.

const noPathBuffer = new Buffer(
    Array.from({ 'length': 0x10000 }, () => colors.unexploredPathByte)
);

Segmentation fault on really old map files

I tried to generate PNG files from my old personal .map files and it crashes.

I believe it should acknowledge that the files are wrong and then exit the program gracefully.

You can download a small sample that crashes for testing here: Automapcrash.zip

Thank you.

Install guide for windows 10 (might work for others)

2019-03-02; Updated and make sure it works had issues with 32bit for some reason so edit it for 64bit

Install guide for tibia map script converter by Mathias.
https://github.com/tibiamaps/tibia-maps-script
1: Install node.js v10.x.x LTS 64-bit version
https://nodejs.org/en/

2: Install Python 2.x.x 32-bit version
https://www.python.org/downloads/
Make sure you add Python to your Path during the install

3: Install Microsoft Visual Studio 2015 or higher i recommend 2017

Important: During the install make sure to select custom and make sure that your installing stuff related to C++

2017:
https://www.visualstudio.com/downloads/

*I made a link to my personal Dropbox https://www.dropbox.com/s/xjd99fhnv428xmj/.vsconfig?dl=0 where you can the .vsconfig file to directly setup Visual studio 2017 with the absolut minimum requiered stuff to save you people some space and time.
You load it during the regular setup (if you do this you can ignore step 4 it does it for you)

4: Install Windows 10 SDK
https://developer.microsoft.com/en-us/windows/downloads/windows-10-sdk

5: Download GTK+ and extract it on C:\ so it looks like C:\GTK(around 8 folders one of em should be named bin)
http://ftp.gnome.org/pub/GNOME/binaries/win64/gtk+/2.22/gtk+-bundle_2.22.1-20101229_win64.zip
If you need 32bit version/info it can be found on link below
https://github.com/Automattic/node-canvas/wiki/Installation%3A-Windows

6: Set system path for GTK+ by going to "Control Panel\System and Security\System" then "Advanced System Settings" then "Environment Variables" then under System variables find the one called Path and click Edit... then Add ";C:\GTK\bin" to the end without the ""

7: open CMD in administrator mode and write the following commands:
change the first command to whatever version of visual studio your using

npm config set msvs_version 2017 --global
npm install -g canvas
npm install -g tibia-maps

DONE. Enjoy!

FAQ:

Error 1:

Canvas fails to install.

Error 1 Fix:

canvas install fail is most likely due to wrong version of GTK either 64bit or 32bit depending on what you had first.. try the other.

-------------------------------------

Error 2:

Converting maps doesn't work:

E:\PATH\ver2>tibia-maps --from-maps=./Automap --output-dir=./data
module.js:440
  return process.dlopen(module, path._makeLong(filename));
                 ^

Error: The specified module could not be found.
\?\C:\Users\USERNAME\AppData\Roaming\npm\node_modules\tibia-maps\node_modules\canvas\build\Release\canvas.node
    at Error (native)
    at Object.Module._extensions..node (module.js:440:18)
    at Module.load (module.js:357:32)
    at Function.Module._load (module.js:314:12)
    at Module.require (module.js:367:17)
    at require (internal/module.js:16:19)
    at Object.<anonymous> (C:\Users\USERNAME\AppData\Roaming\npm\node_modules\tibia-maps\node_modules\canvas\lib\bindings.js:3:18)
    at Module._compile (module.js:413:34)
    at Object.Module._extensions..js (module.js:422:10)
    at Module.load (module.js:357:32)

Error 2 Fix:

Go to C:\GTK\bin\ and copy ALL .dll files and place them in: C:\Users\USERNAME\AppData\Roaming\npm\node_modules\tibia-maps\node_modules\canvas\build\Release\. Now it should work.