gtker / wow_messages Goto Github PK
View Code? Open in Web Editor NEWAuto generated messages for the World of Warcraft network protocol
License: Apache License 2.0
Auto generated messages for the World of Warcraft network protocol
License: Apache License 2.0
It would be handy to be able to look up items by name directly in the spells/items libraries.
The function should look something like:
fn lookup_item_by_name(name: &str) -> Vec<(String, &'static Item)>;
Hey @gtker! Today I was doing some testing to see if I want to take the leap to convert my entire project (wrath-rs) to use the wow_messages crate.
As an experiment I've been fiddling around with the wowm language and ran into a little issue that restricts how usable it is (or perhaps I'm misunderstanding).
When I enter a versions = "3"
tag, I expect that object to match for all Wotlk versions. However, when I create a message for versions = "3.3.5"
, during parsing it complains that there is no version of the first object for 3.3.5. Are versions not matching in a hierarchical way? I expect a message for 3.3.5 to be satisfied with an object for all versions of 3.
I've attached a screenshot with my code and the resulting error.
Wrath MovementInfo
requires
struct MovementInfo {
MovementFlags flags;
ExtraMovementFlags extra_flags;
u32 timestamp;
Vector3d position;
f32 orientation;
if (flags & ON_TRANSPORT) {
TransportInfo transport;
if (extra_flags & INTERPOLATED_MOVEMENT) {
u32 transport_time;
}
}
if (flags & SWIMMING || flags & FLYING || extra_flags & ALWAYS_ALLOW_PITCHING) {
f32 pitch1;
}
Test case is
flag A : u8 {
ONE = 1;
TWO = 2;
}
flag B : u8 {
FOUR = 4;
EIGHT = 8;
}
struct T {
A a;
B b;
if (a & ONE || a & TWO) {
u8 one_two;
}
}
and
struct T {
A a;
B b;
if (a & ONE || a & TWO || b & FOUR) {
u8 one_two_four;
}
}
It is unknown exactly what the generated Rust code should look like.
CMSG_WARDEN_DATA
and SMSG_WARDEN_DATA
seem to be encrypted using RC4 of some kind.
The current descriptions are a little difficult to understand.
If possible C and Python examples should be added like in PackedGuid.
The protocol is between 1.12 with only PIN
and 2.4.3
with PIN
, MATRIX_CARD
and AUTHENTICATOR
, maybe it supports one or more of these.
There are several world messages that contain structs with a size field.
This should be handled using self.size
like for login messages.
Fuzzing of the library could help find inconsistencies between parsing and encoding payloads, and also panics reachable from inputs.
The most generic fuzzer I can think would be:
opcode
, generate size
, generate u8[size - opcode_size]
Fuzzing is usually a separate crate that depends on your crate, so it can only access pub
items.
This needs:
read_opcodes
as pub fn read_opcodes(opcode: u16, body_size: u32, r: &mut &[u8]) -> Result<...>
write_unencrypted_server
from #77
Client version: 1.12.1
Compared Wireshark packets to a standard vanilla private server and there doesn't seem to be any difference between the auth packets, but the client interface refuses to show the CharacterSelectAddonsButton
. The client code calls GetNumAddOns
, but I'm not sure where this is pulling data from, and it's not sending any packets (using wow
filter in Wireshark) other than the standard RealmList/auth procedure stuff.
The addons are loaded (roughly) correctly on loading into the world, so the issue is definitely something to do with the server response. Perhaps you need to load in correctly into the world and receive some kind of World message in order to initialise addons correctly on a new server?
https://gtker.com/wow_messages/docs/smsg_addon_info.html only seems to supported in 3.3.5, so I don't think that's the problem.
EDIT: It looks like the WOWW display filter in Wireshark doesn't work out of the box, so I imagine there's some packet I'm missing that might explain this.
Very nice project, I was looking for a rust lib to handle WoW SRP when I was playing around with Vmangos auth and stumbled across it!
@Dantsz is trying to get items working in wrath-rs
and it would help out.
SMSG_GUILD_BANK_LIST
for TBC and Wrath requires this when sending item_random_property_id
.
This is for
smsg SMSG_AUCTION_COMMAND_RESULT = 0x025B {
u32 auction_id;
AuctionCommandAction action;
AuctionCommandResult result;
if (result == OK) {
if (action == BID_PLACED) {
u32 auction_outbid1;
}
} else if (result == ERR_INVENTORY) {
InventoryResult inventory_result;
} else if (result == ERR_HIGHER_BID) {
Guid higher_bidder;
u32 new_bid;
u32 auction_outbid2;
}
}
and
smsg SMSG_SEND_MAIL_RESULT = 0x0239 {
u32 mail_id;
MailAction action;
MailResult result;
if (result == ERR_EQUIP_ERROR) {
u32 equip_error;
}
/* TODO: requires elseif for different thing */
else {
if (action == ITEM_TAKEN) {
u32 item_guid {
comment = "cmangos/vmangos: item guid low?";
}
u32 item_count;
}
}
}
In which the order of the variables used in if statements are reversed and so can't be moved into the if statement to prevent the problem.
A generalized version of this is either
enum A : u8 {
ONE = 1;
TWO = 2;
}
enum B : u8 {
THREE = 3;
FOUR = 4;
}
struct T {
A a;
B b;
if (a == ONE) {
if (b == THREE) {
u8 one_three;
}
}
else if (a == TWO) {
u8 two;
}
}
or
struct T {
A a;
B b;
if (A == ONE) {
u8 one;
}
else {
if (b == THREE) {
u8 other_three;
}
}
}
It is slightly unclear what the resulting Rust code should look like in these cases.
Some wrath messages like SMSG_INSPECT_TALENT depend on a container with a length and position based on the set bits of an int, much like the UpdateMask and PackedGuid.
As far as I can tell, there's no way to directly construct a Realm_RealmFlag
from a u8 and Realm_RealmFlag_SpecifyBuild
. This means that converting previously stored realm flags (from the DB) into the correct struct types for sending to the client requires manually checking the mask and constructing the Realm_RealmFlag
struct bit by bit using the builder methods.
I think that either inner
and specify_build
should be public, or there should be a .new(inner: u8, specify_build: Realm_RealmFlag_SpecifyBuild) -> Self
function.
I imagine this is probably an issue for subtypes on other messages as well?
As the title says, with the wrath
feature, I can't call ServerOpcodeMessage::read_encrypted
, because that leads to a stack overflow exception and the problem is not(!) the control flow (as in endless recursion).
When inspecting with the debugger, we're just stuck at the gigantic match statements in read_opcodes
.
When compiling in release, this is bypassable, as well as when using RUSTFLAGS= "-Clink-arg=/STACK:8000000"
(8 MiB, I guess the default is 2MiB?).
Nevertheless, I took a glance using the following debug flag RUSTFLAGS = "-Zprint-type-sizes"
print-type-size type: `std::ops::ControlFlow<std::result::Result<std::convert::Infallible, errors::ParseError>, world::wrath::smsg_inspect_talent::SMSG_INSPECT_TALENT>`: 2880 bytes, alignment: 8 bytes
print-type-size variant `Continue`: 2880 bytes
print-type-size field `.0`: 2880 bytes
print-type-size variant `Break`: 72 bytes
print-type-size padding: 8 bytes
print-type-size field `.0`: 64 bytes, alignment: 8 bytes
print-type-size type: `std::result::Result<world::wrath::opcodes::ServerOpcodeMessage, errors::ExpectedOpcodeError>`: 2880 bytes, alignment: 8 bytes
print-type-size variant `Ok`: 2880 bytes
print-type-size field `.0`: 2880 bytes
print-type-size variant `Err`: 72 bytes
print-type-size padding: 8 bytes
print-type-size field `.0`: 64 bytes, alignment: 8 bytes
print-type-size type: `std::result::Result<world::wrath::smsg_inspect_talent::SMSG_INSPECT_TALENT, errors::ParseError>`: 2880 bytes, alignment: 8 bytes
print-type-size variant `Ok`: 2880 bytes
print-type-size field `.0`: 2880 bytes
print-type-size variant `Err`: 72 bytes
print-type-size padding: 8 bytes
print-type-size field `.0`: 64 bytes, alignment: 8 bytes
print-type-size type: `std::result::Result<world::wrath::smsg_inspect_talent::SMSG_INSPECT_TALENT, errors::ParseErrorKind>`: 2880 bytes, alignment: 8 bytes
print-type-size variant `Ok`: 2880 bytes
print-type-size field `.0`: 2880 bytes
print-type-size variant `Err`: 48 bytes
print-type-size padding: 8 bytes
print-type-size field `.0`: 40 bytes, alignment: 8 bytes
print-type-size type: `world::wrath::opcodes::ServerOpcodeMessage`: 2880 bytes, alignment: 8 bytes
print-type-size variant `SMSG_INSPECT_TALENT`: 2880 bytes
print-type-size field `.0`: 2880 bytes
print-type-size variant `SMSG_PARTY_MEMBER_STATS`: 1672 bytes
print-type-size padding: 8 bytes
print-type-size field `.0`: 1664 bytes, alignment: 8 bytes
print-type-size variant `SMSG_PARTY_MEMBER_STATS_FULL`: 1672 bytes
print-type-size padding: 8 bytes
print-type-size field `.0`: 1664 bytes, alignment: 8 bytes
print-type-size variant `SMSG_NPC_TEXT_UPDATE`: 656 bytes
print-type-size padding: 8 bytes
print-type-size field `.0`: 648 bytes, alignment: 8 bytes
print-type-size variant `SMSG_QUEST_QUERY_RESPONSE`: 608 bytes
print-type-size padding: 8 bytes
print-type-size field `.0`: 600 bytes, alignment: 8 bytes
print-type-size variant `SMSG_ACTION_BUTTONS`: 580 bytes
print-type-size padding: 2 bytes
print-type-size field `.0`: 578 bytes, alignment: 2 bytes
print-type-size variant `SMSG_ITEM_QUERY_SINGLE_RESPONSE`: 544 bytes
print-type-size padding: 8 bytes
print-type-size field `.0`: 536 bytes, alignment: 8 bytes
print-type-size variant `SMSG_TRADE_STATUS_EXTENDED`: 536 bytes
print-type-size padding: 8 bytes
print-type-size field `.0`: 528 bytes, alignment: 8 bytes
print-type-size variant `SMSG_GUILD_QUERY_RESPONSE`: 296 bytes
print-type-size padding: 8 bytes
print-type-size field `.0`: 288 bytes, alignment: 8 bytes
print-type-size variant `SMSG_QUESTGIVER_QUEST_DETAILS`: 288 bytes
print-type-size padding: 8 bytes
print-type-size field `.0`: 280 bytes, alignment: 8 bytes
print-type-size variant `SMSG_QUESTGIVER_OFFER_REWARD`: 256 bytes
print-type-size padding: 8 bytes
print-type-size field `.0`: 248 bytes, alignment: 8 bytes
print-type-size variant `SMSG_GAMEOBJECT_QUERY_RESPONSE`: 248 bytes
print-type-size padding: 8 bytes
print-type-size field `.0`: 240 bytes, alignment: 8 bytes
I've cut the majority of the really verbose output, but since inspect talent is 2.8kB, that's apparently the stack allocated size for that match/arm? It somehow needs to accumulate more, but in general maybe we can do something about how the match statements are designed? If we're commonly pushing hundreds of bytes, it may be worth to box the result? that way the match only needs to prepare usize
bytes for the result type, then we'd still have pushing the data from read_body.
Actually, I have no idea on how to analyze this further, though.
Unclear why, but generated code makes rust analyzer very slow and unresponsive
from #77
SpellCastTargets
for Vanilla/TBC requires using TRADE_ITEM
multiple times in different elseif blocks which is unsupported by the Rust codegen.
When sending a SMSG_OBJECT_UPDATE containing a Values update, if the UpdateMask is not set to be fully dirty, the client does nothing upon receiving it.
A minimal example of this problem can be found here. It is a modified version of the wrath example to contain a "goat loop", a loop that transforms the player character into a goat and back every 2 seconds.
Commenting out this line, which removes the dirtyness of the update mask, is the difference between working and not working.
@bigglesss confirmed that this happens for both 3.3.5 and 1.12.
I know that something like the goatloop should work, I used to have that working in wrath-rs a long time ago before I refactored to use wow_message. You can check out this commit for reference.
Expected behaviour:
The client should accept Values Updates, and show the new values, even when the values update only sends data that has recently changed. Marking the values update as fully dirty should not be required to convey the update to the client.
There are variables where the range of valid values can be narrowed.
It is not necessary to allocate space for a statically sized message. This could reduce the amount of allocations in a running system.
In the Wrath MovementBlock
the first few fields are identical to the fields of the MovementInfo
.
Emulators send an actual MovementInfo
but we have to duplicate the fields because we can't reach into the structs.
We want to be able to do
flag A : u8 {
ONE = 1;
TWO = 2;
}
flag B : u8 {
FOUR = 4;
EIGHT = 8;
}
struct T {
A a;
}
struct Y {
T t;
if (t.a & ONE) {
u8 one;
}
}
It is not certain what the generated Rust could should look like.
For example https://github.com/gtker/wow_messages/blob/22842fa515be36cc2426d6df8702e611be35c671/wow_message_parser/wowm/world/spell/cmsg_cancel_cast.wowm has #tag_all versions "1 2 3";
and the tests have versions = "1.12";
. This is confusing (and undocumented).
from #77
If statements for the same type separated by other types will incorrectly write both blocks twice due to how the folding with RustTypes work.
For example, with b == T2
:
struct T {
Test b;
if (b == T1 || b == T2) {
u8 t1;
}
u8 basic;
if (b == T2) {
u8 t2;
}
}
Will incorrectly write
Test b
u8 t1
u8 t2
u8 basic
u8 t1
u8 t2
This manifests in SMSG_MESSAGECHAT
for Wrath with achievement text.
At the very least SMSG_MESSAGECHAT
for TBC seems to require this.
There is only one place in .wowm (as far as I can see) where array doesn't use comma separation
The comma is listed as optional in https://github.com/gtker/wow_messages/blob/main/wow_message_parser/src/auth.pest#L91
I would propose that:
Set time to 2022-08-13 (Wednesday) 08:10
This is wrong, because 2022-08-13 is Saturday. Should confirm if wrong date+weekday combination is valid to have in packet.
The Wrath MovementInfo
requires
struct MovementInfo {
MovementFlags flags;
ExtraMovementFlags extra_flags;
u32 timestamp;
Vector3d position;
f32 orientation;
if (flags & ON_TRANSPORT) {
TransportInfo transport;
if (extra_flags & INTERPOLATED_MOVEMENT) { /* <-- This */
u32 transport_time;
}
}
Test case is
flag A : u8 {
ONE = 1;
TWO = 2;
}
flag B : u8 {
FOUR = 4;
EIGHT = 8;
}
struct T {
A a;
B b;
if (a & ONE) {
if (b & FOUR) {
u8 one_four;
}
}
}
It is unknown exactly what the example should look like in generated Rust code.
vmangos limits the size of messages to 10240
. This may be worth checking for in the Rust library.
So allowed = 0
and allowed = 1
both mean that Warrior/Human are allowed to use the item.
It's counter intuitive to have to check both allowed.is_empty() && allowed.is_MY_THING()
instead of just allowed.is_MY_THING()
.
Could possibly add a tag for flags that means that inner == 0
is also valid.
wow_messages/wow_world_messages/src/util/functions/base.rs
Lines 19 to 31 in 22842fa
size
is user provided u32, it can cause allocation of 4GB of memory.
Possible fixes:
CString
size.min(some_reasonable_max_size)
0x7f_ff_ff // max size in header
- 2 // min opcode size in header
- 4 // sized string's length
from #77
Wrath of the Lich Ling :)
Currently these tags are being used on fields of messages and structures, looks like they could be simplified and deduplicated.
Perhaps also a list of valid tags could be added and validated against?
maximum_length
SMSG_PARTY_COMMAND_RESULT
CString
str.len() <= tag
contains(&body_size)
check)max_length
SMSG_GMRESPONSE_RECEIVED
Cstring
and CString[4]
str.len() <= tag
maximum_length
?contains(&body_size)
check)maximum_value
CMSG_WHO
u32
val <= tag
maximum_valid_value
SMSG_QUESTUPDATE_ADD_ITEM
u32
value <= tag
maximum_value
?minimum_valid_value
SMSG_QUESTUPDATE_ADD_ITEM
u32
value >= tag
minimum_legal_value
CMSG_SPLIT_ITEM
u32
val >= tag
minimum_valid_value
?valid_range
u8
, u32
value >= tag[0] && value <= tag[1]
compressed
u8[-]
CMD_SURVEY_RESULT.data
?For 3.3.5 SMSG_ACCOUNT_DATA_TIMES
does not survive parse->encode round trip.
SMSG_ACCOUNT_DATA_TIMES::read_inner
reads an an extra u32 (data_decompressed_size
) before reading data
.
The .wowm and also SMSG_ACCOUNT_DATA_TIMES::write_into_vec
do not have this u32
wow_messages/wow_world_messages/src/world/wrath/smsg_account_data_times.rs
Lines 135 to 140 in f05ce8c
I think data_decompressed_size
is incorrect, as I can't find it in other implementations and CacheMask::PerCharacterCache
has 5 bits set, so data
should have 5 elements (see comment for data
in .wowm).
Here is a failing test case for this bug:
#[test]
fn reencode_smsg_account_data_times_char_mask() {
use std::io::Cursor;
use wow_world_messages::wrath::{
expect_server_message, ServerMessage, SMSG_ACCOUNT_DATA_TIMES,
};
let original: Vec<u8> = vec![
0x00, 0x1f, 0x09, 0x02, // packet header
0xf3, 0xb3, 0x16, 0x65, // unix_time
0x01, // unknown 1
0xea, 0x00, 0x00, 0x00, // mask (5 bits set, so data should have 5 elements)
0xe3, 0xaf, 0x16, 0x65, // data?, currently skipped as data_decompressed_size
0x00, 0x00, 0x00, 0x00, // data
0x00, 0x00, 0x00, 0x00, // data
0x00, 0x00, 0x00, 0x00, // data
0x4a, 0xa0, 0x15, 0x65, // data
];
let msg = {
let mut cursor = Cursor::new(&original);
expect_server_message::<SMSG_ACCOUNT_DATA_TIMES, _>(&mut cursor).unwrap()
};
let reencoded = {
let mut buf = Vec::new();
msg.write_unencrypted_server(&mut buf).unwrap();
buf
};
// err: original is 4 bytes longe than reencoded
assert_eq!(original, reencoded);
let reparsed = {
let mut cursor = Cursor::new(&reencoded);
expect_server_message::<SMSG_ACCOUNT_DATA_TIMES, _>(&mut cursor).unwrap()
};
// err: original.data.len() is 4 vs 3 for reparsed
assert_eq!(msg, reparsed);
}
The comment states "For every bit that is 1 in the mask, write one u32 with the time", so it should really be u32[mask.count_ones()]
, I don't think wowm currently supports such array length description, but maybe should be added?
When I try to read the MOTD of an AzerothCore 3.3.5a Server, I only get the first letter of the MOTD Message.
This is, because the MOTD apparently is not one SizedCString
but a sized array of nul-terminated (at least in my observings) strings:
https://github.com/TrinityCore/TrinityCore/blob/3.3.5/src/server/game/Motd/ServerMotd.cpp#L37
Mangos TBC seems to have a matching implementation:
https://github.com/cmangos/mangos-tbc/blob/49deff938c4652417de120cef23452c3fd5df03b/src/game/Server/WorldSession.cpp#L837
Trinity claims it's "new in 2.0.1", so may affect TBC as well.
ImplFeature
s currently are not completely correctly providing a list of required features.
Adding examples would help other languages plan out their layouts without having to dive directly into the larger messages.
This is unnecessary and discouraged by the library guidelines.
Some flags have UNKNOWNX
enumerators as stand ins for unknown values.
Because of the write_into_vec
interface that is used an intermediate buffer is required.
Some types (for example "spell") are not passed to IR
from #77
Having
types make it easier to write library functions for things taking these types as well as making it more difficult to accidentally passing the incorrect thing.
The Wrath SMSG_ADDON_INFO
structure relies on external knowledge to parse the correct amount of Addon
s.
smsg SMSG_ADDON_INFO = 0x2EF {
u32 number_of_addons {
skip_serialize = "true";
}
Addon[number_of_addons] addons;
u32 number_of_banned_addons = 0;
}
To work around this, the skip_serialize
tag is used on a dummy variable that tells it how many Addon
s to send. This works fine for sending from the server, since the rust code just sends as many Addon
s as are in the Vec<Addon>
but it does not work for reading the message.
Since the generated message is never valid it has instead been replaced by a panic.
A way should be found that removes the panic either through removing the ability to read SMSG_ADDON_INFO
or finding a workaround to providing the knowledge of how many Addon
s. The amount of Addon
s to parse is the same as in CMSG_AUTH_SESSION
.
The compressed messages have unique control flow that could likely do with some thorough testing, and it would especially useful for downstream users.
It might make sense to have the "same" test for both compressed and uncompressed version of a message.
Love the library - lots of great work. Would you have any interest in creating an executable for realmd? I see the crate for wow_messages but as far as I can tell its just a library.
Looking to learn Rust so happy to contribute this if interested
Edit: please ignore, this makes no sense (and I need to sleep). Thanks for the great lib!
Hi!
I see that all the packets use specific tokio (and async-std) read and write functions. Would you be open to a PR that implements AsyncRead and AsyncWrite for each of these instead? That way I could just send the packet over the stream directly rather than having to call a function first.
Cheers
Alex
The bitwise And/Or operators make sense for flag types so they should probably be implemented.
SMSG_SEND_MAIL_RESULT
and SMSG_AUCTION_COMMAND_RESULT
use the same type twice in their bodies which the Rust codegen cannot handle.
The decompressed size field should only be used as an optimization for decompressing.
There are messages where the exact layout couldn't be determined which should be investigated.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.