Code Monkey home page Code Monkey logo

yojimbo's Introduction

Build status

yojimbo

yojimbo is a network library for client/server games written in C++.

It's designed around the networking requirements of competitive multiplayer games like first person shooters.

image

It has the following features:

  • Cryptographically secure authentication via connect tokens
  • Client/server connection management and timeouts
  • Encrypted and signed packets sent over UDP
  • Packet fragmentation and reassembly
  • Bitpacker and serialization system
  • Reliable-ordered messages and data blocks
  • Estimates of packet loss, latency and bandwidth usage

yojimbo is stable and production ready.

Source Code

You can get the latest source code by cloning it from github:

  git clone https://github.com/mas-bandwidth/yojimbo.git

Alternatively, you can download the latest release.

Author

The author of this library is Glenn Fiedler.

Other open source libraries by the same author include: netcode, reliable, and serialize

If you find this software useful, please consider sponsoring it. Thanks!

License

BSD 3-Clause license.

yojimbo's People

Contributors

ananace avatar ataulien avatar dbechrd avatar drichardson avatar ekajarmstro avatar fingolfin1196 avatar fredgithub avatar g07cha avatar gafferongames avatar geneotech avatar gpakosz avatar green-sky avatar jorgenpt avatar kbirk avatar kkimdev avatar lukesanantonio avatar manpat avatar maxweisel avatar mckillroy avatar mgriley avatar mikedanylchuk-h13 avatar mohammad-ahmadi-de avatar msinilo avatar nullsoldier avatar r4stered avatar sathwikmatsa avatar sherief avatar tldquantic avatar vavius avatar wangchuan3533 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

yojimbo's Issues

Problem with matcher on Windows platform with latest code

I'm seeing some strange behavior where mbedtls is receiving a truncated https response from the matcher.go. I haven't seen this before and have no idea why it has started happening today (no changes made to yojimbo).

I updated to a latest Docker version and noticed this today, so I think it might be related to that.

I'm still investigating and don't know what it is yet so I'm putting this up so people know what's going on if they see this themselves.

Symptoms are:

"premake5 connect" fails to connect and says "failed to connect to matcher, is the matcher running?" even though the matcher is running.

Windows only. Seems fine on MacOS

Compatibility with iOS

Able to build on iOS but run into runtime issues.

Run into memory access issues (EXC_BAD_ACCESS) involving the allocator/deallocator macros, e.g.

#define YOJIMBO_NEW( a, T, ... ) ( new ( (a).Allocate( sizeof(T), __FILE__, __LINE__ ) ) T(__VA_ARGS__) )

Stack trace:

#0    0x0000000103c6b7fc in yojimbo::ClientServerPacketFactory::CreatePacket(int) at /Users/Dom/Desktop/libyojimbo/./yojimbo_client_server_packets.h:263
#1    0x0000000103c7fb04 in yojimbo::PacketFactory::Create(int) at /Users/Dom/Desktop/libyojimbo/yojimbo_packet.cpp:266
#2    0x0000000103c64b54 in yojimbo::Client::CreatePacket(int) at /Users/Dom/Desktop/libyojimbo/yojimbo_client.cpp:305
#3    0x0000000103c655f0 in yojimbo::Client::SendPackets() at /Users/Dom/Desktop/libyojimbo/yojimbo_client.cpp:321
#4    0x00000001000596d8 in ::copr_pump_frame(double *, double) at /Users/Dom/Desktop/interspace-ios/Classes/copr_client.cpp:268
#5    0x0000000100059510 in ::copr_connect(double *, double, uint32_t, uint64_t, const char *, uint16_t) at /Users/Dom/Desktop/interspace-ios/Classes/copr_client.cpp:125
#6    0x00000001001e9b20 in ::CoprClient_copr_connect_m3420134110(Il2CppObject *, double *, double, uint32_t, uint64_t, String_t *, uint16_t, const MethodInfo *) at /Users/Dom/Desktop/interspace-ios/Classes/Native/Bulk_Assembly-CSharp_0.cpp:23636
#7    0x00000001001e668c in ::CoprClient_CoprThreadRun_m141907639(Il2CppObject *, const MethodInfo *) at /Users/Dom/Desktop/interspace-ios/Classes/Native/Bulk_Assembly-CSharp_0.cpp:23067
#8    0x00000001031a4b54 in ::ThreadStart_Invoke_m225334093(ThreadStart_t124146534 *, const MethodInfo *) at /Users/Dom/Desktop/interspace-ios/Classes/Native/Bulk_mscorlib_7.cpp:45558
#9    0x0000000103927838 in RuntimeInvoker_Void_t2863195528(MethodInfo const*, void*, void**) at /Users/Dom/Desktop/interspace-ios/Classes/Native/Il2CppInvokerTable.cpp:1065
#10    0x00000001045ff064 in il2cpp::vm::Runtime::DelegateInvoke(Il2CppDelegate*, void**, Il2CppException**) ()
#11    0x00000001045d9248 in ::RunThreadWrapper() at /Users/builduser/buildslave/unity/build/Runtime/Threads/Thread.cpp:57
#12    0x00000001045e16e0 in ::RunThreadWrapper() at /Users/builduser/buildslave/unity/build/Runtime/Threads/Thread.cpp:52
#13    0x00000001045e74ec in il2cpp::os::ThreadImpl::ThreadStartWrapper(void*) ()
#14    0x000000018480575c in _pthread_body ()
#15    0x000000018480566c in _pthread_start ()
#16    0x0000000184802d84 in thread_start ()

From debugger:

In a hack, replaced the allocator macros with just plain new and delete which seems to get around the EXC_BAD_ACCESS issue, but clients are unable to establish connections with the server.

The code works on Mac (x86_64), Linux (x86_64) and Windows (x86_64).

Understanding reliable ordered messages

Hello Glenn! Thanks for the killer libraries. Just updated to your latest commit. Cleaned up so nicely.

I've been implementing a Unity plugin with yojimbo. Works really well overall. My biggest hangup is understanding how to tune the ordered reliable channel. For example, with unordered I can get a smooth 90 msgs per sec (256 bytes). With ordered, I am seeing a limit of 8 msgs per second for a 120ms server and about 60 msgs per second on LAN. To get the fastest rate I had to tweak the main loop Hz, min resend time and send rate. I've read your articles before starting. Maybe I will re-read them. Otherwise, any hints would be greatly appreciated!

Geo

Difference between this and netcode.io

I understand that this is built on netcode.io and reliable.io but I'm not sure what exactly this provides over those two libraries.

Also would either this or netcode.io be suitable for a turn based game like worms? From my understanding netcode.io would be but im not sure what yojimbo offers in addition... is it more action game focused?

BlockMessage + Reliable Channel

Heyo,

Apologies for using github issues to ask questions. For the most part I've been able to solve everything from looking at the source, but this one is a little trickier.

In short, does it make sense to send variable length reliable RPC messages using BlockMessage? I know you've previously mentioned that these messages are generally better for downloading a large chunk of data once (like downloading a level) because they only allow one to be in-flight at a time. So I assume the answer is "don't use BlockMessage" for this, but I figured I'd double check.

The longer version of the story is that I have an app that sends a large amount of reliable RPC messages (90-180/sec). Each RPC contains a small buffer of data that varies in size. Most of the time, these buffers are very small (20-40 bytes), however, occasionally we'll see a single message here and there that's larger than the WriteStream buffer (8 * 1024 according to the default connection config).

Architecturally, I'd love to just have a single message type that is a subclass of BlockMessage. However, it looks like I should probably make two subclasses. One that's a subclass of Message and one BlockMessage that I use whenever the buffer is larger than the WriteStream buffer size.

Anyway, I wasn't sure if Yojimbo makes an exception if the entire block message fits inside of a single packet.

Max

Language bindings

Hi, is there any plan to consider other language bindings in the future? Since there are also some other languages used in game dev, I think it will be useful to have those or plan so that it's easier to create bindings(C interface?)

Personally I will be using Rust and willing to work on the binding.

Puzzled about the server AdvanceTime, why initialize to 100.0

In addition, I suggest changing platform_sleep for a varied time like this.

`const int logicFrame = 60;
double accumilatedTime = 0;
double deltaTime = platform_time();
double logicFrameLength = 1.0/logicFrame;

while( !quit )
{
    double now = platform_time();
    accumilatedTime += now-deltaTime;
    deltaTime = now;

    while( accumilatedTime > logicFrameLength )
    {
        server.SendPackets();

        serverTransport.WritePackets();

        serverTransport.ReadPackets();

        server.ReceivePackets();

        server.CheckForTimeOut();

        time += logicFrameLength;

        server.AdvanceTime( time );

        serverTransport.AdvanceTime( time );

        accumilatedTime -= logicFrameLength;
    }

    platform_sleep( logicFrameLength-accumilatedTime );
}`

Add allocateNetworkSimulator flag to NetworkTransport

Currently there is no way to completely disable simulator when using NetworkTransport.

Maybe worth adding that flag to NetworkTransport constructor, but I'm not sure about argument order - if allocateNetworkSimulator flag is the last, then I have to specify all other buffer size related args.

More options:

  • Allocate it at runtime when SetNetworkConditions is called and delete in ClearNetworkConditions.
  • Compile time switch.

NetworkSimulator eats up significant amount of memory and drains some CPU even when network conditions at defaults.

Why only 64 players per instance?

I'm genuinely curious (as some games will demand more than 64 players per area):

  • what types of games are you catering to given this limit?
  • how difficult is it to change this limit? For example, is it sufficient to change the value of MaxPlayers?
  • There are obviously bandwidth considerations. I'm trying to read through the code to see what these are.

No notification when messages are dropped due to budget

Currently, if you try to send a message with a size that happens to exceed the packet budget on it's channel, then the message itself will be silently dropped after failing to fit into a packet.
This is impossible to see from the applications point of view, as no counters are affected, no error state is set, and the packet itself sends correctly - albeit without the oversized message.

Some kind of notification when this happens would be appreciated, instead of the oversized message silently disappearing into the ether.

Issue with mobile networks and ipv6

Due to apparent issues with mobile networks and ipv4, I am attempting to get a client to connected to a server using ipv6. Using my local ipv6 address I can't seem to get the insecure server to see any traffic from the insecure client. I can, however, get the server to detect garbage UDP packets while running from the ipv6 address. I pass the same address to both the client and the server and the client times out while attempting to connect. The same results occur while trying to connect to a remote server.

I can connect successfully using the while running using ipv4, both locally and remotely.

Can you see any obvious flaws that I'm missing?

Linker error building on OSX 10.9.5

Encountered the following error while running premake5 gmake && make all:

Building configurations...
Running action 'gmake'...
Generated Makefile...
Generated test.make...
Generated connect.make...
Generated info.make...
Generated yojimbo.make...
Generated client.make...
Generated server.make...
Generated secure_server.make...
Generated client_server.make...
Generated soak.make...
Generated profile.make...
Generated simple_messages.make...
Done (43ms).
==== Building yojimbo (debug_x64) ====
Creating bin
Creating obj/x64/Debug/yojimbo
yojimbo.cpp
yojimbo_address.cpp
yojimbo_channel.cpp
yojimbo_client_server.cpp
yojimbo_common.cpp
yojimbo_connection.cpp
yojimbo_encryption.cpp
yojimbo_matcher.cpp
yojimbo_network.cpp
yojimbo_packet.cpp
yojimbo_packet_processor.cpp
yojimbo_platform.cpp
yojimbo_simulator.cpp
yojimbo_sockets.cpp
yojimbo_transport.cpp
Linking yojimbo
==== Building test (debug_x64) ====
Creating obj/x64/Debug/test
test.cpp
Linking test
ld: warning: directory not found for option '-L/usr/lib64'
ld: unknown option: --start-group
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[1]: *** [bin/test] Error 1
make: *** [test] Error 2

From what I can tell, there isn't a lib64 and clang targeting Darwin doesn't support --start-group/--end-group on OSX.

premake5 --version outputs: premake5 (Premake Build Script Generator) 5.0.0-alpha9
make -v outputs: GNU Make 3.81
clang -v outputs:

Apple LLVM version 5.1 (clang-503.0.40) (based on LLVM 3.4svn)
Target: x86_64-apple-darwin13.4.0
Thread model: posix

Messages, Packets, Frames and Time

I'm somewhat confused about how messages are carried on packets with regard to the frame concept and time.

Let's say I setup an unreliable, unordered channel between client and server. I want my client to send one message every 20 ms to the serve -- the message size is smaller than the max packet size. After the connection and channel is established between client and server, I would think only one packet would be need to be sent to carry that message.

Based on the code the function PumpClientServerUpdate() in tests/test.cpp, I made this test:

Example client code:

// After connection established
...
client->SendMsg(message);
client->SendPackets();
transport->WritePackets();
...

Example server code:

// After connection established
...
while (true) {
    ...
    transport.ReadPackets();
    server.ReceivePackets();
    ...
}

In my tests, the message is not received by the server when only one packet is sent from the client. In fact, it requires ~300 packets to be sent from the client before the server is able to receive the message.

What's the correct usage of SendMessage() vis-a-vis SendPackets() for a game frame update?
It seems that SendMessage() queues up the message and SendPackets() adds to the transport's packet queue a single connection packet with a single message from the message queue. But in my tests, it seems fore every SendMessage() called, I have to call SendPackets() ~300 times for the message to be received.

Also if I want to send multiple messages every frame (calling SendMessage() multiple times), it seems that a single call to SendPackets() every frame will only send to the transport a single packet with a single message during that frame.

Practically speaking, If I want to send multiple messages every game frame update, do I have to call SendPackets() at a higher frequency than the frame update frequency?

ignored connection request packet. connect token expired

Hi again,

A new problem arises, using InsecureConnect, with every server that is not hosted on my machine, the server will spam
"ignored connection request packet. connect token expired"
when I try to connect to it, other people are managing to connect to the server.
Any idea what could be causing this ?

Client log:
client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client sent connection request packet to server client connect failed. connect token expired client disconnected client changed state from 'sending connection request' to 'connect token expired'

Thanks,
Max

Edit: Added client log

Premake for Windows

I wanted to use the build guildlines outlined in BUILDING.md, but I am unable to create the .sln file from premake. (I maybe doing it wrong)

I installed premake and pulled the latest release (0.6).
Opened up CMD and cd to yojimbo.
Then I typed 'premake5 solution'.

When it runs, I get an error that states: "Windows cannot find 'Yojimbo.sln'. Make sure you typed the name correctly, and then try again."

Am I suppose to make a .sln first? I am using VS 2015 and/or 2017

Running in docker with mismatching ports

Hello,

I am trying to run a simple server in a docker instance, mapping the same port in the host machine as the container's binding port works like a charm.
docker run -it -p 10578:10578/udp is perfect.
As soon as I try to run
docker run -it -p 10579:10578/udp it won't work anymore.

It is something I require to run multiple containers on the same machine, is there something in yojimbo that might be causing this behavior ?

Thanks,
Max

Flow control/congestion avoidance + other questions

  1. Is there any kind of flow-control/congestion avoidance in libyojimbo?

    This post talks about a simple binary congestion avoidance algorithm, but it doesn't seem to be implemeneted in libyojimbo; the packet flow rate seems to be controlled by the frequency in which clients/server call SendPackets()/WritePackets().

  2. Until "snapshots/transport-level packet fragmentation" is implemented, it seems that the maxPacketSize for a Connection should be configured to be less than the MTU, yes? I'm a bit thrown off because the DefaultMaxPacketSize = 4 * 1024; is larger than standard MTU of 1500.

  3. In the code documentation for ReliableOrderedChannel, it mentions:

    Only one message block may be in flight over the network at any time, so blocks stall out message delivery slightly.

    Does single message block in flight restriction hold also for unreliable-unordered channels as well?

Connect vs InsecureConnect

Heyo,

I just wanted to check that the only difference between the secure_client/secure_server and the regular client/server examples is the token from the matcher.

I'm using yojimbo for a game where the sessions can be quite long in duration (possibly days). I have an accounts service that generates a session token (instead of a matcher instance). The user then connects to a yojimbo instance via client.InsecureConnect() and gives the server its session token (which is then used to verify if that user can join this match). I do this because the session token needs to be continually evaluated to make sure it's valid throughout the duration of the session. (Someone can get banned mid-game)

Is this still going to be secure? As you can imagine calling client.InsecureConnect scares me a little. I assume data is still being encrypted with libsodium.

Max

Potential (very unlikely) Overflow in WouldReadPastEnd

bool WouldReadPastEnd( int bits ) const
{
    return m_bitsRead + bits > m_numBits;
}

In WouldReadPastEnd it is possible to overflow the lhs of the comparison if a crafted packet gives an extremely large value for bits. This is pretty much impossible in the codebase as-is. The only place where the user can give a large bits value is serialise_string and only if the buffer_size parameter is unreasonably large.

However it seems that it's probably best to remove the potential overflow from WouldReadPastEnd.

Similarly bits is signed, so a negative number would also cause the same issues, however I think it's quite a lot easier to screw up an upper bound than it is to accidentally allow negative lengths though.

Something like the following would address both issues, provided I've not messed up along the way?

bool WouldReadPastEnd( unsigned bits ) const
{
    return bits > (unsigned)(m_numBits - m_bitsRead);
}

Token failed to decrypt

I ran into a snag trying to make the project work with Unity.
I got as far as getting a connection token from the "web backend" only for my connection request to time out.

We used the matcher to generate the token but when we attempt to connect using it the server outputs the following message:
"ignored connection request packet. connect token failed to decrypt"

Documentation?

The repository already has a few pieces of sample code, but seems to lack any other kind of documentation.

Rename serialize macros

Macros from yojimbo_serialize.h look like functions in namespace, but defines ignore namespaces. I get name clash because of write_int, write_float, etc (a really common names).

Maybe rename all of them to UPPERCASE with yojimbo prefix to match the style?

yojimbo_bitpack.h should check NDEBUG?

in yojimbo_bitpack.h, there are few assert with m_numWords. m_numWords is defined when DEBUG is defined in the same file, but seems like assert is specified to check NDEBUG, not DEBUG. http://en.cppreference.com/w/cpp/error/assert

so should we change to

#ifndef NDEBUG
        int m_numWords;
#endif // #ifdef NDEBUG

I don't like double negative, but seems like this is more correct way? I encountered this issue when I was compiling yojimbo with Rust's cpp interop crate.

Example usage

I'm trying to understand how to use the library, but have found the myself confused by the example client and server in tests/.

In the loop in tests/client.cpp (similar one in tests/server.cpp), the client establishes a connection to the server and continuously they then continuously exchange packets of type TEST_PACKET_CONNECTION. No user packets are exchanged however (type=TEST_USER_PACKET).

    while ( !quit )
    {
        client.SendPackets();

        clientTransport.WritePackets();

        clientTransport.ReadPackets();

        client.ReceivePackets();

        client.CheckForTimeOut();

        if ( client.IsDisconnected() )
            break;

        time += deltaTime;

        client.AdvanceTime( time );

        clientTransport.AdvanceTime( time );

        if ( client.ConnectionFailed() )
            break;

        platform_sleep( deltaTime );
    }

Conceptually, what functionality does the client perform compared to the transport? What does client.SendPackets()do exactly as compared toclientTransport.WritePackets()?

How can user packets (type=TEST_USER_PACKET) be exchanged once the connection is setup?

Also, my understanding is that various user packet formats are defined in shared.h and a factory for these types is generated with YOJIMBO_PACKET_FACTORY_START/STOP macro block?

It'd be great to have some kind of HOWTO on using the library!

windows.h SendMessage conflict

When Windows.h is included, any function named SendMessage gets macro expanded to SendMessageW or SendMessageA.

I'm not sure if this is something you want to handle in the library or let the user deal with, just seems like something other people would hit and be confused about like me.

I personally worked around this by using a MSVC pragma and renaming the function in my inherited Client.

#pragma push_macro("SendMessage")
#undef SendMessage

void GameClient::SendGameMessage( GameMessage* Message )
{
    Assert( IsConnected() );
    Assert( m_messageFactory );
    Assert( m_connection );
    m_connection->SendMessage( Message );
}
#pragma pop_macro("SendMessage")

Static Linking Errors.

Hey guys,
It's been a long time so I've worked in C/C++ so please bear with me. I've started working on a proof-of-concept for some hobbyist game code using yojimbo. It compiles and runs well on my Windows development box using your premake5 compile options for VC++. Unfortunately, when I try running it on any other machine it crashes on startup saying that it's missing the appropriate VC++ runtimes. Since installing the runtimes doesn't seem to fix it, I instead tried compiling & linking staticly (using /MT and /MTd rather than /MD and /MDd) to bake any missing runtimes into the .EXE and thus avoid the problem entirely, and now the wheels are coming off the bus. I'm obviously doing something really silly cause I'm getting errors like these...

...numerous warnings...
2>yojimbo.lib(dhm.obj) : error LNK2019: unresolved external symbol __imp_fread referenced in function dhm_check_range
2>yojimbo.lib(hmac_drbg.obj) : error LNK2001: unresolved external symbol __imp_fread
2>yojimbo.lib(entropy.obj) : error LNK2001: unresolved external symbol __imp_fread
2>yojimbo.lib(ctr_drbg.obj) : error LNK2001: unresolved external symbol __imp_fread
2>yojimbo.lib(md.obj) : error LNK2001: unresolved external symbol __imp_fread
2>yojimbo.lib(pkparse.obj) : error LNK2001: unresolved external symbol __imp_fread
2>yojimbo.lib(md.obj) : error LNK2019: unresolved external symbol __imp_ferror referenced in function mbedtls_md_file
2>yojimbo.lib(bignum.obj) : error LNK2019: unresolved external symbol __imp_fgets referenced in function mbedtls_mpi_read_file
2>D:\Projects\FrankGame01\FrankGame01\bin\x64\Debug\FrankGame01.exe : fatal error LNK1120: 3 unresolved externals
2>Done building project "FrankGame01.vcxproj" -- FAILED.

Any suggestions? I'm sure it's something basic. Thanks in advance for any help.

  • Frank.

Build failed when link with libsodium on macOS

make all
==== Building yojimbo (debug_x64) ====
==== Building test (debug_x64) ====
Linking test
ld: library not found for -lsodium
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[1]: *** [bin/test] Error 1
make: *** [test] Error 2

macOS 10.12.3

Apple LLVM version 8.0.0 (clang-800.0.42.1)
Target: x86_64-apple-darwin16.4.0
Thread model: posix

homebrew config
HOMEBREW_VERSION: 1.1.11
ORIGIN: https://github.com/Homebrew/brew.git
HEAD: 394f9fa0aaa5854aa52bc589708a079665dcf462
Last commit: 3 weeks ago
Core tap ORIGIN: https://github.com/Homebrew/homebrew-core
Core tap HEAD: 146a4bf1580b0cc2ea22b6df6320fab43f3f3e76
Core tap last commit: 23 hours ago
HOMEBREW_PREFIX: /usr/local
HOMEBREW_REPOSITORY: /usr/local/Homebrew
HOMEBREW_CELLAR: /usr/local/Cellar
HOMEBREW_BOTTLE_DOMAIN: https://homebrew.bintray.com
CPU: quad-core 64-bit sandybridge
Homebrew Ruby: 2.0.0-p648
Clang: 8.0 build 800
Git: 2.10.1 => /Applications/Xcode.app/Contents/Developer/usr/bin/git
Perl: /usr/bin/perl
Python: /usr/bin/python
Ruby: /Users/apple/.rvm/rubies/ruby-1.9.3-p194/bin/ruby
Java: N/A
macOS: 10.12.3-x86_64
Xcode: 8.2.1
CLT: N/A
X11: 2.7.7 => /opt/X11

brew ls libsodium
/usr/local/Cellar/libsodium/1.0.12/include/sodium/ (57 files)
/usr/local/Cellar/libsodium/1.0.12/include/sodium.h
/usr/local/Cellar/libsodium/1.0.12/lib/libsodium.18.dylib
/usr/local/Cellar/libsodium/1.0.12/lib/pkgconfig/libsodium.pc
/usr/local/Cellar/libsodium/1.0.12/lib/ (2 other files)

Custom socket

I was wondering what the feasibility is of being able to use a custom socket implementation, i.e. I am working on a restricted hardware platform where instead of normal sockets, there is a vendor provided socket implementation (different functions/params but very similar), which has the same properties of UDP (however I'd like to have reliable, ordered packets). Essentially, rather than passing in an address for yojimbo to open, I'd like to open the socket myself, and then pass yojimbo my socket (as a void* or whatever), and then register functions with yojimbo for sending, receiving and closing said socket.

What are your thoughts?

Possible to support openid connect authentication flow?

Not sure how hard it is to support json web tokens (JWT) as a token authentication method to https servers.

For example being able to use Google accounts / Facebook accounts / Github accounts / Run your own openidc server with keycloak accounts.

Sequence buffer out of order

We are having an occasional issue sending ordered messages. Most of the time everything works fine for an extended run, but sometimes we are hitting the assert in ReliableOrderedChannel::AddMessagePacketEntry.

We are sending reliable ordered messages fairly infrequently and the sequence number that gets passed to the Insert function seems to be getting incremented every time Server::SendPackets is called. We can reliably reproduce the assert by waiting ~32768 frames (i.e SendPacket calls) between sending a reliable ordered message.

It seems like the sequence passed to the sequence buffer should only increment when a message is sent rather than every frame, or else reliable messages need to be sent more often. Is this a bug or are we misusing something?

Question regarding C++ style

While browsing through your code I've noticed a few odd programming choices.

For example, most of your classes have user defined destructors. Due to a very bad (and deprecated) C++ rule the compiler will implicitly generate copy ctors/assignment that do bitwise copy. If a user accidentally copies an object, they will have a double free.

Is there a special reason why you haven't disabled the copy operations?

Note that issue would have been resolved if you've used something similar to std::unique_ptr because it won't require any custom destructors.

Some of the classes are also perfect examples of C++11 move-only types. I know that game prograers have these bias against any newer C++ features but move semantics are useful.

I've also noted that you aren't using the C++ standard library but instead reimplement stuff like the min/max/randint or swap in a (partly) worse way. And you already require the C standard library and a C++ compiler, so there is the C++ library available as well.

Disclaimer: I come from /r/cpp but want to understand more of your reasons not just blindly bash you.

Cmake?

Would you accept a cmake version of the build system? If I made one? Not sure how to deal with two versions of the build system though because one be outdated.

Applicability for network audio?

Curious if libyojimbo would be appropriate for carrying voice/audio streams as an alternative to something like RTP, especially if libyojimbo is already being use to carry game state updates?

I encountered the problem of the server to handle the exception packet

Server crashed when client sent message that cannot be serialized. I think it should be compatible with this dirty data. Reading stream was broke exceptionally caused by the wrong numMessages. I've been reading your code for a week, and I'm trying to fix it. It works. Did I mess it up? This is my backtrace and modification.

(gdb) bt
#0 0x00007f6664949625 in raise () from /lib64/libc.so.6
#1 0x00007f666494ae05 in abort () from /lib64/libc.so.6
#2 0x00007f666494274e in __assert_fail_base () from /lib64/libc.so.6
#3 0x00007f6664942810 in __assert_fail () from /lib64/libc.so.6
#4 0x00000000004214b5 in yojimbo::ChannelPacketData::Free (this=0x226c220, messageFactory=...) at yojimbo_channel.cpp:39
#5 0x0000000000417e5b in yojimbo::ConnectionPacket::~ConnectionPacket (this=0x226c2b0, __in_chrg=) at yojimbo_connection.cpp:45
#6 0x000000000041c494 in yojimbo::PacketFactory::DestroyPacket (this=0x1d24200, packet=0x226c2b0) at yojimbo_packet.cpp:309
#7 0x000000000041b7d5 in yojimbo::Packet::Destroy (this=0x226c2b0) at yojimbo_packet.cpp:40
#8 0x000000000041c0a6 in yojimbo::ReadPacket (info=..., buffer=0x7fff34e07d00 "", bufferSize=93, errorCode=0x7fff34e07c98) at yojimbo_packet.cpp:239
#9 0x000000000042b284 in yojimbo::PacketProcessor::ReadPacket (this=0x1d22080, packetData=0x7fff34e07d00 "", sequence=@0x7fff34e08d50, packetBytes=93,

encrypted=@0x7fff34e08d5f, key=0x0, encryptedPacketTypes=0x1d24120 "\001\001\001\001\001\001\001\001", 
unencryptedPacketTypes=0x1d24120 "\001\001\001\001\001\001\001\001", streamAllocator=..., packetFactory=...) at yojimbo_packet_processor.cpp:237

#10 0x000000000041e9d0 in yojimbo::BaseTransport::ReadPackets (this=0x7fff34e08df0) at yojimbo_transport.cpp:367
#11 0x0000000000401c9b in ServerMain () at tests/server.cpp:145
#12 0x0000000000401e22 in main () at tests/server.cpp:181

template bool SerializeOrderedMessages( Stream & stream, MessageFactory & messageFactory, int & numMessages, Message ** & messages, int maxMessagesPerPacket )
{
const int maxMessageType = messageFactory.GetNumTypes() - 1;

    bool hasMessages = Stream::IsWriting && numMessages != 0;

    serialize_bool( stream, hasMessages );

    if ( hasMessages )
    {
        serialize_int( stream, numMessages, 1, maxMessagesPerPacket );

        int * messageTypes = (int*) alloca( sizeof( int ) * numMessages );

        uint16_t * messageIds = (uint16_t*) alloca( sizeof( uint16_t ) * numMessages );

        memset( messageTypes, 0, sizeof( int ) * numMessages );
        memset( messageIds, 0, sizeof( uint16_t ) * numMessages );

        if ( Stream::IsWriting )
        {
            assert( messages );

            for ( int i = 0; i < numMessages; ++i )
            {
                assert( messages[i] );
                messageTypes[i] = messages[i]->GetType();
                messageIds[i] = messages[i]->GetId();
            }
        }
        else
        {
            Allocator & allocator = messageFactory.GetAllocator();

            messages = (Message**) allocator.Allocate( sizeof( Message* ) * numMessages );
        }

        serialize_bits( stream, messageIds[0], 16 );

        for ( int i = 1; i < numMessages; ++i )
            serialize_sequence_relative( stream, messageIds[i-1], messageIds[i] );

        for ( int i = 0; i < numMessages; ++i )
        {
            if ( maxMessageType > 0 )
            {
                serialize_int( stream, messageTypes[i], 0, maxMessageType );
            }
            else
            {
                messageTypes[i] = 0;
            }

            if ( Stream::IsReading )
            {
                messages[i] = messageFactory.Create( messageTypes[i] );

                if ( !messages[i] )
                    return false;

                messages[i]->AssignId( messageIds[i] );
            }

            assert( messages[i] );

            if ( !messages[i]->SerializeInternal( stream ) )
            {
                numMessages = i+1;    // fix processed messages amount
                return false;
            }
        }
    }

    return true;
}

Thank you for your great job. It's helpful. I will continue to pay attention to.

Wrapper for 3D engine: enhancement for Unity 3D

I think that is very important to create a wrapper for 3D Engine: Unity 3D will be the perfect 3D engine to start.

It is possible to start a Patreon campaign where I would like to contribute.

Problem with Unreal

Hi Glenn,
I tried to integrate libyojimbo with unreal engine, after successfuly sending/receiving message with bare windows client to linux server, it's failed doing excactling the same thing in Unreal Engine tutorial scene. I did some network sniffing with Wireshark, it's weird the client side sends 10 UDP packets of 18 bytes payload to server, and server logged with "ignored encrypted packet, no read packet key for this address", any ideas? Here's my code (in Unreal).

// Copyright 1998-2017 Epic Games, Inc. All Rights Reserved.

#include "physxClient.h"
#include "physxClientGameMode.h"
#include "physxClientHUD.h"
#include "physxClientCharacter.h"
#include "../ThirdParty/libyojimbo/include/yojimbo.h"
#include "../ThirdParty/libyojimbo/include/shared.h"
#include <signal.h>

using namespace yojimbo;

AphysxClientGameMode::AphysxClientGameMode()
	: Super()
{
	// set default pawn class to our Blueprinted character
	static ConstructorHelpers::FClassFinder<APawn> PlayerPawnClassFinder(TEXT("/Game/FirstPersonCPP/Blueprints/FirstPersonCharacter"));
	DefaultPawnClass = PlayerPawnClassFinder.Class;

	// use our custom HUD class
	HUDClass = AphysxClientHUD::StaticClass();
}
double ti = 100.0;

Client* client ;

void AphysxClientGameMode::StartPlay() {
	Super::StartPlay();
	InitializeYojimbo();
	OgameMessageFactory messageFactory(GetDefaultAllocator());
	uint64_t clientId = 1;

	Address clientAddress("0.0.0.0", 6667);
	Address serverAddress("192.168.0.8", 6666);
		 
	ClientServerConfig config;
	config.numChannels = 1;
	config.channel[0].type = CHANNEL_TYPE_UNRELIABLE_UNORDERED;

	client = new Client(GetDefaultAllocator(), clientAddress, config, adapter, ti);

	uint8_t privateKey[KeyBytes];
	memset(privateKey, 0, KeyBytes);
	client->InsecureConnect(privateKey, clientId, serverAddress);
	client->AdvanceTime(ti);
}

void AphysxClientGameMode::Tick(float f) {
	Super::Tick(f);
	
	const double deltaTime = 0.1;

	JoinMessage* message = (JoinMessage*)(client->CreateMessage(JOIN_MESSAGE));
	message->teamId = 1;
	client->SendMessage(0, message);
	client->SendPackets();
	client->ReceivePackets();
	ti += deltaTime;
	client->AdvanceTime(ti);
	//messageFactory.ReleaseMessage(message);

	yojimbo_sleep(deltaTime);

	return;
}

Raspberry Pi build support?

Hi,
I tried to compile yojimbo on pi 3 for some udp communication. I managed to build premake5, sodium and mbedtls. After the execution of premake on yojimbo i tried to use make to build it but it said that there are 2 unknown flags. -m64 and -msse2. Sure, thats because of the arm architecture. I removed these flags and yojimbo compiled. But linking against the resulting libyojimbo.a file results in a linker error on calling the initialize function of yojimbo.
Would be incredible if there are plans to support raspberry pi!

Thank you very much!

Latancy in the real network environment

I made a echo server to test sending messages in the real network with 37ms delay. The latancy was as high as 80ms until I raised the loop frequency to 150 to reduce the latancy to 50ms.

Packet loss? What should I do?

Possible error in matcher, return instead of goto.

I'm curious as to why you have a return here, where all the other branches have a goto cleanup;

https://github.com/networkprotocol/libyojimbo/blob/master/yojimbo_matcher.cpp#L128

        if ( ( ret = mbedtls_ssl_config_defaults( &m_internal->conf,
                        MBEDTLS_SSL_IS_CLIENT,
                        MBEDTLS_SSL_TRANSPORT_STREAM,
                        MBEDTLS_SSL_PRESET_DEFAULT ) ) != 0 )
        {
            m_status = MATCHER_FAILED;
            return; // <-- All the other error branches are `goto cleanup;`
        }
    // ....
cleanup:
      mbedtls_ssl_close_notify( &m_internal->ssl );

I don't actually know enough about how the different mbedtls functions work to say this is a bug, but it looks suspicious to have only one error path return directly.

Endianness detection macro fails on iOS

I've grabbed the similar detection code from RapidJson. It checks for __BYTE_ORDER__ and <endian.h> first, looks more robust.

This can be tricky to debug.

Unreliable-Sequenced with Acknowledgement Channel

Hello, I'm interested in using yojimbo with an unreliable-sequenced stream of object updates with acknowledgements for which messages are actually received by the client. (Something like the UNRELIABLE_SEQUENCED_WITH_ACK_RECEIPT message type from RakNet.)

I've looked through the code, and this seems pretty hard to implement since all channels are multiplexed into one packet with one sequence number. I can easily do sequencing on the client, but I don't think I can use the current system (Channel::onAck) to detect ignored out-of-order packets on the server - there's the possibility of out-of-order acks sent by the client becoming in-order on the way to the server. If I skip sending acks from the client for out-of-order packets, I'd cause unnecessary strain on any reliable channels.

Just curious about your thoughts. If there's a reasonably easy way to do it, I might give implementing it into the library proper a shot. If not, I can just do the sequencing manually inside my messages.

Thanks for the library, good luck with the startup!

Sequence Buffer Question

Just a quick question. I was reviewing this line here: https://github.com/networkprotocol/libyojimbo/blob/master/yojimbo_sequence_buffer.h#L71

    T * Insert( uint16_t sequence )
    {
        if ( sequence_greater_than( sequence + 1, m_sequence ) )
        {
            RemoveEntries( m_sequence, sequence );

            m_sequence = sequence + 1;
        }
       
        // THIS LINE HERE
        else if ( sequence_less_than( sequence, m_sequence - m_size ) )
        {
            return NULL;
        }

        const int index = sequence % m_size;

        m_entry_sequence[index] = sequence;

        return &m_entries[index];
    }

I've been trying to understand what this is checking exactly. My best guess it to catch old packets that would possibly have wrapped around the buffer size, but have yet to wrap around a full uint16_t. Could you possibly shed a little light here? Thank you!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.