Code Monkey home page Code Monkey logo

packetserial's Introduction

PacketSerial

Build Status

An Arduino Library that facilitates packet-based serial communication using COBS or SLIP encoding.

Features

PacketSerial is an small, efficient, library that allows Arduinos to send and receive serial data packets (with COBS, SLIP or a user-defined encoding) that include bytes of any value (0 - 255). A packet is simply an array of bytes.

Documentation

If you're asking Why do I need this?, read the background introduction page. If you're ready to get started, check out the getting started page. You can also learn a lot by reading through the examples. Finally, if you're interested in learning more about how the code works, read the comments in the source code and check out the API documentation.

Support

If you're looking for help, read the support page.

Compatibility

Requrements

Other Platforms

This project has been used successfully with openFrameworks using the ofxSerial addon. In particular, see the ofx::IO::PacketSerial object. Additionally this project has been used with Python using the PySerial package. In particular, check out the COBS (see this discussion) and SLIP packages.

Ultimately, any library that correctly implements a COBS or SLIP encoding scheme should be compatible with this project.

Continuous Integration

Continuous integration tests are carried out on a variety of common Arduino platforms. See this script for a list.

Licensing

This project is licensed under the MIT License.

Project Management

Repository

https://github.com/bakercp/PacketSerial

Contributing

If you'd like to contribute to this project, please check out the Code of Conduct and the contributing guide.

Versioning

This project uses Semantic Versioning. You can check out recent changes in the changelog.

packetserial's People

Contributors

avilleret avatar bakercp avatar latonita avatar marc010 avatar per1234 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

packetserial's Issues

SLIPPacketSerial send() gives me two 0xc0 as the end of packet

I simply do slip.send(buff*,size) and it gives me nice slip packet, but with two 0xc0 in the end instead of one. this makes stream desynchronized and i loose packets on receiving side

I suppose it shall be only one 0xc0 in the end.

what i get:
[0xc0,data,0xc0,0xc0][0xc0,data,0xc0,0xc0]...

what i expect:
[0xc0,data,0xc0][0xc0,data,0xc0]...

thanks

Sending empty (0-byte) data

Sending a 0-byte data actually does nothing on PacketSerial (returns directly in the send method).
However I find it quite common to send such empty data. For example, a simple protobuf struct:

message SimpleMessage
{
    bool value = 1;
}

with its value set to false will be encoded as a 0-byte buffer by nanopb. I guess it's better to just send a PacketMarker for this kind of data.

Expected Behavior

Send a PacketMarker for 0-byte data.

Current Behavior

Does nothing on 0-byte data.

Your Environment

  • Version used: 1.4.0
  • IDE Name and version: N/A
  • Operating System and version (desktop or mobile): Arduino
  • Link to your project: N/A

Raspberry Pi Support

I would like to connect my arduino and raspberry pi together using this protocol.

Has anyone managed to create the code for the raspberry pi (with pyserial or wiringPi)

Thanks!

Teensy 3.2 - reverse echo example not echoing

Hello @bakercp thanks for this library and ofxSerial too.
I'm trying to run the examples in a Teensy 3.2 and answer is not coming back.
I've tried different baud rates and I suppose onPacket is not firing. I've tried to check turning on the built in LED inside the onPacket function.
Thanks

[No-BUG] Any idea why I get invalid packages?

Hi, thank you for this great library, it saved my day (actually month) until now.

I use this in a somewhat complex environment. My PC connected to a Arduino Mega which acts as a gateway. Mega is also connected to 4 other Arduino's (1 Uno, 2 Nano's) though 3 additional HW serial interfaces. I designed a custom protocol to send states, sensor readings, commands etc.

While debugging I find out that the handler is called although there shouldn't be a packet (I send packets rather slow, every 4 seconds).
Here is some related parts of the code on the Mega:
`
...
PacketSerial packetSerialPC;
PacketSerial packetSerialSense;
PacketSerial packetSerialLeft;
PacketSerial packetSerialRight;
...
void setup () {
// SERIAL - PC
Serial.begin(BAUDRATE_UPSTREAM);
Serial.setTimeout(DURATION_SERIAL_TO);
packetSerialPC.setStream(&Serial);
packetSerialPC.setPacketHandler(&packetSerialHandler);
// SERIAL - ARDUINO NANO (LEFT)
Serial1.begin(BAUDRATE_DOWNSTREAM);
Serial1.setTimeout(DURATION_SERIAL_TO);
packetSerialLeft.setStream(&Serial1);
packetSerialLeft.setPacketHandler(&packetSerialHandler);
// SERIAL - ARDUINO UNO (MIDDLE=SENSE)
Serial2.begin(BAUDRATE_DOWNSTREAM);
Serial2.setTimeout(DURATION_SERIAL_TO);
packetSerialSense.setStream(&Serial2);
packetSerialSense.setPacketHandler(&packetSerialHandler);
// SERIAL - ARDUINO NANO (RIGHT)
Serial3.begin(BAUDRATE_DOWNSTREAM);
Serial3.setTimeout(DURATION_SERIAL_TO);
packetSerialLeft.setStream(&Serial3);
packetSerialLeft.setPacketHandler(&packetSerialHandler);
// ....
}

void packetSerialHandler(const void* sender, const uint8_t* buffer, size_t size) {
uint8_t rec_length = buffer[0]; // length in the record
char str[64];
snprintf(str, 64, "[GOT] PACKET FROM=%d TO=%d SIZE=%d LEN=%d", buffer[1], buffer[2], size, buffer[0]);
send_text(ADDR_PC, str);
if ((size <= 3) || (size != rec_length)) { // handle possible invalid packets
return;
}
...
if (sender == &packetSerialPC) {
...
} else if (sender == &packetSerialSense) {
...
} else if (sender == &packetSerialLeft) {
...
} else if (sender == &packetSerialRight) {
...
} else {
send_text(ADDR_PC, "[ERROR] ILLEGAL SENDER!");
}
...some forwarding code...
}
`
I've been using this for a while now implementing upwards messaging, but when I started downward messages/commands (sent by a QT program) I added the "snprintf" debug lines and saw a lot of invalid packet calls which have been filtered out.

The system is on my lab desk, connected through short lines. No particular noise source.

The protocol contains a header:
Byte 0 : Length of the message sent
Byte 1 : Source address
Byte 2 : Destination address

There are 5 devices and 4 special addresses defined in an enum, so the address fields must be between 0-8.

From the debug log on the PC:
This is a valid packet:
2018.12.03 15:46:14 [B] MSG:[GOT] PACKET FROM=1 TO=4 SIZE=22 LEN=22

These are from consecutive invalid calls to the handler:
2018.12.03 15:46:15 [B] MSG:[GOT] PACKET FROM=7 TO=106 SIZE=0 LEN=0
2018.12.03 15:46:15 [B] MSG:[GOT] PACKET FROM=138 TO=11 SIZE=0 LEN=3
2018.12.03 15:46:15 [B] MSG:[GOT] PACKET FROM=138 TO=11 SIZE=0 LEN=3
(about 20 such until)
2018.12.03 15:46:15 [B] MSG:[GOT] PACKET FROM=1 TO=4 SIZE=10 LEN=10

As I get out of scope addresses, 0 size and/or non-matching size/len, the data buffer passed must be invalid.

Any idea why I might get these? I don't see any reason by looking at the code...

Low Baud Rate

I'm using this library with my two Arduinos, the controller (arduino nano) and an airplane. (arduino mega)

I'm using NeoSWSerial on the controller, and the Serial1 on the airplane. Now I'm using the HC-12 to communicate between these two devices (with echo, so airplane => controller => airplane => controller) while always sending new messages if nothing has been received for the last 5 seconds.

The HC-12 has an baud rate of 9600 configured on default, but to increase range, I have to lower it, after configuring everything correctly, I just cant get it to work with a baud rate of 4800 and lower (so everything below 9600), while 9600 works like a charm.

I tried it with a basic example without PacketSerial, and it works, so there has to be a bug or something like that.

Expected Behavior

Current Behavior

Possible Solution

Steps to Reproduce (for bugs)

CONTROLLER

void onPacketReceived(const uint8_t* buffer, size_t size) {
    if (size == sizeof(sensors)) {
        /** Update controls **/
        memcpy(&sensors, buffer, size);
        last_sensor_update = millis();

        /** Send current telemetry back **/
        HC12.send((uint8_t *)&controls, sizeof(controls));
        last_control_broadcast = millis();

        print_all_data();
    }
}

void setup() {
    Serial.begin(9600);             // Serial port to computer
    HC12neo.begin(4800);               // Serial port to HC12
    HC12.setStream(&HC12neo);
    HC12.setPacketHandler(&onPacketReceived);
}

void loop() {
    update_control_data();
    HC12.update();

    if (millis() - last_control_broadcast > 5000) {
        HC12.send((uint8_t *)&controls, sizeof(controls));
        last_control_broadcast = millis();
        Serial.println("send");
    }
}

AIRPLANE

void onPacketReceived(const uint8_t* buffer, size_t size) {
    if (size == sizeof(controls)) {
        set_update_led(HIGH);

        /** Update controls **/
        memcpy(&controls, buffer, size);
        /** Set new gotten controls **/
        set_throttle(controls.throttle);
        set_pitch(controls.x_joy);
        last_control_update = millis();

        /** Send current telemetry back **/
        HC12.send((uint8_t *)&sensors, sizeof(sensors));

        set_update_led(LOW);
    }
//    else {
//        buzz();
//        delay(1000);
//        stop_buzz();
//    }
}

/* SETUP METHOD */
void setup() {
    Serial.begin(9600);
    Serial1.begin(4800);
    HC12.setStream(&Serial1);
    HC12.setPacketHandler(&onPacketReceived);
    Wire.begin();
    Wire.setClock(400000);

    /** Setup sensors etc. **/
    setup_sensors();
    setup_servos();
    setup_leds();

    /* Set Power LED */
    set_power_led(HIGH);
}

void loop() {
    handle_servo_change();
    update_power_led(last_control_update);
    update_sensor_data();

    /** Incoming Throttle from Controller **/
    HC12.update();

//    Serial.print(sensors.x_angle);
//    Serial.print(";");
//    Serial.print(sensors.x_angle);
//    Serial.print(";");
//    Serial.println(sensors.x_angle);
}

Add CRC checksum

The SLIP and COBS encoding is great for aligning with streams, but it doesn't guarantee there are not any errors. It would be great if this library could automatically calculate and append a CRC, so the receiver can automatically throw away messages with errors.

Callback supporting member function

I have the same issue as #13, and found a workaround by adding some code into setup(), but I'd really love to keep PacketSerial fully inside my class. As it stands now, it won't work as 'this' was not captured for this lambda function if I try to setup the lambda function within my class. If I could have the callback return return an optional void* which I setup to point to my class, I could have the callback call a static method in my class.

https://wiki.c2.com/?VirtualStaticIdiom

Please LMK if you'd be interested in supporting this and I can at least prototype how I think it should work, though you might have a better solution than I could come up with. (This is my first time learning about lambda functions let alone trying to use one)

Serial is not sending data

Hi,
I'm using this great library to make USB-Serial binary protocol. I found that .flush() is not working properly and there's no way to flush data to the link.

The only thing I found it working is adding a Serial.println(); after each PacketSerial.send(). Something that adds \r\n of course but at least I can see the data on the other side of the serial link.
Do you think is something wrong with latest versions of arduino lib? 1.8.X

How to send and OSCMessage via PacketSerial ?

Hi Chris,

This is a great library!
I love the OpenFrameworks examples too, especially example_SLIP-OSC !

I'm trying get communication working with OSC the other way as well and got a bit confused on the best way to do that. OSCMessage. doesn't seem to have a method to expose the message as an encoded byte array. It simply writes bytes directly to a Print instance.
PacketSerial has a send which requires this byte array.

So far my workaround is to add a getter for _stream to pass go OSCMessage's send() method. It works, but I'm wondering: is this the right way to tackle this ? What is the clean/recommended way of sending an OSCMessage with PacketSerial SLIP ?

Thank you,
George

Set Packet Handler to member function

Hello,

I'd like to use PacketSerial within my own class, and set the packet handler to a member function using std::bind.

For example, instead of:
myPacketSerial.setPacketHandler(&onPacketReceived);

I'd want use something like:
myPacketSerial.setPacketHandler(std::bind(&MyClass::onPacketReceived, this, std::placeholders::_1, std::placeholders::_2));

I imagine the PacketHandlerFunction typdef would need to be modified to use std::function instead of a static function pointer.

Is this something you'd consider supporting?

Andrew

Seeing strange behavior ... looking for thoughts ...

Hello,

I've been using PacketSerial with great results, and have now ran into a problem that I'm trying to debug.

I'm connecting a Moteino with a ESP8266 ESP-12F. Both are using SoftwareSerial. I'm receiving packets from a radio, and putting it into a structure where I add a CRC32. The "packets" end up being 109 bytes. I've created a simple ACK/NAK protocol with 3 retries.

When sending through PacketSerial there are two issues that I am seeing in this particular project that are blowing my mind. Almost ALL packets are going through just fine. Sometimes - randomly - a packet does NOT go through and there are two different symptoms:

  1. Sometimes I get nothing at all on the receiving side. Even across the 3 retries.

  2. Sometimes I get a 0 byte length packet, followed by 172 byte packet. Should I really see a 0 byte length packet?

I've now added a 2 second "heartbeat" packet that is 5 bytes long. These seem to work every single time ... no problems. So I see 5 byte packets every two seconds, then I'll get a 109 byte packet, and then back to the 5 byte packets. And this goes on for minutes and then I'll get one of the two cases above.

I've flooded my code with debug output and just can't see where this could be failing. I can't think of how - when I see 3 retries of 109 byte packets - the receiving event handler would not fire. Nor can I understand where/how I would ever see 0 byte packets?

I'm not saying it's in the library, but looking for any thoughts or ideas of where I ought to be looking. I appreciate any input.

Compatible with Teensy?

I used this library for an arduino project and it worked great! However; I've since moved to work with Teensy boards and have been having difficulty getting the code to upload. Specifically when I start the myPacketSerial.begin(BAUD); it seems to crash the Teensy 3.5 board. I was wondering if this was going to be implemented in future updates or if I'm doing something incorrect with the Teensy 3.5 board as opposed to arduino.

Closed after reading wind4greg's post about wider compatibility and it fixed my issue.

Sending over serial stops

Hey! I'm using this library to send 44 bytes as a packet over serial. Smaller size packets work fine, however the full data of 44 bytes causes the arduino to stop sending packets beyond a few. For example, on the pc end i use a python script to read 45 bytes, output to screen, then read 45 more. This works the first 24 or 25 times, then stops. I've tried inserting Serial.flush(), and a 1000ms delay, however it is still no go.

Note on wider applicability and compatibility

Thanks for the great library! Just a couple of points that could be in the documentation if desired.

For those looking to use this library with other boards and other serial interfaces here is the approach I used. I applied this to a Feather M0 board which has an additional Serial1 interface defined.

Outside of any functions:

PacketSerial serial1; // Create a specific wrapper, you can call it anything

Inside of the Arduino setup() method:

    Serial1.begin(9600); // Starting AdaFruit's additional serial interface
    serial1.begin(&Serial1); // Wrap it with PacketSerial instance
    serial1.setPacketHandler(&onPacket); // Use it for receiving...

Note 1: I used PacketSerial on Serial1 while using still using Serial for debugging output and such. So unless I read your code wrong, it seems that one can apply PacketSerial to some serial interfaces and not others based on how one wants to use the serial interface.

Note 2: I experienced full compatibility of the COBS implementation with a nice Python COBS implementation receiving data from PySerial. These folks have a good reference to the COBS Transactions on Networking (TON) paper.

Once again thanks for the library!
Cheers
Greg

myPacketSerial.send() or myPacketSerial.update() does not return an error?

I'm debugging my current application and have found - what I think - is one of my issues.

My packets have now become larger, and are 260 bytes. I'm now sending them and do not receive any indication that 260 bytes is too large.

On the receiving side, I am getting packets but they are only 254 bytes.

Upon digging deeper I found that the default buffer in PacketSerial is only 256 bytes.

https://github.com/bakercp/PacketSerial/blob/master/src/PacketSerial.h#L27

I'm curious if this is a "send buffer" size? Or only a "receive buffer" size? Or both?

It seems that there is code to catch the buffer overflow here: https://github.com/bakercp/PacketSerial/blob/master/src/PacketSerial.h#L217

This would seem to indicate it's a receive only buffer limitation?

Is there a way that I ought to be catching this error?

Packet not received

I'm trying to send a packet from Python to the Arduino but the handler is never triggered. Maybe I misunderstand the packet format?

I'm using the COBS library to encode the data. Here's the packet as it is sent using the standard python serial library:
b'\x03\x01\x04\x00'

Here's the Arduino code:

#include <PacketSerial.h>
PacketSerial myPacketSerial;

void setup() {
  myPacketSerial.begin(9600);
  myPacketSerial.setPacketHandler(&onPacketReceived);
}

void loop() {
  myPacketSerial.update();
}

void onPacketReceived(const uint8_t* buffer, size_t size)
{
  uint8_t tempBuffer[size];
  memcpy(tempBuffer, buffer, size);
  uint8_t id = tempBuffer[0];
  myPacketSerial.send(id, 1);
}

Why am I not getting anything back?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.