This is the telemetry repo of @eagletrt
This project is made for the @eagletrt car, in order to save data when the car is running. The telemetry is a program located in the raspberry attached to the canbus of the car and directly to the serial porf of the rover gps, making it working. It reads the messages sent by the sensors attached to the canbus or the rover gps, accumulates them in an organized structure and every 500 milliseconds saves the data in a local mongodb and forwards it via mqtt. In order to avoid saving a huge amount of unneeded data when the car is not moving, the telemetry can be controlled by the steering-wheel, that can send messages via can to start or stop the telemetry to save the data in the database and pass parameters such as pilot and race to track the current session in the database.
The telemetry is a C program, thought to run on a linux system. Being the code related to the saved structure very long and since it changed very frequently, I eventuallly came up with a strange solution. There is the structure.json
file that determines the structure of the saved data and with an npm module the c code related to that part is generated automatically.
The program is a quite simple state machine, implemented in c with a matrix of transitions functions. It has also a mega global variable called condition, accessible everywhere in the code.
The states are:
- INIT: When the telemetry is starting, it reads the config file and sets up everything (canbus interface, gps serial port, mqtt connection, mongodb connection). After being executed, it will pass to the IDLE state.
- IDLE: The telemetry is actually running. There are two threads that read all the can messages and gps messages and save them in a structure. Every 500ms this structure is converted to bson and sent via mqtt. After being executed, it will repeat itself unless a can message to enable the telemetry is received.
- ENABLED: The telemetry does the same thing of the IDLE state, but it also saves the data in mongodb. After being executed, it will repeat itself unless a can message to disable the telemetry is received.
- EXIT: The telemetry tries to gently deallocate all the data and close all connections before exiting. It is usually reached when an error occurs.
There is the config.json file to configure various aspects of the telemetry. The name can be actually changed and is given as a parameter when executing the compiled sender.out file. There is no need of recompiling the telemetry after changing this file.
The options are:
-
mqtt: The options regarding the mqtt connection
- host: The host of the mqtt connection
- port: The port of the mqtt connection
- data_topic: The topic where the bson data is sent
- log_topic: The topic where the telemetry log is sent
-
mongodb: The options regarding the mongodb connection
- host: The host of the mongodb connection
- port: The port of the mongodb connection
- db: The db where the data will be saved
- collection: The collection where the data will be saved
-
gps: The options regarding the rover gps plugged to the telemetry
- plugged: If the gps is plugged to the telemetry. If the gps is not plugged but this option is set to
1
, the telemetry will try to read the gps serial port and will crush. - simulated: If the gps is simulated and not real.
- interface: The interface name of the gps serial port
- plugged: If the gps is plugged to the telemetry. If the gps is not plugged but this option is set to
-
pilots: The array containing the possible pilots who drive the car. Every time the steering wheel enables the telemetry, it specifies also the index of the pilot who drives the car. The pilots name will be added to the current session, to the data saved in the database
-
races: The array containing the possible races that the car can perform. Every time the steering wheel enables the telemetry, it specifies also the index of the race that the car is performing. The race name will be added to the current session, to the data saved in the database
-
circuit: The array containing the possible circuits where the car is running. Every time the steering wheel enables the telemetry, it specifies also the index of the circuit where the car is running. The circuit name will be added to the current session, to the data saved in the database. NB:
currently not implemented
-
can_interface: The interface of the canbus
-
sending_rate: The number of milliseconds every which the telemetry saves the accumulated data, before emptying it and repeating the cycle
-
verbose: If also the debug messages will be logged in the console
The structure of the gathered data is saved in the structure.json
file and is based on the possible messages:
- The structure is a json object, but every primitive key contains the value type instead of the value itself
- For each message there is an array of objects
- Each array contains only two elements: the message object and the message max count
- Each message object represents the model of the parsed message. It has the timestamp of the receive time and a value containing the value of the message (a value or an object of values)
- Each message max count represents the maximum number of messages that can be saved in a single document
- Each array can be saved directly in the "root" of the structure object, or nested in other objects, in order better to organize the structure. For instance, all the bms message arrays are nested in the bms object
- There is an id key, containing a progressive id of the document starting by
1
- There is a timestamp key, containing the timestamp when the document is saved in the db, or sent via mqtt
- There is a sessionName key, that refers to the session of the document itself
Each time the telemetry enters in the ENABLED state, the id number is reset and a new session is created. A session is based on the timestamp when the telemetry was enabled and the parameters (pilot, race) passed by the steering wheel. A session document containing these parameters is created and inserted in the database.
This is an example of session document:
{
"sessionName": "20200309_175011_default_default",
"timestamp": 1583772611,
"formatted_timestamp": "20200309_175011",
"pilot": "default",
"race": "default"
}
This is an example of structure.json:
{
"id": "int",
"timestamp": "long",
"sessionName": "char*",
"throttle": [
{
"timestamp": "long",
"value": "double"
}, 200
],
"brake": [
{
"timestamp": "long",
"value": "double"
}, 200
],
"bms_hv": {
"temperature": [
{
"timestamp": "long",
"value": {
"max": "double",
"min": "double",
"average": "double"
}
}, 500
],
"voltage": [
{
"timestamp": "long",
"value": {
"max": "double",
"min": "double",
"total": "double"
}
}, 500
]
}
}
Since a huge amount of code depended on the structure and the structure changed frequently, lots of hours used to be lost doing repetitive work and copy-paste for every changed message. C is a static type language and is not favorable for structure as the telemetry one. It is anyway essential because of the performance we needed. Javascript is instead a very flexible language and is not statically typed. So I made a repository in Typescript that became an npm module to generate the C code that depends on the structure.
There are some files, the template files, ending with .template.c
or .template.h
. Those files contains some special comments such as // {{GENERATE_BSON_CODE}}
instead of the pieces of code that depends on the structure.
The code generator:
- Reads the
structure.json
- Looks for the template files
- Substitutes the special comments with the code automatically generated and based on the read structure
- Creates a file without the
.template
extension for each template file
So, when the structure changes, it is only needed to change the structure.json
file properly and run npm run transpile
to generate the new C code.
The code is organized in three folders and a file:
- main.c: It is the file containing the
main
function. It has the instance of the condition and runs the state machine - state_machine: It is the folder containing the state machine. It has an header file describing the condition, the mega-unique-global variable that describes the current state. It has the header and c files describing the state machine itself and its state-functions
- services: It is the folder containing the services, such as mqtt service, mongodb service and all the others. It has an extern to make the code know that the condition variable exists.
- utils: It is the folder containing the utils, containing the utilities functions. It does not know that the condition variable exists.
On the raspeberry of the telemetry there is the Ubuntu for ARM operative system. Hence, all the code was thought to run on a Linux system.
- Use a Linux operative system
- Install gcc to compile c programs, on ubuntu
sudo apt install build-essential
- Install nodejs to execute js scripts, on ubuntu I suggest reading this
- Install mosquitto to host an mqtt broker,
sudo apt install mosquitto && sudo apt install mosquitto-clients
- Install canutils to connect to canbus,
sudo apt install can-utils
- Install mongodb to have the local database, on ubuntu I suggest reading this
- Install mongodriver for c to use mongodb from c, on ubuntu
sudo apt install libmongoc-dev && sudo apt install libbson-dev
- Install mqttdriver for c to use mqtt from c, on ubuntu
sudo apt install libmosquitto-dev
- Make sure mongod service is active (check
mongo
works) - Simulate the canbus and the gps serial port (check this repo) This is needed when debugging the telemetry in a local pc
- Clone this repo
- Execute
npm i
to install the nodejs dependencies - Execute
npm run transpile
ornpx eagle generate
to execute the js script and generate the c code - Execute
npm run compile
or./compile.sh
to compile the c code - Execute
npm run start
orsudo ./sender.out config.json
to start the telemetry (executingnpm run serve
does all the last three points with an only command)
- Execute
npm run enable
or./enable.sh
to enable the telemetry and make it saving data on the db - Execute
npm run idle
or./idle.sh
to disable the telemetry and make it stopping saving data on the db This is usually useful when debbugging the application on a local computer, because simulates what does the steering wheel of the car
- Before starting the telemetry execute
mosquitto_sub -t telemetria_log
. It should show the log of the telemetry. - Execute
mosquitto_sub -t telemetria
, it should show the data sent by the telemetry via mqtt (it is sent even when the telemetry is idle). - Open mongo compass and check the data is saved on mogodb