initial plan and ideas about skyline HOT 2 OPEN

Simpler version

To make it easy to build a v1 we could go for predicting throttle + steering from synth dataset. This reduces complexity a lot as we wouldn't need a tracking system, a messaging system, executing MPC, multiple threads and daemons, a state estimator etc... we could just run the model in a python loop and call it a day. We know it can work and it would be a good first step to validate synth -> real (we know it works since we know people who have done it)

We could also try to install openpilot on the jetson nano and gauge how easy/hard it is to mod. I suspect it might be doable! I think we would also need to install ubuntu 20, and we could try this image

from skyline.

Re: state estinator + MPC:

We have a few sensors on the car: a camera, an IMU (accelerometer, gyroscope and magnetometer) and an encoder on the motor. Estimating state (position + velocity) with the IMU only is annoying because small errors are going to accumulate and the state estimation is going to drift.
But I think we only need the velocity estimation along each axis to run the MPC to execute the predicted trajectory. The predicted trajectory is relative to the car so we could just estimate the car position to be 0,0,0. We actually don't need the real position to compare the Model (model as in MPC) output to the target trajectory.
Also, the measured motor speed is absolute, it doesn't drift so we should be able to estimate the speed without drift

from skyline.