amiithinks / robotlearning Goto Github PK

Should we add a new layer of abstraction in the code, to include control and predicting GVFs simultaneously ? @yasuiniko @banafsheh-rafiee @dquail

The state representation of IMU is not what we expect after tiling.
It might be the case that the issue is with out implementation, if that is the case, most probably all the data that we're getting is bad.
Should be fixed ASAP

needs #8 to be done first

use numpy magic to get pixel values rather than the current loop

ObservationManager.py is written for the dynamixel, so the code needs to be replaced with code that will work with our turtlebot.

Need to create a function to generate state for the demobot wall task.

Description:
We want to check the performance of the time between publishing to a topic, and that message being received by subscribers.

Description:
We should review the code from the old robot learning project. And move it from dropbox to the amiithinks github repo.

Apparently, the robot is not getting any data from the IMU. On the same bot the values on the /odom topic are wrong.
Does anyone knows about the hardware of turtlebot ?

Only pose.x is increased as the bot moves forward. It's effectively giving the distance traveled by the robot, while it should be giving the displacement of the bot.

Sift is interfering with the state representation unnecessarily.

• High level welcome
• High level data flow diagram
• High level module description (Learning foreground, action manager, wall demo, etc.)
• Descriptive pydoc comments for functions and classes
  • Learning Foreground
  • Action Manager
  • State Manager
  • GenTest State Manager
  • Wall Demo
  • Wall Demo Example
  • Auto Docking
  • Align IR Example
  • GVF
  • GreedyGQ
  • Policy
  • GTD
  • Tools
  • Visualize
  • Evaluator
  • Offline Wall Demo
• Netbook setup
• User laptop setup { ssh vs calling; python locally, etc. }
• Examples (simple use cases)
  • Wall Demo Example
  • Align IR example
• Describe branches
  Note anything else that will be useful for future users of our code

Since other people want to use the turtlebots too, it would be useful to have a commented template that they can fill out with their experiment-specific RL code and hardcode the turtlebot-specific code.

We can also use this template ourselves while making the demobot.

A good place to start might be LearningForeground.py

It is a feature that is always equal to 1.

Need to create a real policy that depends on the most recent prediction of hitting a wall -- our demo bot only turns if the prediction of hitting the wall is sufficiently high, which means to do this we also need to add predictions to our state representation.

Example:

Publisher queue size 10

Publish 10 "angular velocity = 2" messages
Publish 10 "linear velocity = .35" messages

The turtlebot goes mostly forward but with a slight turn to the side. How can we send actions in a way that is more precise?

The agent appears to learn to avoid two of the walls of the pen and fails to learn to avoid the other two. I suspect it is hard to represent the walls with a linear function of pixel color values. Maybe we should:

1. Tile code the RGB values of each pixel together. Haven't done the calculations on this yet.
2. Use kernel magic, which I am reading about now. There seem to be some decent online methods.

Any other suggestions appreciated!

The state representation is totally dependent on the image existing. At the very least, the image bits should be set to 0 so we can still access bumper information.

Even better, we could store and use the most recent image if no image already exists

create the state representation for the wall demo (including the GVF prediction)

Currently there are only GVF placeholders, but no code for the GVFs themselves.

Data is not smooth enough to be useful at all. There are too many fluctuations, so there is too much noise.

(See "Generate and Test" in list of recommended paper for details of implementation)
We will later have to test if this is indeed better than our current naive random pixel picker.

We currently use the action_manager process to publish movement messages at a pre-specified rate. Depending on how often the bot updates its movement message, the action_manager may not be sufficient and we might have to go to a lower level. Further testing on the action_manager's effectiveness is required.