Search for literature approaches to

• world representations and formalism
• Suitable learning approaches

The number of cases appears to be too high for the pushTask simulation (since action is always the same and interactions are very similar)!

Currently there are some interesting bugs.
Also it might be possible that features do not change enough for the AC split to work properly

General ticket covering writing the actual thesis

• For structure see here (https://github.com/BlackZ/masterThesis/wiki/Thesis-structure)

Check if you cited them correctly and you display all relevant information!

They are currently left out since it is still unclear if this will be part of the evaluation

The current rating is not very successful/powerful. The current features (maybe use different ones) should not be weighted identically!

Especially in the first chapters, acronyms are often hardcoded. Make sure to change them to the
acronyms so that all of them are present in the list of acronyms.

General ticket for the gate model

• Replace hardcoded actuator with learning model
  • Switch already implemented to use ITM instead
• Replace hardcoded gate function with classifier
• Move feature specifics to statespace if possible
• Make model as general as possible

Currently distance is only calculated to the bottom line of the block.
Easy fix: Compute distance to all edges and take closest one.

The model should select an action in order to improve it's predictions.
-> If prediction was bad or a new abstractCase was created, try to perform a similar action (imitate goal babbling?)
-> If prediction for all Cases is satisfactory, perform random action/No action?

Just to see if the current removing hinders it's performance

Possible alternatives:

• Check output of winner/second
  • Remove worse one if other is good enough

As a "first" step for the moveToTarget-Task, it might be easier if the gripper starts in contact with the block, so that the model does not first need to find subtargets in order to establish the contact.

Concept of the interaction model currently describes only that the next interaction state is predicted, whereas the gate model describes that changes are predicted and added to the current states.

This needs to be made consistend between both of them (including the figures).

Bundle description and justification for using only changes somewhere (either in concept chapter or realization)

Create Testsuits:

• Give target position (for block) and evaluate required number of training iterations and performance
• Active exploration (for fixed amount of time) before prediction task etc

I should focus on the prediction and planning of actions and their effects on objects, rather than try to learn concepts or something like that.

• How does the current model work
• What kind of predictions where tested -> GP, ITM, (LSTM)
• World representation
• Issues
  • Similarities
  • AC selection

Current PushTask Simulation based on AC sometimes does not give scores to any AC, figure out why

Consider two kinds of Cases:

1. Movement cases
2. Interaction cases

Gripper and other objects are identical safe the ability to control the gripper
-> Movement and Interaction cases could be identical for gripper and objects
-> Potentially quicker learning

Adapted Instantaneous Topological Map sounds strange

A list of features that could be considered if time and motivation:
Order does not indicate any priority!

• Reduce number of required cases
• Try GP or other regression model instead of ITM
• Normalize metric by dividing each feature by it's average range of values
• Visualize resulting AC's/cases etc
• Handle outlier problem
• Investigate implementing AC transitions (closed #25)
• Improve AC selection (related to #24)
• Investigate TARPIR (#22)

List of things that need to be done

• Improve old model performance and evaluate
• Evaluate different settings (see #32)
• Implement exploration strategy
• Add stuck detection to gate model
• (Re)move hacks from gate model
• Create images/figures for thesis
• Rewrite thesis according to #31 and protocol of today's meeting
• Try multiple objects (first make adjustments to allow that)
• Try using different models for training and testing

Ideally the model does not care what the features represent. Try to remove as much feature related information from the model. It is ok if it's in the state representation (Object/Action) since those are domain specific anyways. The interface of the State Representation should be minimal though.

For the thesis, we are only going to work with 2D anyways so it does not really make sense to carry two additional features around that never change anyways (position and velocity). If anything they may only introduce some bugs.

Try something similar to the way the ITM tries to figure out the threshold, so that the models can adapt themselves to different time intervals between updates without changes to the code

Sometimes, the interface does not reset the world properly or the action arrives too late. Causing strange timing errors, such as not setting the starting position in the MoveToTarget training correctly, or receiving training frames, where an action is selected, but no change happens for PushTaskSimulation.

For finding good actions, it might be useful to know that, in order to move the block, we need contact.
Therefore if no contact is present, consider ACs that predict contact etc.

Start with batch training every step something changes, and if that workes try online training!

General ticket for things related to the pushTaskSimulation task and prediction

• Try different regression models (#8)
• Try with multiple objects if possible
• Try using offline trained AC selection classifier
• Evaluate block prediction, by ignoring all instances where only gripper moves. Otherwise gate is favoured too much

Having just one file to define the look for all plots helps with consistency. We can also set the same font as used in latex!

http://matplotlib.org/users/customizing.html

Find some way to visualize progress and internal structure
(e.g. graphViz)

If in a known situation (an exact case is known for a certain situation) something unexpected happens
(e.g. Tutor moves something), be careful how the model is updated! General prediction should become tolerant against outliers with experience!

No real way around doing it right, even if it sucks

Start with only 1 other object and learn interactions with it
Try to implement some sort of "smart" exploration learning that chooses actions in order to improve predictions.

The learner will need some kind of feature vector. What should it look like.

First idea:
List of relations between object pairs

• not so easily extendable

Since circling already used hardcoded knowledge, it does not make sense to force him to make an entire round just to undo a small mistake

If given a current state and a desired state, find an action that reduces the distance to the desired state!

General ticket for the evaluations

Task specific things are listed in #29 (moveToTarget) and #30 (pushSim)

• Make models as similar as possible without impacting their performance (evaluation loop, features etc)
• Compare 10HZ vs 100HZ
• Compare different number of (hardcoded) train runs
• Evaluate AC selection (online with offline performance etc)
• Evaluate effect of hardcoded parts (gate function/actuator)
• Evaluate performance (speed/memory) with respect to the number of timesteps

Since it is mentioned in the theoretical discussion of the concept, at least some table should exist that shows the distribution (at least if something can be said about that afterwards)

Long-Short-Term-Memory (Schmitt Huber)
http://lstm.iupr.com/
http://www.felixgers.de/papers/phd.pdf

• Present the problem
• Present the two concepts/models
• Present evaluation criteria
• Briefly mention metric problem
• Ask about evaluation hints

General ticket for the move to target evaluation task.

• Improve Performance (mainly AC model)
  • Try MetaNetwork for AC model
  • Improve/Retry inverse matrix (#27)
• (Re)move "hacks" from GateModel
• Implement active/exploration!!! (#6)
• Make greedy metaNodes smarter, so that they choose the better of two options

• PredictionError (e.g. PushTask)
  • Cummulative (Simulation), Each step error etc
• Move Block to target position/orientation
  • initially without blocking object
  • Have chain of targets, e.g. move up and then down

We need a model storing what we know about the world.
Scene containing objects with different properties (i.e. Position, Type)

Ideally linked to learner -> Allows predictions