vlebedynska / openroberta-lab Goto Github PK

Extend program statements validation (validator visitor) to prevent unexpected behavior in simulation

• check order of RL statements in program
• generate proper warning messages as feedback

For encapsulation purposes, the Qlearning algorithm should be extracted into a separate module and connected to robotMbedBehaviour via require.js as a module.

• link between nodes should be highlighted when it's active
• add a circle to each line a slider
• the clickability of the thin lines should be improved
• display node values/labels within the nodes
• display the current value of the weight
• align output layer to the center of input layer

Create tests and test resources for:

• ai_q_apply_learning_rule
• ai_q_barrier_mountain
• ai_q_draw_best_path
• ai_q_learner_config
• ai_q_learning_states_and_actions_matrix_3_x_3

• ai_q_apply_learning_rule
• ai_q_barrier_mountain
• ai_q_draw_best_path
• ai_q_learner_config
• ai_q_learning_states_and_actions_matrix_3_x_3

Following input and output node types should be implemented:

Input node types:

• colour sensor for EV3 basic colours
• light sensor

Output node types:

• Text displayed on EV3 Brick
• Ton
• LED

The block ai_neural_network for the ai extension should be integrated to java backend.
tbd

• UI - concept
• Data exchange, Event handling when changing the weights in node's connections
• Tests: responsible design, usability, performance

QLearning algorithm should be integrated into the simulation:

• Integrate the visualisation of the learning process in a popup
• After the learning process is completed, the SVG with the optimal path should be dynamically loaded into the simulation as a background.

• connect new block to java backend
• set up the test environment
• jaxbToAst implementation
• definition of visitor methods

For better user experience purposes, the reinforcement learning pop up should be re-designed.

• extend the reinforcement learning module and redesign its architecture (anticipated approach: Player as controller and data manager, Clock for regulating the speed of every single step, Visualizer for visualizing every step on the SVG, prepared SVG, QLearning-Algorithm itself)

• explore reactjs technology for optimal value changes in the popup

• implement dynamic changes on popup values

• Explore Q-Learning-Algorithm
• implement the basic structure of the algorithm

• blockly side implementation

• java side integration of the block (extension of AST, visitor extension)

• implementation of the block into the simulation

Overview of the blocks:

• tbd
• tbd
• tbd

For a better understanding of the results of q-learning algorithm tests for hyperparameter tuning should be provided.

• improvement of the workflows for the visualization of the algorithm
• bug fixes
• add new backgrounds
• design changes

Elements that are found and used multiple times (elements, paths, and texts) should be cached for better visualization performance of qLearner algorithm.

A test environment should be set up for testing of the reinforcement learning and neural networks feature.

• RlEnvironment
• RlGainExperience
• RlObstacle
• RlSetUpQLearningBehaviour
• RlDrawOptimalPath