A great deal of code is currently documented in an unconventional fashion. To be more precise, there seems to be a tendency to use headings to describe attributes, parameters and returned values, as opposed using info field lists.

For instance consider the docstring for rcognita.controllers.ctrl_selector:

    Main interface for various controllers.
        Parameters
        ----------
        mode : : string
            Controller mode as acronym of the respective control method.
        Returns
        -------
        action : : array of shape ``[dim_input, ]``.
            Control action.

The conventional way to produce a docstring bearing such information would be:

    Main interface for various controllers.

    :param str mode: Controller mode as acronym of the respective control method.
    :return: Control action
    :rtype: array of shape ``[dim_input, ]``

Here are some requirements for implementation of Rcognita framework for tests.

• A framework should be easy-to-use

• It should be provided with comprehensive and clear instructions on how to create tests using this framework

• It should cover all currently implemented presets

• It should prevent code duplication

• There should be an out-of-the-box possibility to generate a reference data for unit-tests

Fix simulation animation to work in default Python interpreter instead of only ipython

Suggestion (example for one argument):

dt = []
parser.add_argument('--dt', type=float, metavar='dt',
                    default=0.1,
                    help='Controller sampling time.' )

__init__ should have a docstring of its own. The "Attributes" section in class dosctrings is reserved for attributes. Violating this convention really messes up the wiki. We should fix that (preferably by next release I think) and from now on proceed to document new classes conventionally.

Code cleaning and refactoring needed for the modifications done throughout Q1, Q2 of 2021, including those done for education.

This concerns:

• actor constraints
• new methods: JAC-stab, SQL, SQL-stab, SQL-V etc.
• critic constraints for the respective methods, e.g., JAC-stab, SQL-stab etc.
• new systems
• generic main module (make to a class)
• model estimation
• loggers
• ROS integration

and so on

We need a call capability like:

python main_3wrobot_NI.py -ctrl_mode JACS -dt 0.01 ...

Required parameters:

Optional parameters, set to default values unless specified otherwise:

This needs to be reflected in the readme, as an example call of an example present. Could probably be translated from this text.

• Formatting tests
• Unit-tests
• (optional) pre-commit hooks?

Create a ROS_harnesses.py module to separate a ROS preset and a ROS setting utility

• Make an NN model
• Make a torch optimizer for NN model

Right now, the full state vector in the closed loop function of the system interface contain components related to the disturbance, even if the latter is switched off. Need case distinction as:

1. is_disturb => dim_full_state = dim_state + dim_disturb
2. not is_disturb => dim_full_state = dim_state

• no switch cases inside classes, only on pipeline or configuration level
• self.critic_clock into Critic class and, in general, all class-related field put in classes (with corresp. renaming)

Ubuntu/Debian:

sudo apt-get install -y build-essential gfortran cmake libopenblas-dev

Arch

pacman -Sy gcc gcc-fortran cmake base-devel openblas

Then after that

pip install scikit-build

or

conda install scikit-build

• Create estimators.py and observers.py modules
• Embed the NN estimator written by @kefir8888 to estimators.py
• Embed the EKF to observers,py
• Connect them to the whole pipeline

Update:

• Definitions of command line args
• Definitions of variables in rlframe

The documentation needs update according to the actual rsts

@osinenkop
Is it possible to move it

• Add DiscreteStatePredictor() implementation
• Move in there EulerStatePredictor from controllers

Please use this branch https://github.com/AIDynamicAction/mpldatacursor/tree/rcognita-v0.1

Add command line arguments for main script

The solution here is a class which has the following structure:

class abstract_config:
    def __init__(self):
        self.name = "some_agent"
    def argument_parser(self):
        pass
    def post_processing(self):
        pass
    def get_env(self):
        pass

It's very intuitive separation of command-line arguments and other arguments together with their post-processing.

• Completely refactor the code: separate the symbolic case and numerical case and move the code out of the controller.py
• Create tests for CASADI integration
• Make some benchmarks and create a table of comparison

• Create optimizers.py and implement an abstract class and all current realizations into it.
• Test out

Test package in jupyter notebook and fix bugs

There were currently implemented a framework for testing and reference data generation. To make it possible to test preset and improve readability of the code all presets were implemented using pipeline approach that appeared to be a good pattern for implementation of presets. To transfer Rcognita to the new preset implementation pattern, it is necessary to:

• Implement a CLI-interface consistent with the pipeline
• Refactor and implement pipelines for all presets

• Ubuntu 20 on lab's laptop
• main_ros.py

Need:

1. System: pendulum
2. Scenario for Monte-Carlo learning
3. REINFORCE

Visualizer: as always (like 3wrobot), but upper left screen: pendulum and its trajectory (dotted line like 3wrobot)

Monte-Carlo scenario:

1. loop over policy gradient updates
2. each such update needs several episodes (former runs), so loop over episodes
3. each episode is like the current main loop, i.e., it iterates over steps
4. when all episodes are done, experience is used to update policy parameters

Policy must be a PDF (probability distro func). Useful policy parametrizations -- see S&B, p. 322 book.
REINFORCE algorithm can also be found there

Add --config flag to allow change default arguments to custom settings

Check everything for correctness and necessity.
Split into different readable folders.
All data on kompaso's ssd.