prost-planner / prost Goto Github PK

Original report by tkeller (Bitbucket: 557058:280236d3-4090-4dc9-9a03-b6e1425df4e7, GitHub: thomaskeller79).

One of the strength of RDDL is it intuitive way of using math inside CPFs. Numerical state fluents are absolutely necessary to make use of that, currently it is only possible in the reward function. (issue from 29.03.2013)

Original report by geisserf (Bitbucket: 557058:e7a9f9a5-3ea8-4154-97d2-10446425dce3, GitHub: geisserf).

Currently, search/utils and rddl_parser/utils are nearly identical. A change to one utility function in search often requires copy/pasting the entire file to the parser. Furthermore, while a new implementation could work in search, there could be issues in the parser, that are only noticed after the file was copied, making a lot of extra work.

I propose to merge the utility classes to one location, e.g. src/utils.

Original report by tkeller (Bitbucket: 557058:280236d3-4090-4dc9-9a03-b6e1425df4e7, GitHub: thomaskeller79).

For issue #31, we need a build system that compiles the planner both on Linux and Windows (and, if we ever want to add it as well, for MacOS).

The pull request where we discuss this can be found here:

https://bitbucket.org/tkeller/prost-dev/pull-requests/16

Original report by tkeller (Bitbucket: 557058:280236d3-4090-4dc9-9a03-b6e1425df4e7, GitHub: thomaskeller79).

A parser that does not need the intermediate step of translating rddl to rddl prefix would be nice. (issue from 28.06.2016)

Original report by tkeller (Bitbucket: 557058:280236d3-4090-4dc9-9a03-b6e1425df4e7, GitHub: thomaskeller79).

IDS is replaced with an Maximum Lookahead Search component if caching is disabled, the maximal search depth of the IDS component is set to 1 and the time it takes to compute a single heuristic value is stil too large. The current implementation is very ad-hoc and can be improved. (issue from 01.12.2015)

Original report by tkeller (Bitbucket: 557058:280236d3-4090-4dc9-9a03-b6e1425df4e7, GitHub: thomaskeller79).

The current class structure is such that each THTS algorithm is implemented in its own classes. However, the THTS structure allows the combination of different ingredients at will, and the implementation should reflect this (also to get rid of duplicate code). (issue from 11.08.2015)

Original report by geisserf (Bitbucket: 557058:e7a9f9a5-3ea8-4154-97d2-10446425dce3, GitHub: geisserf).

Currently, whenever a code style violation occurs during a commit, the hook script only shows which files violate the code style, but not where the style is violated. The developer then can:

• Manually check which lines violate the style: use clang-format on the file, save the output, diff the output with the original file.
• (clang-)format the whole file without caring which lines violated the style.

Since the first option is cumbersome, a developer might prefer option two; especially when one just wants to commit changes at the end of a day. However, there are times when code style violations are better fixed by refactoring the code in question.

If the first option is already automated by the hook script, developers might be more inclined to refactor some code.

Original report by tkeller (Bitbucket: 557058:280236d3-4090-4dc9-9a03-b6e1425df4e7, GitHub: thomaskeller79).

Some classes and methods (e.g., LogicalExpression and utils/*) are currently both in the parser and the search part. Create a directory for shared code (e.g., src/shared) that is used by both components.

Original report by geisserf (Bitbucket: 557058:e7a9f9a5-3ea8-4154-97d2-10446425dce3, GitHub: geisserf).

Our current implementation of logical expressions and related functions has three major disadvantages:

• no memory management. Whenever a new logical (sub-)expression is created, we have a new raw pointer which never gets deleted, even if the expression is not used anymore. This leads to problems when we implement algorithms with many expression transformations (e.g. EVMDD implementations in our icaps2016 paper).

• due to the way logical expressions are implemented, methods transforming logical expressions have to do dynamic casting if their logic requires the knowledge of the specific type. This not only leads to boilerplate code, but has also influence on the runtime of such methods, e.g. state evaluation.

• the current implementation is cumbersome to extend. When we want a new method which works on expressions, it has to be implemented in every expression type (logical_expressions.h has already nearly 1000 lines of code).

A new implementation should consider all three points.

Original report by tkeller (Bitbucket: 557058:280236d3-4090-4dc9-9a03-b6e1425df4e7, GitHub: thomaskeller79).

The discount factor is often used in MDP planning and also supported by RDDL, but not implemented in PROST (it is currently ignored). (issue from 08.10.2013)

Original report by tkeller (Bitbucket: 557058:280236d3-4090-4dc9-9a03-b6e1425df4e7, GitHub: thomaskeller79).

There is no clear convention on the coding style that is used within the PROST planner. Diffs between versions therefore become unnecessarily large, which should be avoided when possible. We are currently looking into the Google C++ Style Guide and consider to adapt it to our needs. (issue from 07.12.2015)

Original report by tkeller (Bitbucket: 557058:280236d3-4090-4dc9-9a03-b6e1425df4e7, GitHub: thomaskeller79).

In GGP-PROST (a variant of PROST for general game playing developed by Daniel Brand), we implemented support of dynamic state-action constraints ("preconditions"). It would be nice to have this behaviour ported here sometime in the future. (issue from 17.08.2012)

Original report by tkeller (Bitbucket: 557058:280236d3-4090-4dc9-9a03-b6e1425df4e7, GitHub: thomaskeller79).

At ICAPS 2013, I was asked by several people that they wish to use PROST without the IPPC2011 Server but with an user defined environment instead. (issue from 17.06.2013)

Original report by Anonymous.

Hi,

The function RDDLParser::parseCPFDefinition present in rddl_prefix_parser folder is not working as it should. Can you please look into the issue?

Original report by tkeller (Bitbucket: 557058:280236d3-4090-4dc9-9a03-b6e1425df4e7, GitHub: thomaskeller79).

The THTS framework has been implemented within PROST, but there have been some changes to PROST since then. Merging the two into a single planner with different search engines seems reasonable. (issue from 29.03.2013)

Original report by tkeller (Bitbucket: 557058:280236d3-4090-4dc9-9a03-b6e1425df4e7, GitHub: thomaskeller79).

A task representation where Boolean variables that cannot be true at the same time are merged into finite-domain variables are important for many techniques from classical planning to work properly. Since we have stumbled across this problem several times when trying to port these techniques to MDPs, it is about time to actually add a simple invariant synthesis that is able to find at least some of these finite-domain variables.

Original report by geisserf (Bitbucket: 557058:e7a9f9a5-3ea8-4154-97d2-10446425dce3, GitHub: geisserf).

Currently, parsing aborts if some discrete distribution contains conditional probabilities, as we do not yet support most-likely determinization for such expressions (see TODO in rddl_parser/logical_expressions_includes::156). Example domain containing such a distribution is attached.

Original report by tkeller (Bitbucket: 557058:280236d3-4090-4dc9-9a03-b6e1425df4e7, GitHub: thomaskeller79).

An assertion in instantiate.cc (line 65) is violated in the presence of nested quantifiers. (issue from 19.10.2015)

Original report by tkeller (Bitbucket: 557058:280236d3-4090-4dc9-9a03-b6e1425df4e7, GitHub: thomaskeller79).

It is possible that parameters are dynamic if they are given as an object or enum valued fluent. (issue from 05.03.2014)

Original report by tkeller (Bitbucket: 557058:280236d3-4090-4dc9-9a03-b6e1425df4e7, GitHub: thomaskeller79).

The parser has been created with a tight deadline before IPPC 2011. Refactoring it should increase code readability. Moreover, we'll split it from the rest of PROST for standalone usage. (issue from 20.09.2013)

Original report by tkeller (Bitbucket: 557058:280236d3-4090-4dc9-9a03-b6e1425df4e7, GitHub: thomaskeller79).

MCTS algorithms (most importantly, UCT) can be modelled as THTS algorithms, and similar versions can be run in PROST, but they always differ in some details. Adding a random walk search engine and an initialization component that allows to add only a single successor to the tree should suffice to model UCT exactly in the version that is described by Kocsis and Szepesvari. (issue from 26.11.2015)

Original report by tkeller (Bitbucket: 557058:280236d3-4090-4dc9-9a03-b6e1425df4e7, GitHub: thomaskeller79).

Supporting finite-domain variables and domain constants would be another step towards a richer input language. As object-fluents have shown to be a convenient tool in PDDL, we additionally implement a variant in RDDL. (issue from 11.04.2013)

Original report by tkeller (Bitbucket: 557058:280236d3-4090-4dc9-9a03-b6e1425df4e7, GitHub: thomaskeller79).

The training set creation currently depends on a timeout and is hence not deterministic. Switch it to use a fixed number of randeom walks instead.

Original report by tkeller (Bitbucket: 557058:280236d3-4090-4dc9-9a03-b6e1425df4e7, GitHub: thomaskeller79).

Deterministic domains currently lead to a planner crash. (issue from 05.06.2013)

Original report by tkeller (Bitbucket: 557058:280236d3-4090-4dc9-9a03-b6e1425df4e7, GitHub: thomaskeller79).

The introduction of the classes for THTS ingredients in issue-18 has rendered most tests to not compile or to fail. Additionally (yet unrelated), parts of uncrustify are still in the system.

Original report by geisserf (Bitbucket: 557058:e7a9f9a5-3ea8-4154-97d2-10446425dce3, GitHub: geisserf).

Currently a job created with scripts/create-jobs writes the server output (rddl-.log) directly in the testbed directory. It would be better if a separate directory is created.

Original report by tkeller (Bitbucket: 557058:280236d3-4090-4dc9-9a03-b6e1425df4e7, GitHub: thomaskeller79).

After issue #46 has been resolved, we'd like to compile all CPFs into conjunctions of conditional effects. This "normal form" should allow us to parse PPDDL tasks into the same data structure as RDDL tasks.

Original report by tkeller (Bitbucket: 557058:280236d3-4090-4dc9-9a03-b6e1425df4e7, GitHub: thomaskeller79).

In order to integrate the PPDDL parser, we need a common datastructure both for (finite horizon) RDDL tasks and (goal-oriented) PPDDL tasks. In a first step, we'd like to introduce goals that express that the steps to go variable must be 0. From there, it shouldn't be too hard to generalize to arbitrary goal formulations.

Original report by tkeller (Bitbucket: 557058:280236d3-4090-4dc9-9a03-b6e1425df4e7, GitHub: thomaskeller79).

Clean up and merge the delete relaxation code that was implemented for our IJCAI 2015 paper. It is quite likely that this issue is too large and should be split in several parts (at least one for the creation of goals and SAS+ like operators, and one for delete relaxation). (issue from 01.12.2015)

Original report by tkeller (Bitbucket: 557058:280236d3-4090-4dc9-9a03-b6e1425df4e7, GitHub: thomaskeller79).

Create a single class that both writes the output file in the parser component and parses that output in the search component. This is advantageous when new data that has to be communicated is added, as all changes are in a single place.

We could also use this issue to change the output format to a well-known data-interchange format like, e.g., json.

Depends on issue #33, where shared classes between both components are introduced.

Original report by tkeller (Bitbucket: 557058:280236d3-4090-4dc9-9a03-b6e1425df4e7, GitHub: thomaskeller79).

Some basic datastructures (PlanningTask, CPFs, SACs etc) are not implemented in a clean way. (issue from 01.04.2013)

Original report by tkeller (Bitbucket: 557058:280236d3-4090-4dc9-9a03-b6e1425df4e7, GitHub: thomaskeller79).

Supporting finite-domain variables would be a first step towards a richer input language. Currently, only enums are possible to model this feature with RDDL, but they are not supported by PROST (albeit there is some code to parse them). As object-fluents have shown to be a convenient tool in PDDL, we additionally implement a variant in RDDL. (issue from 29.03.2013)

Original report by tkeller (Bitbucket: 557058:280236d3-4090-4dc9-9a03-b6e1425df4e7, GitHub: thomaskeller79).

The whole caching or not question is a bit messy currently: In some parts of the code, caching is always enabled (e.g., cacheActionsToExpand in PlanningTask). In others, it depends on the state size (e.g., cachingEnabled in DepthFirstSearch). We should change this by adding a virtual function to the CachingComponent interface that is called initially to decide where caching is used and where not.

Original report by Anonymous.

Hi,

Definition of pictureTaken in recon_mdp.rddl is

pictureTaken'(?o) =
KronDelta( exists_{?x : x_pos, ?y : y_pos, ?a: agent, ?t: tool} [CAMERA_TOOL(?t) ^ agentAt(?a, ?x, ?y) ^ objAt(?o, ?x, ?y) ^ useToolOn(?a, ?t, ?o) ^ ~damaged(?t)] );

shouldn't it be

pictureTaken'(?o) =
KronDelta( pictureTaken(?o) | exists_{?x : x_pos, ?y : y_pos, ?a: agent, ?t: tool} [CAMERA_TOOL(?t) ^ agentAt(?a, ?x, ?y) ^ objAt(?o, ?x, ?y) ^ useToolOn(?a, ?t, ?o) ^ ~damaged(?t)] );

Original report by Relić Đorđe (Bitbucket: 557058:f6befa2c-c3fc-4e1b-965b-ab7eca65a967, ).

A Bison/Flex parser for ppddl. (issue from 10.04.2017)

Original report by geisserf (Bitbucket: 557058:e7a9f9a5-3ea8-4154-97d2-10446425dce3, GitHub: geisserf).

Update the Timer class to use the modern c++11 interface.

This issue is related to issue #24 .

Original report by tkeller (Bitbucket: 557058:280236d3-4090-4dc9-9a03-b6e1425df4e7, GitHub: thomaskeller79).

There are still some action selection methods (and maybe additional stuff?) that were evaluated in my dissertation and that have not been made publically available. (issue from 07.12.2015)

Original report by tkeller (Bitbucket: 557058:280236d3-4090-4dc9-9a03-b6e1425df4e7, GitHub: thomaskeller79).

Interm-fluents are specified in RDDL, but they can't be used to specify derived fluents that are used in the SACs (because interm-fluents must be evaluated after actions are selected). For that reason, we'd like to have derived-fluents implemented (additionally to interm-fluents), getting an evaluation chain of: 1. Have state. 2. Evaluate derived-fluents. 3. Evaluate SACs to determine applicable actions. 4. Select applicable action. 5. Evaluate interm-fluents based on. 6. Evalute successor state and continue with 1. (issue from 07.12.2012)

Original report by tkeller (Bitbucket: 557058:280236d3-4090-4dc9-9a03-b6e1425df4e7, GitHub: thomaskeller79).

In my PhD thesis, I extended the THTS framework by a recommendation function and showed that different implementations shine in different recipes. This should be reflected in the THTS class. (issue from 11.08.2015)

Original report by geisserf (Bitbucket: 557058:e7a9f9a5-3ea8-4154-97d2-10446425dce3, GitHub: geisserf).

If the IDS learning phase exceeds the time limit for the first depth step of a training state, memory will get corrupted, which may lead to a segfault later on in the search.

This bug appeared in 1 of 6000 runs on the grid (with IPPC2011 as configuration).

The bug is reproducible with
./prost-debug ../../testbed/benchmarks/ippc-all/prost/game_of_life_inst_mdp__10 [PROST -s 1 -se [IPPC2011 -init [Expand -h [IDS -st 0.0000002]]]]
This will however not lead to a segfault.

I was able to reproduce the segfault on my machine with a small hack in depth_first_search.cc, which reproduces the behaviour of the grid. I attached the modified depth_first_search.cc as well as the game of life problem file for which the segfault occurs (of course this bug only appeared with the specific training set of the grid file and not with the training set generated on my machine...).

With these files the following command should lead to a segfault in the first few search steps:
./prost game_of_life_inst_mdp__10 [PROST -s 1 -se [IPPC2011]]

I've tracked down the bug and the reason is that if the timeout already occurs for the first search step (i.e. depth 2) the elapsedTime vector will be resized with 2 iterative_deepening_search.cc:268) and maxSearchDepth is set to 1 (line 269). If the next training state is selected, estimateQValues starts a DFS with a depth of 2 (line 201/202). MoreIterations is called with a depth of 2 and will call line 264: elapsedTime[stepsToGo].push_back(time);,
where stepsToGo is 2, but the size of elapsedTime is 2 (because of elapsedTime.resize(2)), i.e. an out of bounds call.

I think the memory corruption only shows its effect (at least the segfault) when some caches grow too large and are rehashed and specific cache indices are called afterwards, which is why the debug version was not able to detect this segfault (this was fun to find out).

Original report by tkeller (Bitbucket: 557058:280236d3-4090-4dc9-9a03-b6e1425df4e7, GitHub: thomaskeller79).

In order to integrate the PPDDL parser, we need a common data structure both for RDDL tasks, where actions are represented by action-fluents, and PPDDL tasks, where actions are explicitly defined via precondition-effect pairs. As a first step, we'd like to create separate CPFs, SACs and reward formula for each action.

Original report by geisserf (Bitbucket: 557058:e7a9f9a5-3ea8-4154-97d2-10446425dce3, GitHub: geisserf).

Currently random numbers are generated by calling the rand() method, which is terrible.

Besides implementing the generation of random numbers by using , we want a utility class to generate random numbers. The baseline implementation should use the common c++11 way to generate random numbers, but we want to be able to insert some "fake" random engine. This may be useful for testing or for other experimental stuff.

Original report by tkeller (Bitbucket: 557058:280236d3-4090-4dc9-9a03-b6e1425df4e7, GitHub: thomaskeller79).

Action fluents can be non-boolean just like all other fluents in RDDL, but the planner only supports boolean actions at the moment. (issue from 20.02.2014)

Original report by tkeller (Bitbucket: 557058:280236d3-4090-4dc9-9a03-b6e1425df4e7, GitHub: thomaskeller79).

Florent Teichteil-Koenigsbuch recently provided a patch (which is attached) that allows to run the planner on Windows. We should evaluate the planners performance on Windows before we incorporate the (awesome, thanks Florent!) addition into the main repo.

Original report by tkeller (Bitbucket: 557058:280236d3-4090-4dc9-9a03-b6e1425df4e7, GitHub: thomaskeller79).

Stochastic condition of the form if(Bernoulli(0.75)) then... violate an assertion even though the term is legal. (issue from 08.10.2013)

Original report by tkeller (Bitbucket: 557058:280236d3-4090-4dc9-9a03-b6e1425df4e7, GitHub: thomaskeller79).

This is not really an issue, but the name of the repository branch that contains all changes that are done with respect to IPPC 2014. (issue from 23.04.2014)

Original report by tkeller (Bitbucket: 557058:280236d3-4090-4dc9-9a03-b6e1425df4e7, GitHub: thomaskeller79).

Learning makes the IDS non-deterministic, which cannot be fixed easily in general as the current setup is really strong. A solution would be to create the possibility to skip the learning process altogether and set the max_search_depth manually for testing / debugging purposes.

Original report by tkeller (Bitbucket: 557058:280236d3-4090-4dc9-9a03-b6e1425df4e7, GitHub: thomaskeller79).

Here in Basel, we have recently changed the grid engine running our cluster from Sun Grid Engine to Slurm. It'd be good to support both.

Original report by geisserf (Bitbucket: 557058:e7a9f9a5-3ea8-4154-97d2-10446425dce3, GitHub: geisserf).

We decided to use clang-format to enforce a certain code style for PROST. After this issue is completed we should have:

• Removed the uncrustify config file
• Inserted the .clang-format config file in /src
• Some way of enforcing the coding style before a push is accepted

The first two points are trivial, attached to the issue is the clang-format style file.

The third point is one worth of more discussion:

• How do we want to reject pushes?
• Do we only check for the files which are changed? (This would allow us to close this issue before we converted everything into the new style. Old files will then get updated, whenever someone modifies them.)
• Do we want a script that modifies commited files by itself before the push, or that tells the contributor which lines of code break the code style and require him to fix it (or to manually call a script)

This issue is related to issue #24.

Original report by geisserf (Bitbucket: 557058:e7a9f9a5-3ea8-4154-97d2-10446425dce3, GitHub: geisserf).

Looks like clang-format was also applied to the gtest directory, which apparently broke gtest. Should be simple to revert the gtest files and exclude the gtest directory from clang-format checks.