Code Monkey home page Code Monkey logo

moveit_task_constructor's Introduction

MoveIt Task Constructor Framework

The Task Constructor framework provides a flexible and transparent way to define and plan actions that consist of multiple interdependent subtasks. It draws on the planning capabilities of MoveIt to solve individual subproblems in black-box planning stages. A common interface, based on MoveIt's PlanningScene is used to pass solution hypotheses between stages. The framework enables the hierarchical organization of basic stages using containers, allowing for sequential as well as parallel compositions.

Branches

This repository provides the following branches:

  • master: ROS 1 development
  • ros2: ROS 2 development, compatible with MoveIt 2 main
  • humble: ROS 2 stable branch for Humble support

Videos

Tutorial

We provide a tutorial for a pick-and-place pipeline without bells & whistles as part of the MoveIt tutorials.

Roadmap

Feedback, reports and contributions are very welcome.

The current roadmap is to replace MoveIt's old pick&place pipeline and provide a transparent mechanism to enable and debug complex motion sequences.

Further planned features include

  • Entwined planning and execution for early execution, monitoring and code hooks
  • Subsolution blending
  • Parallel planning
  • Iterative solution improvement

Ideas and requests for other interesting/useful features are welcome.

Citation

If you use this framework in your project, please cite the associated paper:

Michael Görner*, Robert Haschke*, Helge Ritter, and Jianwei Zhang, "MoveIt! Task Constructor for Task-Level Motion Planning", International Conference on Robotics and Automation (ICRA), 2019, Montreal, Canada. [DOI] [PDF].

@inproceedings{goerner2019mtc,
  title={{MoveIt! Task Constructor for Task-Level Motion Planning}},
  author={Görner, Michael* and Haschke, Robert* and Ritter, Helge and Zhang, Jianwei},
  booktitle={IEEE International Conference on Robotics and Automation (ICRA)},
  year={2019}
}

moveit_task_constructor's People

Contributors

rhaschke avatar v4hn avatar cpetersmeier avatar janebert avatar llach avatar jafarabdi avatar captain-yoshi avatar eirtech avatar j-kuehn avatar progtologist avatar henningkayser avatar videosystemstech avatar iamrajee avatar abishalini avatar danigarcialopez avatar davetcoleman avatar jspricke avatar mamoll avatar mvieth avatar voido avatar mechwiz avatar nfiedler avatar sea-bass avatar sjahr avatar stephanie-eng avatar tylerjw avatar wyattrees avatar yannickriou avatar simongoldstein avatar

Stargazers

Jihoon Oh avatar  avatar Shang-Ching Liu avatar  avatar Rekkal Mohamed Arezki avatar ShivaKPuppala avatar  avatar  avatar Xiaoyun Wang avatar ⚡Muhammed Elyamani avatar wel2018 avatar Ning Guo avatar Rod Pérez avatar Danil Pavlenko avatar TiderFang avatar  avatar corleonechensiyu avatar  avatar  avatar jirui avatar  avatar 黑白灰 avatar Yang Jichun avatar Zhu Shize avatar  avatar  avatar Buddhika Laknath avatar Bruno Schettini avatar Xinjue avatar Brady Sheehan avatar injae hwang avatar Martina Gloria avatar Danning Zhao avatar  avatar  avatar  avatar Masato Kobayashi avatar Zhanbin Yang avatar Nicola Conti avatar Chris Rowe avatar Kawin Nikomborirak avatar Héctor Herrero avatar Øystein Solbø avatar  avatar Trim Bresilla avatar Kim JeongHan avatar Yoon, Seungje avatar Haegu Lee avatar  avatar  avatar Pablo Iñigo Blasco avatar Axel Wickman avatar Zilong Chen avatar Jinlong Pang avatar Gabriele Sisinna avatar  avatar  avatar  avatar Jens Vanhooydonck avatar Faris Hamdi avatar Rinat Nazarov avatar Satyam Tiwary avatar  avatar Grzegorz Bartyzel avatar  avatar Hao Wang avatar  avatar Rolf Green avatar  avatar  avatar  avatar yangzhongyuan avatar Appselskapet.no avatar Sakura丶舞随樱雨 avatar Muhammad Arshad Khan avatar 吴凯荣 avatar Dr. Anubhav Dogra avatar Trung Kien avatar wujiajun avatar maleicacid avatar zhongcl-thu avatar starry avatar Jade Cong avatar Kazuki Shin avatar  avatar jiawei avatar  avatar tuialexandre avatar  avatar Abhishek Kashyap avatar  avatar quyinpeng avatar WANG XIAOLI avatar  avatar Reshu Singh avatar  avatar JS00000 avatar Ben Duffy avatar Felix Herbst avatar Tatsuya Matsushima avatar

Watchers

 avatar Simon Schmeisser avatar  avatar James Cloos avatar Ian McMahon avatar Michael Ferguson avatar Mario Prats avatar  avatar Justin Carpentier avatar Isaac Saito avatar Sachin Chitta avatar justiceli avatar  avatar Carlos J. Rosales avatar HY avatar G.A. vd. Hoorn avatar  avatar  avatar RyuYamamoto avatar  avatar  avatar AndyZe avatar Luca Rinelli avatar Vatan Aksoy Tezer avatar Leroy Rügemer avatar Tatsuya Matsushima avatar  avatar Jan Behrens avatar  avatar  avatar  avatar Yang Gao avatar  avatar  avatar

moveit_task_constructor's Issues

non-empty solution in `onNewSolution`

After longer execution (2-3 minutes) of my 10+ stages serial container and possibly close to the end of exhaustive planning I currently see the following quite often:

moveit_task_constructor/core/src/container.cpp:347: virtual void moveit::task_constructor::SerialContainer::onNewSolution(const moveit::task_constructor::SolutionBase&): Assertion `solution.empty()' failed.

I did not investigate further yet, because I guess you have some ideas what the problem might be @rhaschke .

Moverelative stage fails with only one joint offset entry in setDirection()

While experimenting with the Python API's demo, cartesian.py I found that in the "joint offset" stage if a dictionary containing a single entry (e.g. dict(panda_joint1=pi/6)) is passed to the setDirection function, the stage results in a failed solution attempt.

This also happens when I check out master branch, change the corresponding line (103) in cartesian.cpp so it offsets just one joint, build and try to run that code.

For both the Python and the C++ demos I used the scene from moveit_task_constructor_demo demo.launch for testing.

Object collision with itself before and after it is being attached to robot

(Unable to proceed further, need urgent help!)
I'm getting below error while running this MTC demo:
Terminal 1 : roslaunch moveit_task_constructor_demo demo.launch

Fake execution of trajectory
Found a contact between 'object' (type 'Object') and 'object' (type 'Robot attached'), which constitutes a collision. Contact information is not stored.
Collision checking is considered complete (collision was found and 0 contacts are stored)
Trajectory component '10/19 - subsolution 0 of stage 0' is invalid after scene update
Stopping execution because the path to execute became invalid(probably the environment changed)
Fake trajectory execution cancelled
Stopped trajectory execution.
Completed trajectory execution with status PREEMPTED ...

Terminal 2 : roslaunch moveit_task_constructor_demo pickplace.launch

[ INFO] [1579090525.982145137]: Planning succeded
[ INFO] [1579090525.982788967]: Executing solution trajectory
[ERROR] [1579090529.816725764]: Task execution failed and returned: ABORTED
[ INFO] [1579090529.817463899]: Execution complete

Attaching below screenshot of the same here.

Screenshot from 2020-01-15 17-55-31

My system information are as:
Ubuntu 18.04
ros melodic

UPDATE :
I found this work around. Is there any better solution?

rviz interaction

I'm struggling with some design decisions.

  1. Relationship of rviz Panel and Display
    As discussed at ROSCon, having multiple displays and a unique panel poses some challenges:

    1. Do we want several displays to visualize their (received) solution in parallel?
      I think, yes. But it might be confusing.
    2. If they do so, which (yellow) feedback should be displayed in the panel to mark the currently visualized part? From all of them in parallel? This probably will be confusing.
    3. When the user interacts with the panel and navigates solutions, I suggest to hide/block all other displays to avoid confusion. Hence, in this case there is a unique active solution. The interaction "ends", when there is no solution selected anymore. Agreed?
    4. If the interaction ends, the solution will not be shown anymore (also see #65).
    5. If there is no interaction "blocking" other displays, they will show their solution. If this replay is finished, the visualization automatically vanishes. Agreed?
    6. What about the solution slider? I suggest to have only a single slider, embedded into the panel, which is used to navigate the panel's active solution in time. However, this requires the panel to be open in order to have the slider.
  2. Visualization / Editing of stage properties

    1. There are/will be two modes of operation for the panel: a) Monitoring a remote planning process via the topics */description, */solution, */statistics and b) Editing/Executing/Monitoring an interactively configured task, running locally in rviz.
      Do we want editing for both of them? IMHO, for monitoring a remote process, a "back channel" to edit it's stage properties is not needed and eventually will interfere with an already running planning process. In local mode, editing, of course, is mandatory.

    2. In local mode, I think of custom rviz::Properties to e.g. provide an interactive marker to adjust a PoseStamped MTC property. To this end, we will need an rviz::PropertyFactory, which creates rviz properties from MTC properties.

    3. For remote mode, https://github.com/ubi-agni/moveit_task_constructor/tree/property-edit provides a basic, read-only display of properties received from the remote process via strings in */description topic. Obviously the purely string-based display is not suitable for complex hierarchical properties like geometry_msgs/PoseStamped or moveit_msgs/Constraints. Currently this string serialization simply uses the stream operator, which is provided for most basic types as well as for all ROS msg types. For ROS msgs the output format is YAML. For example, Constraints look like this:

      name: 
      joint_constraints[]
      position_constraints[]
      orientation_constraints[]
      visibility_constraints[]
      

      Hence, we could device a generic YAML -> rviz::Property converter to display all elements hierarchically. I have considered two alternatives to this:

      • Working on python integration anyway, I have a solution which uses YAML serialization of Python in C++ as well (calling the Python interpreter from C++). However, I consider this as nasty, introducing the boost::python dependency to core.
      • Using ROS msg serialization we could encode and transmit the ROS msg in binary format. However, this will render the */description topic as not human readable. On the rviz side, however we could use the same rviz::PropertyFactory used for local properties, to also display remote ones.
        Which solution is preferred?
  3. Implicit Properties
    Although properties need to be declared in stages, currently it's possible to set non-declared properties as well: Stage::setProperty(name, value) or Stage::properties().set(name, value), which implicitly declares the new property. If a property with that name already exists, a warning is issued. Question: Should we forbid this? In the past, we discussed to explicitly declare properties (and their propagation from parent -> child, or via interfaces).

fix ParallelContainer's interface methods

A parallel container's required interface is derived from the required interfaces of all of its children.
They must not conflict to each other. Otherwise an InitStageException should be thrown.

  • Implement unittests in core/test/test_container.cpp, following the example of SerialContainer, to validate correct initialization of allowed (and failure detection for forbidden) combinations. The following combinations are feasible:
    • ❘ ❘ = ❘ (connecting stages)
    • ↑ ↑ = ↑ (backward propagating)
    • ↓ ↓ = ↓ (forward propagating)
    • ↑ ↓ = ⇅ = ⇅ ↑ = ⇅ ↓ (propagating in both directions)
    • ↕ ↕ = ↕ (generating)
  • requiredInterface(): correctly derive a consistent interface from children or throw an InitStageException. Not all combinations of child interfaces are possible.
  • init(): use ParallelContainer::requiredInterface() to determine the interface
  • implement pruneInterface(): forward pruning to all children with UNKNOWN required interface, follow SerialContainer's implementation
  • fix validateConnectivity(): an exact match of ParallelContainer's interface to all children's interfaces is not required. ParallelContainer's interface, only needs to be a superset. For example, children interfaces of type ↑ or ↓ match to container's ⇅, but not vice versa.

How to use GenerateGraspPose state in Python API?

I've been trying to recreate pick_place_task using the Python API and I have the following problem, when I try to set the monitored stage for the GenerateGraspPose state:

The PredicateFilter stage is not implemented in the python wrapper, so I cannot wrap the CurrentState stage like in the cpp example. However, if I add it to the task without wrapping it, and then try to set it as monitored stage for the GenerateGraspPose state I get the following error:

Boost.Python.ArgumentError: Python argument types in
    MonitoringGenerator.setMonitoredStage(GenerateGraspPose, CurrentState)
did not match C++ signature:
    setMonitoredStage(moveit::task_constructor::MonitoringGenerator {lvalue}, moveit::task_constructor::Stage*)

What is the correct way to set a monitored stage for the GenerateGraspPose state in the Python API?

Here is the brief task definition so far:

    task = core.Task()

    task.properties.update({'group': robot_name, 'eef': group.get_end_effector_link(), 'hand': hand_name, 'hand_grasping_frame': hand_name, 'ik_frame': hand_name})

    currstate = stages.CurrentState('current state')
    task.add(currstate)  # Adding it to the task results in error for argument types in setMonitoredStage in GenerateGraspPose

    open_hand = stages.MoveTo("open hand", sampling_planner)
    open_hand.group = hand_name
    open_hand.setGoal('open')
    task.add(open_hand)

    connect = stages.Connect('move to pick', [(robot_name, sampling_planner)])
    connect.timeout = 5
    connect.properties.configureInitFrom(core.PARENT)
    task.add(connect)

    grasp = core.SerialContainer('pick object')
    task.properties.exposeTo(grasp.properties, ['eef', 'hand', 'group', 'ik_frame'])
    grasp.properties.configureInitFrom(core.PARENT, ['eef', 'hand', 'group', 'ik_frame'])

    approach_object = stages.MoveRelative("approach_object", cartesian_planner)
    approach_object.properties.update({'marker_ns': 'approach_object', 'link': hand_name})
    approach_object.properties.configureInitFrom(core.PARENT, ['group'])
    approach_object.min_distance = 0.01
    approach_object.max_distance = 0.1
    print(approach_object.properties.__getitem__('group'))   # Why is this None? how to get properties from within SerialContainer?
    approach_object.setDirection(geometry_msgs.msg.TwistStamped(header=std_msgs.msg.Header(frame_id = hand_name), twist=geometry_msgs.msg.Twist(linear=geometry_msgs.msg.Vector3(0,0,0.1))))
    grasp.insert(approach_object)

    generatepose = stages.GenerateGraspPose('generate grasp pose')
    generatepose.properties.configureInitFrom(core.PARENT)
    generatepose.properties.update({'marker_ns': 'grasp_pose'})
    generatepose.pregrasp = 'open'
    generatepose.object = 'base'
    generatepose.angle_delta = pi/12
    generatepose.setMonitoredStage(currstate)

    # To be continued
    task.add(grasp)

I am using moveit task constructor built from source at a97d860 commit of the wip-python-api branch. Any help would be appreciated.

Demo fails for modular executable

The default modular demo works. There are 2 SerialContainer added to the Task. But when adding 8 SerialContainer it fails:

Task createTask() {
	Task t;
	t.stages()->setName("Reusable Containers");
	t.add(std::make_unique<stages::CurrentState>("current"));

	const std::string group = "panda_arm";
	t.add(createModule(group));
	t.add(createModule(group));
	t.add(createModule(group));
	t.add(createModule(group));
	t.add(createModule(group));
	t.add(createModule(group));
	t.add(createModule(group));
	t.add(createModule(group));

	return t;
}

The planning works for the first 4 SerialContainer but fails at SerialContainer 5:

rviz

Attachement of MoveRelative marker

As @v4hn pointed out here, the arrow marker of a MoveRelative stage is attached to the specified ik_frame and not to the frame specified in the direction vector or twist.

As the latter can be specified w.r.t. any reference frame, I though it would make sense to display the marker relative to the ik_frame that will actually move. However, from Michael's TODO comment, I guess, he would like to have the marker attached to the direction goal's frame. If this frame is rigidly connected to the ik_frame, this would even make sense. In all other cases, we should keep the current implementation.

@v4hn It would be more helpful to file github issues, describing the problem in detail, instead of adding a TODO in the source code 😉

How to open gripper in Place stage

Hi,
Thank you for providing such a good tool!

I am trying to implement a Place stage after a Pick stage. Currently, my pipeline seems stuck in the substage "open gripper" under "ungrasp". In Pick stage, we use grasp_generator->setPreGraspPose("open") and grasp_generator->setGraspPose("closed") to define the grasp pose, but I couldn't find a corresponding command in Place stage. I think we need to tell the ungrasp generator to set the gripper pose to "open" after detaching the object. Could you tell me how to do that?

It would be super helpful if you could post some demo that uses the Place stage. Thank you in advance!

Handle Property exceptions provide useful hints

If we have a complex hierarchy of nested stages as the Pick container, propagating properties from the top-level down to the actual compute stages is a nightmare. First a capability is needed, that analyzes catched property exceptions (of different types, see below) and provides more useful hints how to resolve the issue.
The exception is usually thrown by the Property class. The (usually) calling PropertyMap class, could catch the exception, use a to-be-provided method to analyse the error and finally rethrow the exception, but with a more useful error message. The outer context could catch the exception again (placing compute() into a try-catch-block), perform more analyses, print some error messages and/or rethrow with extended message and so on.

  • Introduce different exception types, all deriving from Property::exception:
    • undeclared
    • undefined
    • type_error
  • Utility method in PropertyMap to analyze the error.
    Is the property expected to be inherited from some source, but not provided?
  • introduce a try-catch-block in Stage(Private)::compute() and continue analysis using the parent's PropertyMap.
  • For containers, provide a mechanism to expose a set of properties of specific children. Currently it's necessary to first declare all these properties (again), and second to declare them (in the child's PropertyMap) for inheritance from parent. This should be handled by a convience function in ContainerBase, e.g. exposeProperties(child, {property names}).

Add custom action between 2 stages

Is there a way to add a custom blocking action between 2 stages that doesn't rely on a trajectory? If not, could you give a few pointers for where I could implement this behaviour?

For example if I wanted to dispense some liquid (through a ros action server) after a certain stage.

I have tried adding my action between 2 tasks instead of stages as a workaround without success:

  • Create task 1 to insert a tip (for holding the liquid) and place above a container.
  • Execute task 1
  • Dispense liquid through a custom ros action server
  • Create task 2 to eject the tip in the trash (NOT WORKING)

Task 2 stages can been seen in RViz (Motion Planning Tasks) but dosent compute anything. An error is returned:

[ERROR] [1559245916.561719187]: Tried to advertise a service that is already advertised in this node [/move_group/ompl/set_parameters]

The code below implements a modified pick and place from TAMS-Group mtc_pour with 2 tasks instead of one (To validate the above workaround).

Code that fails

/** 
 *
 * TASK 1: Pick 
 * 
 */
  Task t1("pick");
  t1.loadRobotModel();

  auto sampling_planner = std::make_shared<solvers::PipelinePlanner>();
  sampling_planner->setProperty("goal_joint_tolerance", 1e-5); // 1e-5
  sampling_planner->setProperty("max_velocity_scaling_factor", 0.5);
  sampling_planner->setProperty("max_acceleration_scaling_factor", 0.5);

  // Cartesian planner
  auto cartesian_planner = std::make_shared<solvers::CartesianPath>();
  cartesian_planner->setMaxVelocityScaling(.5);
  cartesian_planner->setMaxAccelerationScaling(.5);
  cartesian_planner->setStepSize(.002);

  t1.setProperty("group", "ur3_arm");
  t1.setProperty("eef", "robotiq_2f_140");
  t1.setProperty("gripper", "robotiq_2f_140"); // TODO: use this

  Stage* current_state= nullptr;
  {
    auto _current_state = std::make_unique<stages::CurrentState>("current state");
    current_state= _current_state.get();
    t1.add(std::move(_current_state));
  }

  // open gripper
  {
    auto stage = std::make_unique<stages::MoveTo>("open gripper", sampling_planner);
    stage->setGoal("open"); 
    stage->setGroup("robotiq_2f_140");
    t1.add(std::move(stage));
  }

  {
    // TODO: overload for single planner case
    auto stage = std::make_unique<stages::Connect>("move to pre-grasp pose", 
    stages::Connect::GroupPlannerVector {{"ur3_bio_ik", sampling_planner}});
    stage->properties().configureInitFrom(Stage::PARENT); // TODO: convenience-wrapper
    t1.add(std::move(stage));
  }

  {
    auto stage = std::make_unique<stages::MoveRelative>("approach object", cartesian_planner);
    stage->setMarkerNS("approach");
    stage->properties().set("link", "robotiq_2f_140_tcp");
    stage->setMinMaxDistance(.0, .75);
    
    stage->setGroup("ur3_arm");
    stage->properties().configureInitFrom(Stage::PARENT);

    geometry_msgs::Vector3Stamped vec;
    vec.header.frame_id = "robotiq_2f_140_tcp";
    vec.vector.x = 1;
    stage->setDirection(vec);
    t1.add(std::move(stage));
  }

  {
    auto stage = std::make_unique<stages::GenerateGraspPose>("grasp workspace pose");
    stage->properties().configureInitFrom(Stage::PARENT);
    stage->setPreGraspPose("open");
    stage->setObject(id_axygen_96_rack);
    stage->setAngleDelta(M_PI); 

    stage->setMonitoredStage(current_state);

    auto wrapper = std::make_unique<stages::ComputeIK>("grasp pose", std::move(stage) );
    wrapper->setMaxIKSolutions(8);

    double gripper_offset = gripper_vertical_offset(0.0778);

    // Offset wrt object (Dont need to add support height)
    // Align approch with grasp wrt object (orientation)
    wrapper->setIKFrame(Eigen::Translation3d(0.044-0.006-gripper_offset,0,0) * Eigen::AngleAxisd(-M_PI / 2.0, Eigen::Vector3d::UnitY()), "robotiq_2f_140_tcp"); // object height - where i want to grip
    // TODO adding this will initialize "target_pose" which is internal (or isn't it?)
    //wrapper->properties().configureInitFrom(Stage::PARENT);
    wrapper->properties().configureInitFrom(Stage::PARENT, {"eef"}); // TODO: convenience wrapper
    wrapper->properties().configureInitFrom(Stage::INTERFACE, {"target_pose"});
    t1.add(std::move(wrapper));
  }



  // TODO: encapsulate these three states in stages::Grasp or similar
  {
    auto stage = std::make_unique<stages::ModifyPlanningScene>("allow gripper->object collision");
    stage->allowCollisions(id_axygen_96_rack, t1.getRobotModel()->getJointModelGroup("robotiq_2f_140")->getLinkModelNamesWithCollisionGeometry(), true);
    stage->allowCollisions(id_axygen_96_rack, id_support_axygen_96_rack, true);
    t1.add(std::move(stage));
  }

  {
    auto stage = std::make_unique<stages::MoveTo>("close gripper", sampling_planner);
    stage->setGroup("robotiq_2f_140");
    stage->properties().configureInitFrom(Stage::PARENT);
    //stage->properties().property("group").configureInitFrom(Stage::PARENT, "robotiq_2f_140"); // TODO this is not convenient
    //stage->properties().configureInitFrom(Stage::PARENT); // TODO this is not convenient
    //stage->properties().configureInitFrom(Stage::PARENT, {"eef"});
    stage->setGoal("close_rack");
    t1.add(std::move(stage));
  }

  Stage* object_grasped= nullptr;
  {
    auto stage = std::make_unique<stages::ModifyPlanningScene>("attach object");
    stage->attachObject(id_axygen_96_rack, "robotiq_2f_140_tcp");
    object_grasped= stage.get();
    t1.add(std::move(stage));
  }


  {
    auto stage = std::make_unique<stages::MoveRelative>("lift object", cartesian_planner);
    stage->properties().configureInitFrom(Stage::PARENT, {"group"});
    stage->setMinMaxDistance(.001,.75);
    stage->setIKFrame("robotiq_2f_140_tcp"); // TODO property for frame

    stage->setMarkerNS("lift");

    geometry_msgs::Vector3Stamped vec;
    vec.header.frame_id= TABLE_FRAME_LINK; // TODO change from table to object frame
    vec.vector.z= 1.0;
    stage->setDirection(vec);
    t1.add(std::move(stage));
  }

  t1.enableIntrospection();
 
  ros::NodeHandle nh("~");

  bool execute= nh.param<bool>("execute", true);

  if(execute){
    ROS_INFO("Going to execute first computed solution");
  }

  ROS_INFO_STREAM( t1 );

  // TODO: try { t.validate(); } catch() {}

  try {
    // TODO: optionally also plan stages if incoming states have infinite cost. This facilitates debugging
    t1.plan(nh.param<int>("solutions", 3));
  }
  catch(InitStageException& e){
    ROS_ERROR_STREAM(e);
  }

  if(!execute || t1.numSolutions() == 0){
    std::cout << "waiting for <enter>" << std::endl;
    std::cin.get();
  }
  else {
    moveit_task_constructor_msgs::Solution solution;
    t1.solutions().front()->fillMessage(solution);
    std::cout << "executing solution" << std::endl;
    executeSolution(solution);
    std::cout << "done" << std::endl;
    std::cout << "waiting for <enter>" << std::endl;
    std::cin.get();
  }

/** 
 *
 * TASK 2: Place 
 * 
 */
  Task t2("place");
  t2.loadRobotModel();

  t2.setProperty("group", "ur3_arm");
  t2.setProperty("eef", "robotiq_2f_140");
  t2.setProperty("gripper", "robotiq_2f_140"); // TODO: use this

  {
    auto _current_state = std::make_unique<stages::CurrentState>("current state");
    current_state= _current_state.get();
    t2.add(std::move(_current_state));
  }

  {
    auto stage = std::make_unique<stages::Connect>("move to pre-place pose", stages::Connect::GroupPlannerVector{{"ur3_bio_ik", sampling_planner}});
    stage->setTimeout(15.0);
    //stage->setPathConstraints(upright_constraint);
    stage->properties().configureInitFrom(Stage::PARENT); // TODO: convenience-wrapper
    t2.add(std::move(stage));
  }

  {
    auto stage = std::make_unique<stages::MoveRelative>("put down object", cartesian_planner);
    stage->setMarkerNS("approach-place");
    stage->properties().set("link", "robotiq_2f_140_tcp");
    stage->properties().configureInitFrom(Stage::PARENT, {"group"});
    stage->setMinMaxDistance(.0, .75);

    geometry_msgs::Vector3Stamped vec;
    vec.header.frame_id = "robotiq_2f_140_tcp";
    vec.vector.x = 1;
    stage->setDirection(vec);
    t2.add(std::move(stage));
  }

  {
    auto stage = std::make_unique<stages::GeneratePose>("place pose");
    geometry_msgs::PoseStamped p;

    tf::poseEigenToMsg(support_axygen_96_rack_pose_1*Eigen::AngleAxisd(0, Eigen::Vector3d::UnitY()), p.pose); 
    p.header.frame_id= TABLE_FRAME_LINK;

    stage->setPose(p);
    stage->properties().configureInitFrom(Stage::PARENT);

    stage->setMonitoredStage(object_grasped);
    auto wrapper = std::make_unique<stages::ComputeIK>("place pose kinematics", std::move(stage) );
    wrapper->setMaxIKSolutions(32);

    wrapper->setIKFrame(Eigen::Translation3d(0,0,-0.025) * Eigen::AngleAxisd(0, Eigen::Vector3d::UnitY()), id_axygen_96_rack);
    wrapper->properties().configureInitFrom(Stage::PARENT, {"eef"}); // TODO: convenience wrapper
    wrapper->properties().configureInitFrom(Stage::INTERFACE, {"target_pose"});
    t2.add(std::move(wrapper));
  }

  {
    auto stage = std::make_unique<stages::ModifyPlanningScene>("coll allow");
    stage->allowCollisions(id_axygen_96_rack, id_support_axygen_96_rack_1, true);    
    t2.add(std::move(stage));
  }

  {
    auto stage = std::make_unique<stages::MoveRelative>("hack put object in ", cartesian_planner);
    stage->properties().configureInitFrom(Stage::PARENT, {"group"});
    stage->setIKFrame("robotiq_2f_140_tcp"); // TODO property for frame

    stage->setMarkerNS("lol");

    geometry_msgs::Vector3Stamped vec;
    vec.header.frame_id= TABLE_FRAME_LINK; // TODO change from table to object frame
    vec.vector.z= -0.02;
    stage->setDirection(vec);
    t2.add(std::move(stage));
  }

  {
    auto stage = std::make_unique<stages::MoveTo>("release object", sampling_planner);
    stage->setGroup("robotiq_2f_140");
    stage->properties().configureInitFrom(Stage::PARENT);
    //stage->properties().property("group").configureInitFrom(Stage::PARENT, "robotiq_2f_140"); // TODO this is not convenient
    stage->setGoal("open");
    t2.add(std::move(stage));
  }

  {
    auto stage = std::make_unique<stages::ModifyPlanningScene>("allow gripper->object collision");
    stage->allowCollisions(id_axygen_96_rack, t2.getRobotModel()->getJointModelGroup("robotiq_2f_140")->getLinkModelNamesWithCollisionGeometry(), false);
    t2.add(std::move(stage));
  }

  {
    auto stage = std::make_unique<stages::ModifyPlanningScene>("detach object");
    stage->detachObject(id_axygen_96_rack, "robotiq_2f_140_tcp");
    //object_grasped= stage.get();
    t2.add(std::move(stage));
  }
  {
    auto stage = std::make_unique<stages::MoveRelative>("retreat after place", cartesian_planner);
    stage->properties().configureInitFrom(Stage::PARENT, {"group"});
    stage->setMinMaxDistance(.04,.75);
    stage->setIKFrame("robotiq_2f_140_tcp"); // TODO property for frame

    stage->setMarkerNS("post-place");

    geometry_msgs::Vector3Stamped vec;
    vec.header.frame_id= "robotiq_2f_140_tcp";
    vec.vector.x= -1.0;
    //vec.vector.z= 0.75;
    stage->setDirection(vec);
    t2.add(std::move(stage));
  }

  ROS_INFO_STREAM( t2 ); 

  try {
    // TODO: optionally also plan stages if incoming states have infinite cost. This facilitates debugging
    t2.plan(nh.param<int>("solutions", 3));
  }
  catch(InitStageException& e){
    ROS_ERROR_STREAM(e);
  }

  if(!execute || t2.numSolutions() == 0){
    std::cout << "waiting for <enter>" << std::endl;
    std::cin.get();
  }
  else {
    moveit_task_constructor_msgs::Solution solution;
    t2.solutions().front()->fillMessage(solution);
    std::cout << "executing solution" << std::endl;
    executeSolution(solution);
    std::cout << "done" << std::endl;
    std::cout << "waiting for <enter>" << std::endl;
    std::cin.get();
  }

  return 0;

WARN : Computed trajectory is too short to detect jumps in joint-space

I'm trying to pick and place two objects one after another similar to what done here in the demo. I'm getting below warning while running the same.
The computed trajectory is too short to detect jumps in joint-space Need at least 10 steps, only got 7. Try a lower max_step.

Also, there is no successful planning solution is found for the "move to pick" stage for the second object. All other planning stage has successfully found the solution. You may see full code here.
I'm attaching a screenshot of warning and planning stages.
Any help will be highly appreciated.
Screenshot from 2020-01-23 16-13-07
Screenshot from 2020-01-23 16-29-08
Screenshot from 2020-01-23 16-29-22

ComputeIK does not automatically set "target_pose" to INTERFACE anymore

Since 157caac#diff-32b368f8353f812a564f161db5e3f48fL68 ,
"target_pose" is not automatically read from interface state anymore.

This entails that pretty much every use of the stage has to include
props.configureInitFrom(Stage::INTERFACE, {"target_pose"});
now. E.g. here.

Was there a particular reason for this change @rhaschke ?

I don't see a motivation for this change and would like to get the default back.

That being said, I just forwarded master to your latest rhaschke-master with subsequent minor changes.

How to use parallel container?

I'm trying to use a parallel container to execute a very simple task(only MoveTo using PipelinePlanner) for two different panda arm. I had modified urdf and srdf accordingly and I'm able to perform the same task using the serial container(see code below) but while doing it with ParallelContainerBase I got the compile-time error and while using another wrapper of Parallel Container like Alternatives, Fallbacks & Merger I got the runtime error. I'm not sure how to proceed ahead. Any help in this regard will be greatly appreciated.

Code of the simple task is as below:
I don't know why code appearing below is not same as I pasted here, so I'm also attaching an image of the code as:
Update: Alternatively you can see the full code here.
Image version:
Screenshot from 2020-02-17 14-35-52

Text version:
// ====================== Current State ====================== //
{
auto _current_state = std::make_uniquestages::CurrentState("current state");
t.add(std::move(_current_state));
}
// ====================== test container ====================== //
{
// auto test_container = std::make_unique("test container"); //worked fine !!
// auto test_container = std::make_unique("test container"); //gave compiletime error ??
auto test_container = std::make_unique("test container"); //gave runtime error ??
// auto test_container = std::make_unique("test container"); //gave runtime error ??
// auto test_container = std::make_unique("test container");//gave runtime error ??

	// ====================== Move to Home ====================== //
	{
		auto stage = std::make_unique<stages::MoveTo>("move home", sampling_planner);
		stage->setGroup(arm_group_name_);
		stage->setGoal(arm_home_pose_);
		test_container->insert(std::move(stage));
	}
	// ====================== Move to Home 2 ====================== //
	{
		auto stage = std::make_unique<stages::MoveTo>("move home2", sampling_planner);
		stage->setGroup(arm2_group_name_);
		stage->setGoal(arm2_home_pose_);
		test_container->insert(std::move(stage));
	}
	t.add(std::move(test_container));
}

Compile time error is as below:
Image version:
Screenshot from 2020-02-17 13-57-27

Text version:
usr/include/c++/7/bits/unique_ptr.h: In instantiation of ‘typename std::_MakeUniq<_Tp>::__single_object std::make_unique(_Args&& ...) [with _Tp = moveit::task_constructor::ParallelContainerBase; _Args = {const char (&)[15]}; typename std::_MakeUniq<_Tp>::__single_object = std::unique_ptr<moveit::task_constructor::ParallelContainerBase, std::default_delete<moveit::task_constructor::ParallelContainerBase> >]’: /home/rajendra/ws_moveit/src/moveit_task_constructor/demo/src/pick_place_task8.cpp:141:81: required from here /usr/include/c++/7/bits/unique_ptr.h:825:30: **error**: invalid new-expression of abstract class type ‘moveit::task_constructor::ParallelContainerBase’ { return unique_ptr<_Tp>(new _Tp(std::forward<_Args>(__args)...)); } ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ In file included from /home/rajendra/ws_moveit/src/moveit_task_constructor/core/include/moveit/task_constructor/task.h:42:0, from /home/rajendra/ws_moveit/src/moveit_task_constructor/demo/include/moveit_task_constructor_demo/pick_place_task8.h:8, from /home/rajendra/ws_moveit/src/moveit_task_constructor/demo/src/pick_place_task8.cpp:1: /home/rajendra/ws_moveit/src/moveit_task_constructor/core/include/moveit/task_constructor/container.h:123:7: note: because the following virtual functions are pure within ‘moveit::task_constructor::ParallelContainerBase’: class ParallelContainerBase : public ContainerBase ^~~~~~~~~~~~~~~~~~~~~ /home/rajendra/ws_moveit/src/moveit_task_constructor/core/include/moveit/task_constructor/container.h:71:15: note: virtual bool moveit::task_constructor::ContainerBase::canCompute() const virtual bool canCompute() const = 0; ^~~~~~~~~~ /home/rajendra/ws_moveit/src/moveit_task_constructor/core/include/moveit/task_constructor/container.h:72:15: note: virtual void moveit::task_constructor::ContainerBase::compute() virtual void compute() = 0; ^~~~~~~ /home/rajendra/ws_moveit/src/moveit_task_constructor/core/include/moveit/task_constructor/container.h:75:15: note: virtual void moveit::task_constructor::ContainerBase::onNewSolution(const moveit::task_constructor::SolutionBase&) virtual void onNewSolution(const SolutionBase& s) = 0;

Runtime error is as below:
Image version:
Screenshot from 2020-02-17 14-14-19

Text version:
[ERROR] [1581928246.963531562]: Initialization failed: Error initializing stages: current state: required interface is not satisfied test container: start interface of 'move home' (???) doesn't match mine (?) test container: start interface of 'move home2' (???) doesn't match mine (?) test container: no child provides expected push interface

Other Information is as below
-> I'm using master branch of MTC and moveit in ros melodic on ubuntu 18.04.
Below is the two arm setup of moveit:
Screenshot from 2020-02-17 14-21-00

How to create multiple Task?

I'm trying to create two pick & place tasks for two different panda arm(same srdf, different planning group) as suggested in #131 comment.

Firstly, I tried by calling this(similar to what is being done in cartesian.cpp)
task = createTask(); task.plan(); task.introspection().publishSolution(*task.solutions().front()); twice in my code, but with this last task has overwritten the /node_name/solutions/ topic, and hence I was only able to visualise only one task in rviz which was called the lastly. Is there a way to call the two different task from the same node and publish on two different solutions topic for rviz?

Then, I also tried this
Terminal 1: roslaunch moveit_task_constructor_demo demo.launch
Terminal 2: rosrun moveit_task_constructor_demo cartesian
Terminal 3: rosrun moveit_task_constructor_demo cartesian2 (same as above but with different node name)
this way I was able to visualise planning of both arm simultaneously(by adding one more MTC viewer & clicking reset in rviz), but here how can I take the collision between two arms into consideration. Also, I doubt if this is the right way to work with multiple arms? What is the best way to work with multiple arms with MTC for both parallel and sequential task?

Also, how can we execute two different tasks (both involving different planning group) in particular manner e.g parallelly, serially etc similar to what is done with stages using containers?

Crash when OMPL_ERROR

Hi, just found MTC might crash if OMPL raises an OMPL_ERROR. Here is the error:


[ INFO] [1563330382.250656588]: Planner configuration 'manipulator' will use planner 'geometric::RRTConnect'. Additional configuration parameters will be set when the planner is constructed.
[ INFO] [1563330382.250938641]: RRTConnect: Starting planning with 1 states already in datastructure
[ INFO] [1563330382.301680845]: Constraint satisfied:: Joint name: 'shoulder_pan_joint', actual value: -0.862016, desired value: -0.861956, tolerance_above: 0.000100, tolerance_below: 0.000100
[ INFO] [1563330382.301706924]: Constraint satisfied:: Joint name: 'shoulder_lift_joint', actual value: -1.225333, desired value: -1.225381, tolerance_above: 0.000100, tolerance_below: 0.000100
[ INFO] [1563330382.301715709]: Constraint satisfied:: Joint name: 'elbow_joint', actual value: 1.545479, desired value: 1.545482, tolerance_above: 0.000100, tolerance_below: 0.000100
[ INFO] [1563330382.301723375]: Constraint satisfied:: Joint name: 'wrist_1_joint', actual value: 1.653272, desired value: 1.653322, tolerance_above: 0.000100, tolerance_below: 0.000100
[ INFO] [1563330382.301732369]: Constraint satisfied:: Joint name: 'wrist_2_joint', actual value: 0.272998, desired value: 0.272977, tolerance_above: 0.000100, tolerance_below: 0.000100
[ INFO] [1563330382.301740413]: Constraint satisfied:: Joint name: 'wrist_3_joint', actual value: 1.413279, desired value: 1.413316, tolerance_above: 0.000100, tolerance_below: 0.000100
[ INFO] [1563330382.301897636]: Found a contact between 'wrist_2_link' (type 'Robot link') and 'forearm_link' (type 'Robot link'), which constitutes a collision. Contact information is not stored.
[ INFO] [1563330382.301911324]: Collision checking is considered complete (collision was found and 0 contacts are stored)
[ERROR] [1563330382.351721638]: RRTConnect: Unable to sample any valid states for goal tree
[ INFO] [1563330382.351751785]: RRTConnect: Created 1 states (1 start + 0 goal)
[ INFO] [1563330382.351790976]: No solution found after 0.101040 seconds
[ INFO] [1563330382.351816087]: Unable to solve the planning problem
[ur5_gripper-2] process has died


I guess RRTConnect could not find a valid goal states (collision was found). However, the compute_ik returned 4 potential IK solutions. It seems MTC only tried the first one, found collision and crashed.

Screenshot from 2019-07-16 22-37-56

Unhelpful error message during task construction

We had the following error when using the task constructor. The error was due to a stage "connector" type which was missing information from the stage below. The error message was not really helpful and redrawing the task structure in the command line would be more helpful.

[ERROR] [1541083635.557176600]: Error initializing stage:
task pipeline: output interface of 'move to pre-pour pose' doesn't match mine

@v4hn

Error in execution - Could not find JointModelGroup that actuates ...

Hello,

I am trying to use MTC to do a pick/place task on a UR5 with a Robotiq 2-finger gripper. task->plan() works well for me. I can visualize the movement in Rviz after doing task->publishAllSolutions().

However, when I try to execute the robot using task->execute(* (task->solutions().front())), I got the following error:

Could not find JointModelGroup that actuates {shoulder_pan_joint, shoulder_lift_joint, elbow_joint, wrist_1_joint, wrist_2_joint, wrist_3_joint, finger_joint}

A similar issue was reported here.


After reading some source code, I found the error is from the findJointModelGroup function in execute_task_solution_capability.cpp. My solution has two sub_trajectory. If I understand it correctly, one sub_trajectory is corresponding to the arm (6 DoF), so the sub_trajectory should have 6 joint_names. The other 'sub_trajectory' is corresponding to the gripper (1 DoF), so the second sub_trajectory should have only one joint_names. However, currently, both the sub_trajectory have 7 joint_names. Neither of my JMGs has the 7 joints, so the error occurs.

I noticed there is a property called group, which seems like to indicate which JMG should be used. I printed the group property for each stage:

/ my                                                        manipulator
-- / current                                                None
-- / approach_my_pick                                       manipulator
-- / my_pick                                                None
----- / approach object                                     manipulator   
----- / grasp                                               gripper
-------- / compute ik                                                
----------- / return pre-defined grasp poses                gripper 
-------- / allow object collision                           None 
-------- / close gripper                                    gripper   
-------- / attach object                                    None    
----- /lift object                                          manipulator

The group properties seem right for me. Another thing I think might be relevant is how I declare the JMG:

    <group name="manipulator">
        <chain base_link="base_link" tip_link="fts_toolside" />
    </group>
    <group name="gripper">
        <joint name="finger_joint" />
        <joint name="left_outer_finger_joint" />
        <joint name="left_inner_finger_joint" />
        <joint name="left_inner_knuckle_joint" />
        <joint name="right_inner_knuckle_joint" />
        <joint name="right_outer_knuckle_joint" />
        <joint name="right_outer_finger_joint" />
        <joint name="right_inner_finger_joint" />
        <joint name="right_finger_tip_joint" />
        <joint name="left_finger_tip_joint" />
        <joint name="ur5_ee_link-gripper_base" />
    </group>

I got stuck here for a long time, so any suggestion or example would be greatly appreciated!

Many thanks!

Unregistered type error while running cartesian.cpp

I'm getting unregistered type error while running cartesian demo as

Terminal1: roslaunch moveit_task_constructor_demo demo.launch
Terminal2: rosrun moveit_task_constructor_demo cartesian
[ERROR] Unregistered type: St3mapINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEdSt4lessIS5_ESaISt4pairIKS5_dEEE
image

Although I can still run the demo but just wondering why I got this error and which object have this weird looking type?

Uable to execute task

I'm running cartesian task using for two panda arm in a similar way done here in the demo. Although task planning work just okay but cannot execute it as it doesn't return(I think stuck at waitForResult). Yes, I have manually included the changes made in PR #136 and confirm that task execution works perfectly and also return back for singe arm but not for my manually created two panda arm.
Could it be because I forgot to add some controller/planner(which might require for task execution) in setup assistance while creating the srdf of these two panda arms?
I'm creating, planning and executing task as below

        string s = "Task1"; 
        Task t(s);
        t.stages()->setName(s);

        //task stages defined here (planning for only one arm out of two here)

	try {
		if (t.plan()){
                        t.enableIntrospection();
			// t.introspection().publishSolution(*t.solutions().front()); //automatic display plan

			moveit_msgs::MoveItErrorCodes execute_result = t.execute(*t.solutions().front()); //stuck at this line and dont proceed further

			if (execute_result.val != moveit_msgs::MoveItErrorCodes::SUCCESS) {
				ROS_INFO_NAMED(LOGNAME, "Task t execution Failed!");
			}else{
				ROS_INFO_NAMED(LOGNAME, "Task t execution completed!");
			}
        }
	} catch (const InitStageException& ex) {
		std::cerr << "planning failed with exception" << std::endl << ex << t;
	}

Also, I can confirm that I had created the fake controller for both arms appropriately in setup assistance as shown here.

catkin build error: ‘class collision_detection::World’ has no member named ‘moveObject’

I was so impressed the "MoveIt! Task Planning" presentation at ROSCon 2018.
I'm trying to use it but I found a catkin build error as follow.

Starting  >>> moveit_task_constructor_core                                                                                 
Finished  <<< moveit_task_constructor_capabilities                 [ 7.4 seconds ]                                         
___________________________________________________________________________________________________________________________
Errors     << moveit_task_constructor_core:make /home/youtalk/ros/kinetic/logs/moveit_task_constructor_core/build.make.003.log
/home/youtalk/ros/kinetic/src/ros-planning/moveit_task_constructor/core/src/stages/fix_collision_objects.cpp: In member function ‘moveit::task_constructor::SubTrajectory moveit::task_constructor::stages::FixCollisionObjects::fixCollisions(planning_scene::PlanningScene&) const’:
/home/youtalk/ros/kinetic/src/ros-planning/moveit_task_constructor/core/src/stages/fix_collision_objects.cpp:150:30: error: ‘class collision_detection::World’ has no member named ‘moveObject’
    scene.getWorldNonConst()->moveObject(name, Eigen::Affine3d(Eigen::Translation3d(correction)));
                              ^
make[2]: *** [src/stages/CMakeFiles/moveit_task_constructor_core_stages.dir/fix_collision_objects.cpp.o] Error 1
make[1]: *** [src/stages/CMakeFiles/moveit_task_constructor_core_stages.dir/all] Error 2
make: *** [all] Error 2

Actually the World class has not moveObject method. Do you have any idea?

wip-refactor

Dear Michael,
When thinking about the ROS debug interface, I got a whole bunch of new ideas how to generalize and further improve your implementation. Hence, I started to implement some of them here:
https://github.com/ubi-agni/moveit_task_constructor/tree/wip-refactor

Currently, this branch implements some more specialized classes derived from SubTask, namely

  • PropagatingAnyWay
  • PropagatingForward
  • PropagatingBackward
  • Connecting
  • Generator

I also worked on hiding the implementation details. I think in a next step, these specialized classes can be used for task validation at runtime.

However, another generalization I want to work on next, is generic SubTaskContainers. The idea is to group subtasks in serial or parallel fashion. However, parallel subtasks require a more generalized handling of the InterfaceStateList. I have some rough ideas on that, but I need to sleep about them...
The proposed new API is open for discussion and not yet intended for merging. So, stay tuned ;-)

How to execute multiple task simonteneously?

I can execute two independent tasks on two separate panda arm serially. Now I want to execute them simonteneously. I have an blocking execute_helper function of type

moveit_msgs::MoveItErrorCodes execute_helper(moveit::task_constructor::Task& t) {
 //execute task here and return only after task execution complete
}

How can I call it twice with different inputs? I tried something as below but I dont know how can I push my costum callback function on GlobalCallbackQueue to be considered by spinner.start()

ros::AsyncSpinner spinner(2);
spinner.start();
execute_helper(task1);
execute_helper(task2);
ros::waitForShutdown();

Alternatively, how to push costum callback function in my constum queue to be used as

#include <ros/callback_queue.h>
// #include <callback_queue_interface.h> //gave error "No such file or directory" did I forget to add some dependencies?

ros::CallbackQueue my_callback_queue;
//how to push execute_helper in my_callback_queue here
//my_callback_queue.addCallback(execute_helper); // gave "no matching function" error
ros::AsyncSpinner spinner(0, &my_callback_queue);
spinner.start();
ros::waitForShutdown();

How can I decide which spinner to be used here, i.e MultiThreadedSpinner or AsyncSpinner?

Joint '' not found in model 'my_scene'

I got the following error wile experimenting with the Python API's customized version of cartesian.py:

Joint '' not found in model 'my_scene'

when trying to use moverelative with joint offsets and a Cartesian planner like this:

move = stages.MoveRelative("joint offset", cartesian)
move.group = group
move.setDirection({'b_bot_shoulder_lift_joint': pi/6, 'b_bot_wrist_1_joint': -pi/6})
task.add(move)

In the custom 'my_scene' scene I would like to use a UR5 robot with the Python API of MTC. Currently it is working with the C++ API, but using the Python API I always get the above error.

One thing I realized, is that if the specified joint name is longer than 15 characters it is not displayed properly in the error message, so for example using

move.setDirection({'b_bot_shoulder_': pi/6})

results in an error message:

Joint 'b_bot_shoulder_' not found in model 'my_scene'

and using

move.setDirection({'b_bot_shoulder_l': pi/6})

results in an error message:

Joint '??[ZUV' not found in model 'my_scene'

Segmentation fault in MoveRelative::getJointStateFromOffset

Not a moveit guru here, but I am getting a segmentation fault that starts in this line.
I am using MoveRelative::setDirection(const std::map<std::string, double>& joint_deltas) with the Panda robot trying to set a relative position for the panda_finger_joint1. L84 on the cpp returns an index (value 14) that is out of bounds in the getJointModel, which in turn sets jm to be nullptr causing a segfault in robot_state.enforceBounds(jm);
Personally I fixed this by changing L85 with

auto jm = robot_state.getRobotModel()->getJointModel(j.first);

but I am not sure that this is the best solution so I am not making a PR for this yet.

How to constraint panda arm eef in upright orientation in "move to pre-pour pose"?

I'm trying to replicate pouring task using panda arm which is done using UR5 here in mtc_pour package. Here they have constrained the "move to pre-pour position" as below.
// don't spill liquid moveit_msgs::Constraints upright_constraint; upright_constraint.name = "s_model_tool0:upright:20000:high"; upright_constraint.orientation_constraints.resize(1); { moveit_msgs::OrientationConstraint& c= upright_constraint.orientation_constraints[0]; c.link_name= "s_model_tool0"; c.header.frame_id= "table_top"; c.orientation.w= 1.0; c.absolute_x_axis_tolerance= 0.65; c.absolute_y_axis_tolerance= 0.65; c.absolute_z_axis_tolerance= M_PI; c.weight= 1.0; }

How can I impose similarly constraint on panda_link8 or panda_hand frame(whichever convenient)?
It's hard for me to do so because in the above example s_model_tool0(see below image) frame orientation is the same as world frame(so c.orientation.w= 1.0 will do the job) but for our case, panda_link8 is tilted and not in the same orientation(see below image) as of the world. I don't have advanced knowledge of tf transform, so it will be a great help if someone can point out me how do I constrain panda_link8/panda_hand to keep the bottle upright.

Screenshot from 2020-02-17 17-01-11
Screenshot from 2020-02-17 17-01-32
Screenshot from 2020-02-17 17-17-25

I'm have tried and expected something like below-given code, but it didn't work out. I have confirmed that if I don't impose any constraint on the move to pre-pose it works perfectly fine with panda arm.

`
moveit_msgs::Constraints upright_constraint;
upright_constraint.name = "panda_hand_constraints";
upright_constraint.orientation_constraints.resize(1);
{
moveit_msgs::OrientationConstraint& c= upright_constraint.orientation_constraints[0];
c.link_name= "panda_hand"; //best choice of frame to constrain
// c.link_name= "panda_link8"; //or this but seems more complex
// c.link_name= "bottle"; //seems easy but wrong as it is not a part of robot model

	c.header.frame_id= "world"; //reference frame
        
            //below quaternium is equalavent to rpy= -pi, pi/2, 0, which is the world to panda_hand orientation transfrom while picking the bottle in upright position(please check if this is correct?)
	c.orientation.w= 0; //?? what should be correct value for this 
	c.orientation.x= -0.707; //??
	c.orientation.y= 0; //??
	c.orientation.z= -0.707;//??

	c.absolute_x_axis_tolerance= 0.65; //M_PI
	c.absolute_y_axis_tolerance= 0.65; //I can confirm, if I also make both x & y tolerance equal to M_PI, Then it work fine(Which is obvious!)
	c.absolute_z_axis_tolerance= M_PI;   //is this tolerance is wrt parent frame(world) or child frame(panda_hand)??
	c.weight= 1.0;
}`

@v4hn could you please also have look at it.

@abhinand4as Subscribe this issue.

ComputeIK: initialize timeout from ros parameter kinematics_solver_timeout

Currently we use hard-coded default timeouts for all stages, 1s for ComputeIK.
When no IK can be found, this dramatically slows down computation, as the full time is used to eventually find a solution. ComputeIK should use the kinematics_solver_timeout specified in kinematics.yaml for a particular planning group, because that's a much better estimate of typical run times of the actual IK plugin.

Store stages as std::shared_ptr

... instead of std::unique_ptr. The MonitoringGenerator needs to reference another stage, which is currently only possible via raw pointers. Using std::shared_ptr as storage, MonitoringGenerators could use a std::weak_ptr to safely store the stage reference.

Also add a mechanism to retrieve child stages by name or index, possibly even recursively.

compilation problem

Hello,

I clone moveit task constructor successfully. I am going to compile it and give me error about

moveit_task_constructor/core/src/solvers/cartesian_path.cpp:72:52: required from here
/usr/include/eigen3/Eigen/src/Core/util/StaticAssert.h:32:40: error: static assertion failed: YOU_PERFORMED_AN_INVALID_TRANSFORMATION_CONVERSION
#define EIGEN_STATIC_ASSERT(X,MSG) static_assert(X,#MSG);

Please let me know, what am I doing wrong.

debug output for compatibility check

One of the remaining problems that are hard to debug occurs when Connect rejects all incoming states as incompatible.

In this case everything looks nice in RViz & the logs but connect does not produce a single solution for no obvious reason.

To debug, I used 598753e over here, but this just spams the logs when many actually-incompatible states are checked.

Because compatibility is checked before computation, we can't easily produce feedback to the user via Solutions (and comments).

We probably do not want another orthogonal mechanism in the framework either - unless we need it for something else too.

I'm unsure how to resolve this in a clean way.

allow markers to be specified w.r.t. any planning scene frame

Currently markers are specified w.r.t. planning frame. However, this condition should be relaxed.
As soon as solutions are received by rviz, the DisplaySolution constructor could transform all markers into the planning frame, using the start? scene's transforms.
We must not rely on TF here, as the planning scene is completely decoupled from current TFs.

Add to MoveIt Melodic Release

Can we add this to the moveit.rosinstall melodic release of MoveIt?

Is it time to deprecate the old pick place functionality in Melodic?

Memory leak in RViz display

I currently see a huge memory leak in RViz when I plan and visualize a task with our display.
I didn't have a chance to look into it yet though and it might be a problem of my local ROS setup.

@rhaschke you fixed memory leaks in there before, do you still observe them here?

Property's type doesn't match it's declared type

[ERROR] : Initialization failed: Error initializing stage:
generate grasp pose2: Property: type (N13geometry_msgs12PoseStamped_ISaIvEEE) doesn't match property's declared type (NSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE)

Getting above error while running in below piece of code. Unable to understand its reason? I am trying to perform a task similar to pick place demo for two different panda arm(modified the panda arm urdf and srdf). I doubt might be because of introducing extra line1. Full PickPlace code can be found here and here is the srdf & urdf of two_panda_arm. I'm using melodic, moveit master branch on ubuntu 18.04.

CODE:
`

	{// ---------------------- Generate Grasp Pose ---------------------- //
		// Sample grasp pose
		auto stage = std::make_unique<stages::GenerateGraspPose>("generate grasp pose2");
		t.properties().exposeTo(stage->properties(), { "eef", "hand", "group", "ik_frame","eef2", "hand2", "group2", "ik_frame2" }); //line1
		stage->properties().configureInitFrom(Stage::PARENT, { "eef", "hand", "group", "ik_frame","eef2", "hand2", "group2", "ik_frame2" }); //line2
		stage->properties().set("marker_ns", "grasp_pose");
		stage->setPreGraspPose(hand2_open_pose_);
		stage->setObject(object2);
		stage->setAngleDelta(M_PI / 12);
		stage->setMonitoredStage(current_state);  // Hook into current state

                // Compute IK
		auto wrapper = std::make_unique<stages::ComputeIK>("grasp pose IK2", std::move(stage));
		wrapper->setMaxIKSolutions(8);
		wrapper->setMinSolutionDistance(1.0);
		wrapper->setIKFrame(grasp_frame_transform_, hand2_frame_);
		wrapper->properties().configureInitFrom(Stage::PARENT, { "eef", "hand", "group", "ik_frame","eef2", "hand2", "group2", "ik_frame2" });
		wrapper->properties().property("eef").configureInitFrom(Stage::PARENT, "eef2");
		wrapper->properties().configureInitFrom(Stage::INTERFACE, { "target_pose" });
		grasp->insert(std::move(wrapper));
	}

`
Screenshot from 2020-02-11 11-59-06

TODOs

This issue is intended to outline the roadmap of the development in the very next future, listing and prioritizing open issues. If you intend to work on any of these issues, please create a separate issue for it and assign yourself. Maintainers will update this issue accordingly to refer to the opened PR.


  • Now: example tasks for ROS paper
    • grasp bottle - reach glass - pour - place bottle (TAMS)
    • bi-modal grasping, choosing left or right arm autonomously based on solution costs (Luca)
    • Later: bimanual grasping of a (large) object with both arms exploiting the Merge stage (Luca)

  • Implement path constraints API
    • Now: Implement Path constraints in solver API (Michael)
    • Now: Implement Path constraints for all relevant stages
    • Implement path constraints for cartesian solver
  • Implement goal constraint API

  • Now: restricted Connect
    • start-end pairs should only be connected if they ``unify''/agree in essential properties
      • poses/existence of objects
      • transforms of AttachedCollisionObjects (Bodies)
    • states spawned by referencing wrapper stages in different Generator parts should (optionally?) only connect to their corresponding counterparts.

  • #21 MonitoringWrapper: generator-style wrapper, that references an arbitrary other stage (Robert)

  • Solution Execution
    • Now: execute planned solution as full trajectory
    • Later
      • Using a ThinWrapper container around any subset of stages, the user can directly control, which (partial) solutions are considered for early execution (while planning of further stages continues).
      • Alternatively, we could define some ExecutionFlag(s) in all stages, which define the execution behavior, e.g. allowing / postponing early execution.
      • Define behavior if execution fails for some reason. Probably, the error handling strategy needs to depend on the type of failure. An explicit ExecuteWrapper stage could define hooks (as virtual methods) to implement an arbitrary complex error handling behavior, including e.g. continuing / replan from current state, or reverting to a previous stage (executing the reverse trajectory), or propagating the error upwards the task hierarchy. This finally allows to implement arbitrary (hierarchical) state machine behavior.
      • Allow to define a different set of controllers for each JMG in different ExecuteWrapper stages.

  • rviz GUI (Jan)
    • Now: icons indicating the determined control flow (Jan)
    • Now: establish a signal-slot-connection from TaskSolutionVisualization to BaseTaskModel to highlight the currently active stage during solution replay.
    • Later, use rviz' mechanism to discover SolutionDisplay from available topic types, see https://github.com/ubi-agni/tactile_toolbox/tree/kinetic-devel/rviz_tactile_plugins for an example (Luca?)
    • Later, save state of splitters and column widths in the panel
    • Much later: PropertyMap editor: provide (eventually custom) rviz::Properties for a stage's properties (Robert + Jan)

  • Pick stage (Robert)
    • #18: Handle Property exceptions and provide useful hints how/where to resolve undeclared/unknown properties.

  • implement and validate containers
    • unittests for nesting serial containers
    • #19 unittests for parallel containers: basic + nesting
    • #19 correctly implement ParallelContainer's init() and pruneInterface()
    • implement Merger stage as a ParallelContainer
    • ThinWrapper: lightweight Wrapper implementation, not distinguishing solutions of the wrapped child and the wrapper itself. In this case, the wrapped child can directly push towards the wrapper's external interface without the overhead for synchronizing Interface lists.

  • "one of them" / "all of them" concepts
    • for-loop container
    • stage to add states with properties that indicate which is e.g. the target object


  • Scheduling
    Replace Container::compute() with schedule() method, returning a list of (priority-ordered?) stages to be scheduled for execution.
    The Task should provide a central scheduler to compute stages. During init() containers can add their children selectively to the scheduler's queue (also indicating a priority?). A stage that is dependent on another stage's solution, should only be added to the queue if the stage canCompute something.
    A SerialContainer would first schedule Generator-style stages, then propagating stages, and finally - if these created fully-connected partial solutions - Connecting stages. This indicates that containers should handle the management of the scheduling queue.
    • Different prioritization mechanisms could be used for the scheduling queue.
    • Use a ThreadPool to execute scheduled stages. This will require thread-safety throughout the API, which we didn't take care of yet.

  • Pick-and-Place
    • ObjectDetection stage, wrapping a call to an external object classifier and spawning collision objects into the planning scene
    • PlaceDetector stage, identifying horizontal surfaces for object placement, filtering them by a no-collision-with-object constraint. I believe something like this was available in the old ros_manipulation stack. (TAMS)
    • GraspService stage wrapping ros_manipulation's grasp service (Michael)
    • test AGNI's python-based grasp generator for primitive shapes as an external grasp service
    • optionally, migrate AGNI's grasp generator to C++, now exploiting goal constraints
    • GPD stage, implementing https://github.com/TAMS-Group/moveit_gpd_pick_object as a stage?

Use Grasp msg in Pick / Place stages

Pick / Place as well as SimpleGrasp / SimpleUnGrasp stages should be modified to use Grasp msg.
Currently, we have only GenerateGraspPose, which directly operates on named joint states.
However, extending this to support full Grasp msg will facilitate replacing it with more elaborate grasp providers.

We should also rethink Grasp msg:
Its fields pre_grasp_posture and grasp_posture are trajectories, which is neither reflected in the name nor in currently used semantics. I suggest to merge them into grasp_trajectory, which can be any finger trajectory. The current use case is implemented, adding pre_grasp_posture as first and grasp_posture as last waypoint of the trajectory.
Additionally, it would be useful to define an ungrasp_trajectory.

MonitoringGenerator: use solutions of a monitored stage as basis for generation

Sometimes its necessary to reuse a previously planned solution, e.g. to traverse it in reverse order or to access the state of another generator. The MonitoringGenerator stage hooks into the onNewSolution().

  • For now: reference via raw pointers
  • Later: make Stage::pointer a shared_ptr (instead of a unique_ptr)
  • Rewrite CurrentScene stage such that it fetches the current planning scene from get_planning_scene service (Luca)
  • Rewrite grasp generators to hook up on initial CurrentScene stage.
  • Rewrite Stage::init() to receive a RobotModel argument instead of the weird PlanningScene argument. All but generator stages, receive their planning scene through the Interface API.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.