Filter the simulation state takes hours about subt HOT 9 CLOSED

Original comment by Martin Dlouhy (Bitbucket: robotikacz).

• Edited issue description

from subt.

Original comment by Martin Dlouhy (Bitbucket: robotikacz).

OK, submitted "raw" status was gziped to 229MB and filtered version 22MB, but the conversion took 8 hours! The simulation itself was around 50 minutes.

It is really necessary to re-sample original poses to 60Hz?

from subt.

Original comment by Martin Dlouhy (Bitbucket: robotikacz).

p.s. simple reading original gazebo log file with Python sax, concatenating blocks, zlib.uncompress and dump sizes of each chunk took 32.520 seconds ... so if "filtering" should just drop some iterations it should be much much faster. Also I am not sure if it is necessary to dump pose of each tile for every iteration? In other words the filtered poses file could be probably much smaller.

(example how to extract readable XML is here: robotika/osgar@bbd26b5)

from subt.

Original comment by Zbyněk Winkler (Bitbucket: Zbyněk Winkler (robotika)).

I am working on a computer I have just bought for this. It is fairly high end
Intel(R) Core(TM) i5-8300H CPU @ 2.30GHz
4 cores, 2 threads each, turboboost up to 4Gz.

Never the less it still takes hours to do such a simple thing. There must be a bug of some kind because also loading the original log into gazebo does not work (or I haven't waited long enough).

from subt.

Original comment by Zbyněk Winkler (Bitbucket: Zbyněk Winkler (robotika)).

Can we use

  --record_period arg (=-1)     Recording period (seconds).
  --record_filter arg           Recording filter (supports wildcard and regular expression).

directly when launching gazebo? That would provide the log with the information that is needed for submission without the need for any filtering at the end.

from subt.

Original comment by Zbyněk Winkler (Bitbucket: Zbyněk Winkler (robotika)).

I tried using "-r --record_period 0.0167 --record_filter *.pose/*.pose" for the extra_gazebo_flags and the generated log file seems to have about the right size (similar to the filtered one, which btw took 362 minutes to create).

from subt.

Original comment by Nate Koenig (Bitbucket: Nathan Koenig).

We are in the process of moving a new logging system, which should bypass the filtering requirement. This new logging system will be available in May. Until then, please try to work through the current constraint.

from subt.

Original comment by Nate Koenig (Bitbucket: Nathan Koenig).

• changed state from "new" to "resolved"

The subt code base has transition from gazebo9 to ignition, which utilizes a different logging strategy. Filtering is no longer required, and the log files produced by simulation can be submitted as-is.

from subt.

Original comment by Alfredo Bencomo (Bitbucket: bencomo).

• changed state from "resolved" to "closed"

from subt.