Dear Nobuhiro,I think that they can easily use the new format in the viewer.I have prepared the javascript wrapper for the new log format:The source code is available on https://github.com/roborescue/rcrs-web-viewer/tree/protobuf-log-v4/test-protobuf

The backward compatibility is not possible for the moment, However, I will try to add a convertor to convert old format to the new format. by the way I prefer to go forward.

You can visit https://roborescue.github.io/rcrs-web-viewer/test-protobuf/log-reader-protobuf.html to test the code (please press ctrl+shift+i to see debug console and then click on the console to view the logs)

For instance, you can see in the codethere is the following   

switch (log.getLogCase()){
  case proto.LogProto.LogCase.START:
	  console.log("Start Record is Find",log.getStart());
	  break;
  case proto.LogProto.LogCase.COMMAND:
	  console.log("Command Record is Find",log.getCommand());
	  break;
  case proto.LogProto.LogCase.CONFIG:
	  console.log("Config Record is Find",log.getConfig());
	  break;
  case proto.LogProto.LogCase.INITIALCONDITION:
	  console.log("Initialcondition Record is Find",log.getInitialcondition());
	  break;
  case proto.LogProto.LogCase.UPDATE:
	  console.log("Update Record is Find",log.getUpdate());
	  break;
  case proto.LogProto.LogCase.PERCEPTION:
	  console.log("Update Record is Find",log.getPerception());
	  break;
  case proto.LogProto.LogCase.END:
	  console.log("End Record is Find",log.getEnd());
	  break;
}

For each log type we need to handle different things:

Start

nothing!

INITIALCONDITION related to this file

let entities=log.getInitialcondition().getEntitiesList()
// for each entity you need to handle entities

Entity

function processEntity(e){
   let entityId=e.getEntityid();
   let urn=e.getUrn();
   let properties=e.getPropertiesList();
}

Properties

function processProperty(p){
   let urn=p.getUrn();
   let isDefined=p.getDefined();
   let value=null;
  switch (p.getValueCase()){
    case proto.PropertyProto.ValueCase.BOOLVALUE:
	    value=p.getBoolvalue();
	    break;
    case proto.PropertyProto.ValueCase.BYTELIST:
	    value=p.getBytelist();
	    break;
    case proto.PropertyProto.ValueCase.DOUBLEVALUE:
	    value=p.getDoublevalue();
	    break;
    case proto.PropertyProto.ValueCase.EDGELIST:
	    value=p.getEdgelist().getEdgesList();
	    break;
    case proto.PropertyProto.ValueCase.INTLIST:
	    value=p.getIntlist();
	    break;
    case proto.PropertyProto.ValueCase.INTMATRIX:
	    value=p.getIntmatrix();
	    break;
    case proto.PropertyProto.ValueCase.INTVALUE:
	    value=p.getIntvalue()
	    break;
    case proto.PropertyProto.ValueCase.POINT2D:
	    value=p.getPoint2d()
	    break;
  }
}

EdgeList:

function processEdgeList(edges){
    //for e in edges:
          e.getStartx();
          e.getStarty();
          e.getEndx();
          e.getEndy();
          e.getNeighbour();

Point2D:

function processPoint2D(p){
          p.getX();
          p.getY();

UPDATE

update=log.getUpdate()
time=update.getTime()
changeset=update.getChanges()

PRECEPTION

perception=log.getPerception()

let entityId=perception.getEntityid()
let perception.getCommunicationsList()
let time=perception.getTime()
let changeset=perception.getVisible()

changeset

     deletedIds=changeset.getDeletesList()
     changes=changeset.getChangesList()
    // for c in changes:
        c.getEntityid()
        c.getUrn()
        c.getPropertiesList()

COMMAND

command=log.getCommand()
command.getTime()
//for c in command.getCommandsList():
      urn=c.getUrn();
      components=c.getComponentsMap().map_;
      componentKeys=Object.keys(components)

CONFIG

let config=log.getConfig().getConfig().getDataMap().map_;
let keys=Object.keys(config);
// get a specific value: 
     timesteps=config['kernel.timesteps'].value

I will provide a simple HTML file for you soon

from rcrs-server.

@NONONOexe It is not related to using LZMA at all.
for a temporary solution, you can comment the following line and it will resolve the issue:

I will work on fixing the problem soon.

from rcrs-server.

Hi, I'm a student at Nobuhiro's lab.
I'm helping to investigate the impact of the change to a new-style log using Google protobuf.
Thanks for sharing the information about the new-style log so quickly.

Programs such as logviewer.sh, logextract.sh that read the logs in our environment are running slowly and consuming a lot of memory.
I suspect this has something to do with streaming lzma files.
If you know of a solution, could you please let me know?

from rcrs-server.

@NONONOexe Thank you for your comment. Would you please share your new logfile so I can see the problem.

I have created a working example in javascript.
https://roborescue.github.io/rcrs-web-viewer/test-protobuf/completeExample.html

I also added a library named RCRSLogReader.js
It decompresses the log file creates a world model for each cycle and shows the information in the HTML.

I hope that it will help you a lot.

You can also see the URNs in URN.js file

from rcrs-server.

Dear Ali,
Thank you for your comments.
We need to improve our system. So I understand what you say, well.

While it is important, we should maintain our server carefully.
Therefore, when we update the server, we need to minimize the negative influence of the updates.
It is important to share the contents of changes and consensus for updates in this view.

In this update to ver.2, we do not know the details of the changes, although we know the draft.
If we have shared them, developers can respond to the updates.

Besides, our resources of past RoboCup are significant.
Especially, nobody can create the log files concerning past contests, although we can apply past agent programs to the new version's server.

Anyway, I have to manage JapanOpen 2021 next week.
The contest is a completely remote event.
We are expected to explain/comment on some games in Japanese events.
In commenting, it is convenient to use the comparable viewer system we presented.
If impossible, I guess that we need to use the default viewer.
Is it correct?

Please let me know if you have other options.

from rcrs-server.

@modaresimr I send you a log file that is having problems playing back in logviewer.

When I try to play this log file, logviewer throws an exception and exits.
I use the -Xms option of java to specify 8 GB of memory at runtime.

from rcrs-server.

@nobuhiro-ito I have replied by Email but it seems that it is not received here:

Dear Nobuhiro,
You are right. However, changing to protobuf was necessary for developing python and a better representation for future use.

I planned to preserve the compatibility to the server but it is really time-consuming.

In any case, we can schedule an online meeting to talk about it and fix the compatibility issue.

from rcrs-server.

Dear Ali,

The latest server outputs the jlog directory in the logs directory.
Do you plan to use jlog files which is the same as RoboCup2021's log.jlog?
If so, please let me know the schedule.
And, if you have the converter from the latest version's log file to log.jlog, I'd like you to release it.

If not, we guess that you have a plan to prepare an intermediate file for a web viewer.
If so, please tell us the information about the intermediate file.

Anyway, we would like to follow the maintenance committees.
Please share your idea with us about the direction of the project.

Best regards,
Nobuhiro

from rcrs-server.

Dear Nobuhiro,
Thank you for your contact.
jLog files was a temporary file designed by one of the MCs. I think it is not good to have multiple formats for different viewers. In addition, protobuf protocol is supported in the browser and also I have prepared the lzma decompression in js for that. So I think jlog is not needed anymore.
By the way, we need to update the web viewer.

from rcrs-server.

Dear Nobuhiro,
Do you have any news?
I have updated the server to automatically select the log format using the file extension in the new_log_format branch.
It is not completely tested.
Would you please check if it is compatible with your viewer?
Please note that regarding the log format in the config it will work as follows:
.log ->old log format
.xz -> current log format
.7z -> new log format (not released)

from rcrs-server.

Dear Ali,

Thank you for your careful and kind support!
We will check your improvements.
Thank you again.

Best regards,
Nobuhiro

from rcrs-server.

from rcrs-server.