simonbowly / grblogtools Goto Github PK
View Code? Open in Web Editor NEWThis project forked from gurobi/gurobi-logtools
Extract and visualize information from Gurobi log files
License: Apache License 2.0
This project forked from gurobi/gurobi-logtools
Extract and visualize information from Gurobi log files
License: Apache License 2.0
See failing regression tests on master: missing Log, LogFilePath, ModelFile, Seed in the timelines dataframes. How much of this information should be included?
To finalise this:
Replicate behaviour of merged_logs=True. This is a pretty small change to the main loop, just needs to keep a new SingleLogParser in reserver to check for the start of a new log.
We have many sections like
for pattern in PresolveParser.presolve_intermediate_patterns:
match = pattern.match(line)
if match:
self._summary.update(typeconvert_groupdict(match))
return True
which could probably be turned into a utility function, e.g.
def run_regexes(patterns, line, target):
for pattern in patterns:
match = pattern.match(line)
if match:
target.update(typeconvert_groupdict(match))
return True
return False
Should this be done by header, presolve, or in post-processing?
The regexes would be better as globals in their respective modules, putting them under classes makes them ambiguous to address (lead with class name or self?).
glass4 relaxation is too easy and we have no pure LPs, so there is currently no simplex/barrier log data in the repo for testing. Would like to add this to the regression tests before changing this part of the parser.
Let's do this in a meeting to avoid future merge conflicts.
This should cover:
First thought is that this should be a composition of other parsers for continuous models, e.g.
class RelaxationParser:
def __init__(self):
self.barrier_parser = BarrierParser()
self.simplex_parser = SimplexParser()
def start_parsing(self):
# custom matchers for concurrent indicator lines / root node info
# defer to barrier or simplex parsers for who should take over from then
def continue_parsing(self):
# defer to active barrier or simplex parser
# watch out for termination lines like "barrier solved model" from concurrent solves
To discuss - should we now merge continue_parsing and start_parsing to a single parse() method for most classes?
The return value meaning for both are now the same (True = the line matched something).
Classes could keep internal state so that they know whether they have been "started", but this is not strictly necessary, it would just reduce the number of patterns to check initially. It's perhaps better for performance oriented things like this to be handled internally.
This could be much cleaner now, without column regexes, since we store parameter values in a separate dataframe.
From discussion in #22
Simon:
Integrating the termination parser with the different termination messages from barrier and nodelogs is looking tricky. Switching the termination parser settings on the fly once we determine the model type doesn't seem to fit very well. What do you think of the following?
- Include "Optimal objective" in the barrier parser
- Include "Best objective/best bound/solution count" as well as "Optimal solution found" in the nodelog parser, since they are specific to models with a node log
- Instead of the termination parser, have a status message parser which picks up "Time limit reached", "Interrupted" and so on (without interrupting other parsers).
- In the case of a MIP where barrier solves the relaxation but the node processing times out, when merging all the log info, Status="TIME_LIMIT" from the status parser would then taken precedence over Status="OPTIMAL" coming from the barrier parser, and we would get the right result.
Maliheh:
One other idea to address the termination stage is to treat this parser differently because the termination messages might be scattered throughout the log.
The idea would be that we do not include the termination parser in the list of future parsers. In the method continue_parser() of the SingleLogParser(), if the line is not already matched with a pattern in the current parser and if the line does not match the start pattern of any other parser, we pass the line to the termination parser (we just need to have one method named parse for this parser) and save the information in a dict (the same as we do in the other parsers).
This will work because the lines that appear later in the log file will always overwrite whatever optimal objective value and status we have already saved in the termination parser.
ParseResult code could be tidied up a bit - a pandas join could be used to add common data to the progress log.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.