hukumka / reformation Goto Github PK

Parsing complex struct could negatively impact performance.

Lets take a look at example:

Assuming we have to parse logs of following format:

[{datetime}:{process_uuid}] {message}

With typical usage: Filter out interesting messages, then process time deltas between then, aggregating by process uuid.

Where are two ways to define struct to parse such a line:

Fully parsing:

#[derive(Reformation)]
#[reformation("[{header}] {message}", no_regex=true)]
struct LogLine {
    header: Header,
    message: Message,
}

#[derive(Reformation)]
#[reformation("{datetime}:{process_uuid}")]
struct Header {
    datetime: CustomDatetime,
    process_uuid: UUID,
}

Or only parsing message at first

#[reformation("[{header}] {message}", no_regex=true)]
struct LogLine {
    #[reformation("[^\[]*")]
    header: String,
    message: Message,
}

Problem with first one is that it is less performant then the second, if we need too look at header only 1 out of 10 lines.

Problem with second one is that types become harder to follow, and code wanting to use header would need to execute Header::parse manually.

Combining the two

The way I see solution to this problem, is to introduce wrapper LazyParse, which will allow to define struct like this:

#[derive(Reformation)]
#[reformation("[{header}] {message}", no_regex=true)]
struct LogLine {
    #[reformation("[^\[]*")]
    header: LazyParse<Header>,
    message: Message,
}

Internally, LazyParse would be implemented as

enum LazyParse<T> {
    Raw(String),
    Parsed(T),
}

And parsing from string would only create raw version, while actual value would be parsed only then first requested.

Drawbacks and extra notes

• Accessing value could return error
• To support in place parsing, where should be LazyParse counterpart that contains &'a str

Correct me if I'm wrong, but it doesn't seem that this crate supports enums in any way. Is this planned on being supported?

Example:

#[derive(Reformation)]
#[reformation("$", no_regex=true)]
Dollar{}

Expected regex:

r"\$"

Produced regex:

r"$"