Print filter again when removing rows? about tidylog HOT 3 CLOSED

I can understand where you're coming from, but it feels to me once I start to add that, then it also needs to be done with all the other commands, which would then really flood the output with information.

For long pipes that need debugging often, maybe it's better to split the pipe into smaller parts, or even to just have one command per line. See below with the %<>% operator. This way, each line is associated with its output.

library("dplyr", warn.conflicts = FALSE)
library("tidylog", warn.conflicts = FALSE)
library("magrittr")

summary <- select(mtcars, mpg, cyl, hp, am)
#> select: dropped 7 variables (disp, drat, wt, qsec, vs, …)
summary %<>% filter(mpg > 15)
#> filter: removed 6 out of 32 rows (19%)
summary %<>% mutate(mpg_round = round(mpg))
#> mutate: new variable 'mpg_round' with 15 unique values and 0% NA
summary %<>% group_by(cyl, mpg_round, am)
#> group_by: 3 grouping variables (cyl, mpg_round, am)
summary %<>% tally()
#> tally: now 20 rows and 4 columns, 2 group variables remaining (cyl, mpg_round)
summary %<>% filter(n >= 1)
#> filter (grouped): no rows removed

^{Created on 2019-02-16 by the reprex package (v0.2.1)}

from tidylog.

Thanks for your explanation. I understand why baking this in would mean basically printing the entire pipe again.

There might be a way to do this with verbosity options though, no?

To give you a little context, the way I use R, I don't want to work with variables such as summary in your example—to me it seems that a pipe should do an operation that returns data frames or values in a well-defined state, whereas in the above case, I can never be quite sure what's inside summary at any given point.

Perhaps what I'm really looking for would be a log like:

mtcars %>%
    select(, mpg, cyl, hp, am) %>%
    filter(mpg > 15) %>%
    mutate(mpg_round = round(mpg)) %>%
    group_by(cyl, mpg_round, am) %>%
    tally() %>%
    filter(n >= 1)

#> select: dropped 7 variables (disp, drat, wt, qsec, vs, …)
#> filter: removed 6 out of 32 rows (19%), used filter(s): mpg > 15
#> mutate: new variable 'mpg_round' with 15 unique values and 0% NA
#> group_by: 3 grouping variables (cyl, mpg_round, am)
#> tally: now 20 rows and 4 columns, 2 group variables remaining (cyl, mpg_round)
#> filter (grouped) n >= 1: no rows removed

But I understand that this might not be what you originally intended.

from tidylog.

I get what you're saying. That's the point of using pipes, and I wouldn't recommend the style in my earlier response generally -- but when a pipe becomes too long to be debugged easily, then it's worth thinking about splitting the pipe.

Anyway, this is a bit besides the point. One solution would be to make this an option, but then I don't think it would make sense to just make it available for filter. I could potentially imagine something like you see below, where first the command is repeated and then the tidylog message is printed on the next line:

mtcars %>%
    select(, mpg, cyl, hp, am) %>%
    filter(mpg > 15) %>%
    mutate(mpg_round = round(mpg)) %>%
    group_by(cyl, mpg_round, am) %>%
    tally() %>%
    filter(n >= 1)
#> select(mpg, cyl, hp, am)
#> -> dropped 7 variables (disp, drat, wt, qsec, vs, …)
#> filter(mpg > 15)
#> -> removed 6 out of 32 rows (19%)
#> mutate(mpg_round = round(mpg)
#> -> new variable 'mpg_round' with 15 unique values and 0% NA
#> group_by(cyl, mpg_round, am)
#> -> 3 grouping variables (cyl, mpg_round, am)
#> tally()
#> -> now 20 rows and 4 columns, 2 group variables remaining (cyl, mpg_round)
#> filter(n >= 1) 
#> -> within groups, no rows removed

The downside would be that when there is only one statement, tidylog would print a lot of redundant information:

filter(mtcars, mpg > 15)
#> filter(mpg > 15)
#> -> removed 6 out of 32 rows (19%)

However, this is not a priority for me right now. But it's maybe worth thinking a bit more about this.

from tidylog.