optimizer: push any filter down to storage about risinglight HOT 3 OPEN

What's the difference between filtering arbitrary expression on the executor and filtering on the storage? Is that really helpful to push it down? I can only understand that range filters on primary key are more efficient. That's why I didn't add rules to push any filter down. 🤪

from risinglight.

Filters with high selectivity can also be super efficient if it's pushed down to the storage side. Consider two examples:

1. The current storage engine supports conditional read. That is to say, if we have something like select * from t where a > 10 and b < 10 even if they are not pks, we will first read a block of column a and column b, and if all data in that block does not match the condition, we will not read the remaining columns and will move the cursor.
2. Storage (may) support filtering by metadata, whereas we store min/max for each of the 1MB storage block. Therefore, if we have a high selectivity filter like b=10 on any column, we should be able to filter a lot of things.

from risinglight.

I see. Thanks for the explanation. Then we should revert some changes from #786, where I removed the code for filtering in RowsetIterator. 🤪

from risinglight.