Interoperability between DataFrames using different query compilers about modin HOT 6 OPEN

In terms of testing, I think unit tests are the way to go. We don't need to test every combination of APIs, as long as the conversion is working properly. We can add some canary testing on one or two APIs to ensure that end to end is working properly. Does this make sense?

from modin.

@devin-petersohn, thanks for the suggestion! It does make sense. We can start with unit tests to verify if API works with a single set of parameters.

from modin.

It seems to me we could have unit tests even for the first impl. Just copy tests from the dataframe folder to another one and leave a single set of parameters for every tests. @devin-petersohn, what do you think?

from modin.

@arunjose696, thanks for your research. I think we should copy those tests to a different directory (e.g., modin/tests/pandas/native_df_mode) and update them to specifically test interoperability. This way, we would not bloat up existing tests and would make navigation for interoperability tests easier.

from modin.

After the first implementation of small QC is done, I will open a PR with interoperablilty and have unit tests to verify if the API works with single set of parameters.

For the first implementation #7259., would it suffice to go with tests in modin/modin/tests/pandas/dataframe folder for now by setting the MODIN_NATIVE_DATAFRAME_MODE, to verify the query compiler works for dataframes, or should we add unit tests even for the initial implementation?

from modin.

That makes sense to me. Thanks @YarShev and @arunjose696 !

from modin.