Update Transformers4Rec to use new dataloader package about dataloader HOT 2 CLOSED

T4Rec has T4RecTrainingArguments(data_loader_engine='nvtabular', ...). Is the goal to add an option for data_loader_engine='dataloader'? Should dataloader become the new default?

from dataloader.

The NVTabularDataloader class is defined here as an extension of the class TorchAsyncItr defined in NVTabular. This class was already updated to use the new data loader so I believe T4Rec is currently using the new data loader.

Two possible enhancements that I can think of are:

• removing the explicit import of nvtabular and defining the T4Rec data loader directly from merlin.loader.torch.Loader ( taking into account the additional line of TorchAsyncItr)
• renaming the option nvtabular in the data_loader registry to something like merlin-loader?

from dataloader.