This tutorial is inspired by the Microsoft's example how to copy data in Data Factrory with a control table https://learn.microsoft.com/en-us/azure/data-factory/solution-template-delta-copy-with-control-table.

As this is a typical task during the data ingestion in a daily life of a data engineer I have decided to create an Azure independent pipeline with can be further adopted.

• installed docker-compose

Run command:

docker-compose up

Open appeared link in console in browser

http://127.0.0.1:8888/?token=YOUR_TOKEN

Execute cells in the Jupyter Notebook and observe outputs.