This tutorial is inspired by the Microsoft's example how to copy data in Data Factrory with a control table https://learn.microsoft.com/en-us/azure/data-factory/solution-template-delta-copy-with-control-table.
As this is a typical task during the data ingestion in a daily life of a data engineer I have decided to create an Azure independent pipeline with can be further adopted.
- installed docker-compose
Run command:
docker-compose up
Open appeared link in console in browser
http://127.0.0.1:8888/?token=YOUR_TOKEN
Execute cells in the Jupyter Notebook and observe outputs.