- ๐ Hi, Iโm @SourabhSinghRana
- ๐ Iโm interested in Data Engineering
- ๐ฑ Iโm currently learning AWS
- ๐๏ธ Iโm looking to collaborate on Data Engineering projects
- ๐ซ How to reach me (My socials are attached on my profile)
sourabhsinghrana / real-time_crypto_data_pipeline_using_kafka Goto Github PK
View Code? Open in Web Editor NEWI am using confluent Kafka cluster to produce and consume scraped data. In this project, I've created a real-time data pipeline that utilizes Kafka to scrape, process, and load data onto S3 in JSON format. With a producer-consumer architecture, I ensure that the data is in the right format for loading onto S3 by performing minor transformations
License: Creative Commons Zero v1.0 Universal