Name: Gonzalo Bautista
Type: User
Company: @Bluetab
Bio: Master in Big Data and Chemical Engineer. Apache Spark, HDFS, Kafka, SpringBoot, Flume, SQL, Cassandra, Mongodb,...
Scala2.8, Java11 and Python3
Location: Madrid, Spain
Blog: https://es.linkedin.com/in/gonzalobautista
Gonzalo Bautista's Projects
Run a local HDFS container quickly for tests
Fetch data from Twitter using Flume. It will be sent to a Kafka topic and to a HDFS data warehouse.
fetch live data about likes and comments to an Instagram account
Fetching instagram data from httpInstaGet and storing it to Cassandra
Some C.R.U.D operations using Java API of MongoDB
A brief example of unit testing of pyspark with unittest module and using mocks.
Spark(v2)-Scala template project. Different modules for SparkSession, configuration file, load dataframe and jobs
Batch processing example using Apache Spark Python Api
Using Kafka to filter streaming data received with flume.We minimize large json lines to get only the selected fields.
Fetching instagram likes from httpInstaGet via Kafka and produces a csv with the coordinates of the pictures uploaded by the profiles who liked.