jpacerqueira-zz Goto Github PK

airflow-executions

Apache Airflow for K8s Clusters with Docker-compose orchestration. Example includes used in Workflows for Jobs like WebHooks and WebScrapers

akamai-log-analysis-sparkml-h2o

Transformation of Akamai Logs with Spark ETL and discover of Values and similarities in logs used SparkML and H2O ML

als-benchmark-scripts

Scripts to benchmark distributed Alternative Least Squares (ALS)

cluster-management-python-pyspark-ngrams-samples

cluster-management-python-pyspark-ngrams-samples

confluent-kafka-xperiments

Experimentation of confluent Kafka Tools and Client solutions

container_cryptominer

databricks-cloud-formation

deeplearning-malwaredetection

docker-container-jupyter

Docker-Container for Jupyter Notebooks using as a baseline hook other repo

fivecooltest

Techical assignment

hadoop

Hadoop Cloudera investigations

jpac-flume-logs

My adaptation of the flume-logs ingestion process

jpac-sparklyr

H2O and sparklyr setup in Rstudio with demo/trials for Hadoop Spark

jupyter_spark_h2o_kafka_client_setup

This is the core of project lost_saturn . The project lost_saturn project is a modern approach to datascience, focus on enabling DataScience on containerised environments everywhere. Built first with a local setup and transformed into a container solution. It has tools centralized in Jupyter , with Spark and AutoML H2O.ai . Ideal to run Notebooks in Jupyter in WSL (Windows Subsystem Linux), or Docker containers with Ubunto 18.4 LTS

mydockersetupnordvpn

project_lost_saturn

This is the core of project lost_saturn . The project lost_saturn project is a modern approach to datascience, focus on enabling DataScience on containerised environments everywhere. Built first with a local setup and transformed into a container solution. It has tools centralized in Jupyter , with Spark and AutoML H2O.ai . Ideal to run Notebooks in Jupyter in WSL (Windows Subsystem Linux), or Docker containers with Ubunto 18.4 LTS

spark-on-kubernetes

An Deployment and Setup of Apache Spark for multi-tenant usage in Kubernetes Clusters. This deploys 1 Executor per K8S POD , scales linearly.

sparkelasticsearchpublisher

Elasticsearch publisher using Hadoop as source and Spark 1.6 as ETL engine :: Running package for Cloudera CDH 5.9.0 Cluster

technical-test-jupyter-spark-delta-pandas

Technical Test Github Repo for Container of Test

tensorflowjava

TensorFlow in Java. If Google Can do it! I can Do it!

terraform_start6nodes_cdh5.xcluster

AWScli Terraform for 6 Node Cloudera CDH with Hadoop Spark Hive