Light

James Mwangi photo

james-wachuka Goto Github PK

followers: 32.0 following: 29.0 repos: 31.0 gists: 0.0

Name: James Mwangi

Type: User

Bio: data engineer

Location: Kenya

About me

I am a passionate and skilled data engineer 👨‍💻 with three years of experience in the field. My expertise lies in using tools like GCP, Python, Spark, SQL, Talend Open Studio, Tableau, Power BI, Docker, Terraform , dbt, and Airflow . I have successfully designed and developed robust data pipelines and architectures, ensuring data integrity and accuracy ✔️. With my expertise in cloud computing platforms like GCP ☁️, I have implemented scalable solutions. I love transforming complex data sets into actionable insights using visualization tools like Tableau and Power BI 📈. I have also mastered containerization 📦, infrastructure provisioning ⚙️, and workflow orchestration 🎼. I am driven to deliver excellence in data engineering and thrive on pushing boundaries 💪. Request resume.

I also write about data engineering on

🌐communities:@DataTalksClub @TechUp Africa

📫 lets talk: mail me at [email protected] or through my socials

Skills

• Data Modelling • DataOps • Data versioning and source control • Cloud data engineering • Extract Transform Load (ETL) • Data Warehousing • Data stream and real time processing • Orchestration & Workflow automation • Data governance and quality management

Tech stack

Latest Articles

Title	Link
Running Transformations on BigQuery using dbt Cloud: step by step	https://dev.to/wachuka_james/running-transformations-on-bigquery-using-dbt-cloud-step-by-step-11bo
Debugging Python Data Pipelines	https://dev.to/wachuka_james/debugging-python-data-pipelines-a-step-by-step-guide-11g7
Using pyspark to stream data from coingecko API and visualise using dash	https://dev.to/wachuka_james/using-pyspark-to-stream-data-from-coingecko-api-and-visualise-using-dash-5g43

James Mwangi's Projects

awesome-opensource-data-engineering

An Awesome List of Open-Source Data Engineering Projects

coingecko-streamapp

a streaming app and a dashboard for visualizing cryptocurrency data fetched from the CoinGecko API. The streaming app retrieves real-time cryptocurrency information using Spark Streaming and stores it in a PostgreSQL database.

data-diff

Compare tables within or across databases

dataquest_de_learningpath

code from my data engineering learning path by dataquest

dta_warehouse_example

using mysql and talend open studio to perform ETL

dta_warehouse_hive

A data warehouse implementation in hive.

event-driven-microservices

This project demonstrates an event-driven microservices architecture using Apache Kafka for event streaming and webhook integration with external services

face_det

face detection in python using Open CV

flaskapp

serving a ml model in flask

james-wachuka

mapreduce

mapreduce techniques in hadoop-joins, job counters, inputs/outputs

mentalhealth_analysis-data-pipeline

An end to end data pipeline for for mental health analysis

mysql_gcp

using airflow to extract data from mysql transform and load into bigquery

ploomber

The fastest ⚡️ way to build data pipelines. Develop iteratively, deploy anywhere. ☁️

podcasts_pipeline

Building a four-step data pipeline using Airflow to download podcast episodes.

prefect-postgresql-sensors

The prefect_postgres_sensors package provides Prefect sensors for monitoring changes or conditions within a PostgreSQL database.

pyspark_optimization

using cache/persit methods to optimize pyspark and Pyspark/SQL to query mysql database

python-kafka_distributed_task_queue

a simple implementation of a distributed task queue

python_etl

Using python-sql to create ETL between mysql and postgresql and windows scheduler to automate the job.

python_flask

python-flask basics

python_mapred

Using mapreduce in hadoop and python to score sentiments

python_tel_chatbot

python telegram chatbot using telegram API

python_tweepy

Using python and tweepy to followback friends on twitter. This task uses the windows scheduler to follow back every 5 minutes

r_examples

R- data science exercises and examples

scrape_selenium

twitter automation with selenium

shell_

first attempt at windows task scheduling

soda-core

:zap: Data quality testing for the modern data stack (SQL, Spark, and Pandas) https://www.soda.io

speeddating_r

supervised learning in R

tweepy_airflow

airflow dag that shows twitter trending hashtags every 20 mins

weatherbot

weatherbot -using weather map API and telegram API

1
2

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.