Code Monkey home page Code Monkey logo

jarvis_data_eng_danny's Introduction

Danny To . Jarvis Consulting

I am a new Computer Science graduate and have been working with Jarvis Consulting. I obtained my Bachelor of Science from University of Ontario Institute of Technology, and worked a brief internship with Tyco International. As a new graduate, software and data engineering seems like the perfect intersection of everything we were taught to appreciate - technical challenges that are best solved with an analytical approach. Of course, there is also the excitement of being able to work with emerging technologies in a thriving industry.

Skills

Proficient: Java/Springboot, Linux/Bash, RDBMS/SQL, Agile/Scrum, Git, Algorithms/Data Structures, Docker, REST APIs

Competent: Python, Data Analytics (Zeppelin, Jupyter), OOP, JUnit, HTML/CSS

Familiar: C++, Javascript, Perforce, Google Cloud Platform, Jenkins

Jarvis Projects

Project source code: https://github.com/jarviscanada/jarvis_data_eng_danny

Cluster Monitor [GitHub]: Developed a monitoring agent to obtain hardware usage stats across a cluster of nodes. All data is automatically pulled by a bash script and inserted into a central PostgreSQL database. Also wrote some sample analytics queries.

Core Java Apps [GitHub]:

  • Twitter App: Developed a CLI app that interacts with Twitter's REST API by forming arguments into HTTP requests. Basic functionality includes posting, deleting, and looking up Tweets. Features dependency injection using Spring framework. Also wrote a number of integration and unit tests using both JUnit and Mockito.
  • JDBC App: Developed a simple app that can connect to a PostgreSQL database using JDBC. Enables users to perform CRUD operations on the database using the DAO pattern.
  • Grep App: Wrote a Grep-like Java app that searches for a regex match within all files in a given directory. Results are written to a specified output file. Utilizes Java 8 Stream APIs to read files.

Springboot App [GitHub]: Developed a SpringBoot application that simulates stock trading using data acquired from a REST API. The application follows a three-tiered microservice architecture, and is built with Spring and Maven providing dependency management. Market data is retrieved from the IEX Cloud REST API and persisted into a PostgreSQL container. Simulated trader accounts and order data are also persisted. JUnit and Mockito were used for integration and unit testing. The application was Dockerized into two separate containers for ease of deployment.

Hadoop [GitHub]: Provisioned a Hadoop cluster on Google Cloud Platform to evaluate data analytics tools and Hadoop's central components. Performed Apache Hive queries on the provided dataset through Zeppelin notebook and evaluated performance differences of storing data in different formats and with partitions. Techniques that were explored in particular included dynamic partitioning of tables, and query performance improvements when using columnar files.

Cloud/DevOps [GitHub]: Deployed Springboot application onto Amazon Web Services. Configured environment to take advantage of AWS features like auto-scaling and automated deployment. Worked with Jenkins to create a CI/CD pipeline to automatically perform builds and deployments.

Spark [GitHub]: Evaluated Spark as a big data analytics tool. Provisioned clusters on both Google Cloud and Azure Databricks. Performed analytics on datasets using Spark's Structured APIs and Pyspark.

Python Data Analytics [GitHub]: Performed analytics on a sample dataset provided by a web-based retailer to answer crucial business questions. Worked with Pandas DataFrame in a Jupyter notebook.

Highlighted Projects

FlipFlop [GitHub]: Developed a prototype Reddit-like app developed on Android studio. Users can post, comment on posts, and vote. All post and user data was sent to a realtime database running on Google Firebase. Data relevant to the current user was stored locally using SQLite.

Wildfire Simulation [GitHub]: Designed a naive, 2D simulation of wildfire spread programmed in Python and realized using cellular automata. Also implements a 2D cave-generation algorithm to generate a mock forest for testing.

Professional Experiences

Software Developer, Jarvis (2020-present): Developing projects that require the use of different technologies such as PostgreSQL, Java Stream APIs, REST APIs, and Spring framework. Participating in the Scrum to become familiar with development cycles and working in a professional team environment.

Software Engineering Intern, Tyco International (2016-2016): Developed an automated test library using Python and Robot framework. Ran detailed regression testing in accordance with specified parameters, and communicated recurring issues to my team. Also helped train team members in the use of the new test library, and monitored their progress to ensure understanding.

Education

Ontario Tech University (2015-2019), Bachelor of Science (Hons), Computer Science

  • GPA: 3.2/4.0

Miscellaneous

  • Dean's Honours List (2015, 2016)

jarvis_data_eng_danny's People

Contributors

danny-to7 avatar jarviscanada avatar

Watchers

James Cloos avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.