Code Monkey home page Code Monkey logo

building-microservices-youtube's Introduction

Building Microservices in Go YouTube

Code repository for my Building Microservices YouTube series https://www.youtube.com/playlist?list=PLmD8u-IFdreyh6EUfevBcbiuCKzFk0EW_

Week by week Building Microservices builds on the previous weeks code teaching you how to build a multi-tier microservice system. The code structure for the course is one of a mono repo. To make it simple to follow along, each episode has its own branch showing progress to date.

Get $100 dollars of Digital Ocean credits, valid for 90 days with my referal link. Not needed for the YouTube tutorials, but helps me pay for my own servers.

DigitalOcean Referral Badge

https://m.do.co/c/c6dee99fad25

Services

Product API ./product-api

RESTful Go based JSON API built using the Gorilla framework. The API allows CRUD based operations on a product list.

Frontend website ./frontend

ReactJS website for presenting the Product API information.

Currency ./currency

gRPC service supporting simple Unaray and Bidirectional streaming methods.

Product Images ./product-images

Go based image service supporting Gzipped content, multi-part forms and a RESTful approach for uploading and downloading images.

Series Content

Over the weeks we will look at the following topics, teaching you everything you need to know regarding building microservices with the go programming language:

  • Introduction to microservices
  • RESTFul microservices
  • gRPC microservices
  • Packaging applications with Docker
  • Testing microservice
  • Continuous Delivery
  • Observability
  • Using Kubernetes
  • Debugging
  • Security
  • Asynchronous microservices
  • Caching
  • Microservice reliability using a Service Mesh

Episode 1 - Building a simple microservice

Branch: episode_1

In this episode I show you how to build the simplest service you can using the standard packages in the Go programming language.

Episode 2 - Building a simple microservice, continued

Branch: episode_2

In this episode we learn more about the standard library and look at how we can refactor last episodes example into a reusable microservice pattern.

Episode 3 - RESTFul microservices

Branch: episode_3

In this episode we start to learn about RESTFul services and reading and writing data using the JSON format.

Episode 4 - RESTful microservices

Branch episode_4

We continue to look at how you can implement RESTFul services with the Standard API

Episode 5 - Gorilla toolkit

Branch episode_5

In this epsode we refactor our Standard library RESTful service and start to implement the Gorill toolkit for routing.

Episode 6 - JSON Validation

Branch episode_6

In this episode we will look at the Go Validator package and how it can be used to validate JSON.

Episode 7 - Documenting APIs with Swagger

Branch episode_7

This epsiode shows you how you can document the API we have been building over the last couple of weeks. As a bonus we will also look at how we can embed ReDoc to build a nice documentation API direct into our service.

Episode 8 - Auto-generating HTTP client code from Swagger documentation

Branch episode_8

In this episode we look at how we can use the Swagger API documentation we created in the last episode and generate a Go client SDK. As it turns out I had a little bug in my code hope you all find the process of debugging this and finding root cause useful too.

Episode 9 - CORS (Cross-Origin Resource Sharing)

Branch episode_9

In this episode we are going to take a look at CORS (Cross-Origin Resource Sharing). CORS is a security feature built into web browsers which restricts upstream requests to sites on different domains. We look at a typical example of a React website on one domain calling a back end API, see the impact of CORS and how to solve it.

Episode 10 - Serving and uploading files

Branch episode_10

In this episode you will learn how to upload and serve files using the Go standard library.

Episode 11 - Handling multi-part form uploads

Branch episode_11

In this episode you will learn how to handle multi-part form uploads. Mult-part forms used to be common place as they are the basic way that browsers would upload data to a server. This pattern has fallen out of fashion as most moder data transfer to the server is done using XHR requests. There might still be a case when you need to know this though.

Episode 12 - Using Gzip compression for HTTP responses

Branch episode_12

In this episode we walk through how to wrap a http.ResponseWriter to enable Gzip compression for HTTP responses.

HTTP Headers Accept-Encoding: https://developer.mozilla.org/en-US/d...

HTTP ResponseWriter: https://golang.org/pkg/net/http/#Resp...

Episode 13 - Introduction to gRPC and Protocol Buffers

gRPC is a high performance framework for client server applications. It is designed to be cross platform and is an awesome alternative to RESTful services.

In this episode we take a quick look at gRPC and Protocol Buffers, and how you can use them to build a simple API. This is the first video in a series of content where we dig into gRPC services.

gRPC Framework: https://grpc.io/

Protocol Buffers v3 Language Guide: https://developers.google.com/protocol-buffers/docs/proto3

Protocol Buffers v3 Encoding format: https://developers.google.com/protocol-buffers/docs/encoding

Branch episode_13

Episode 14 - gRPC Client Connections

In this episode we take a quick look at how you can connect to gRPC services in Go.

Protocol Buffers Enum Specification: https://developers.google.com/protocol-buffers/docs/reference/proto3-spec#enum_definition

gRPC Creating gRPC Clients: https://grpc.io/docs/tutorials/basic/go/#client

Branch episode_14

Episode 15 - Refactoring Part 1/3

This video is part 1 of 3 where we start to clean up the code base a little before continuing to develop our gRPC service. Refactoring is a natural part of software development, it is difficult to get things right first time all the time. Rather than spend too much time on the perfect solution I like to go with the flow and clean up at a later date.

As part of our refactoring we look at the encoding/xml and how it is very similar in use to encoding/json.

Encoding/XML: https://golang.org/pkg/encoding/xml/

Episode 15 - Refactoring Part 2/3

In this episode I continue to refactor the code base so far. These videos are really just intended to ensure that you are not completely confused when looking at the source code changes between episode 14 and episode 16.

Source Code: https://github.com/nicholasjackson/building-microservices-youtube/tree/episode_15_2

Encoding/XML: https://golang.org/pkg/encoding/xml/

Episode 15 - Refactoring Part 3/3

In this episode I finalize the refactoring for the code base.

Episode 16 - gRPC Bi-directional streaming, part 1/2

In this video we start to look at gRPC bi-directional streaming

gRPC streaming allows you to independently receive streamed messages from the client and send a response to it. This episode looks at the basics of streaming API by adding an update to our currency service.

Server-side streaming: https://grpc.io/docs/languages/go/basics/#server-side-streaming-rpc

Client-side streaming: https://grpc.io/docs/languages/go/basics/#client-side-streaming-rpc

Branch episode_16

Episode 17 - gRPC Bi-directional streaming, part 2/2

In this video we continue to look at gRPC bi-directional streaming.

You will see how to take the simple example in the first part and how it can be implemented into the Products API to support independent client and server streams. The simple example allows a client in the Product API to subscribe for currency rate changes in the Currency service. Whenever a rate changes the currency service broadcasts this change to all interested subscribers.

Server-side streaming: https://grpc.io/docs/languages/go/basics/#server-side-streaming-rpc

Client-side streaming: https://grpc.io/docs/languages/go/basics/#client-side-streaming-rpc

Branch episode_17

building-microservices-youtube's People

Contributors

dependabot[bot] avatar mrsoftware avatar nicholasjackson avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

building-microservices-youtube's Issues

put and post not working

after swagger and the client generation branches the validator does not work after a successful product is validated, it fails a bad sku or price fine. keeps returning empty requests on posts and puts. A giant refactor without vids to back it up ๐Ÿ‘Ž

this is code that i used git clone with no code I typed myself up to episode 8 i was typing it out and it was great for learning ๐Ÿ‘

Valid request throws error: Interface conversion error is nil not validator.validationerrors for valid request body

Issue

The below validator works for invalid request body, but for a valid body, it initiates a panic

Code sample, to showcase or reproduce:

data/validation.go

func (v *Validation) Validate(i interface{}) ValidationErrors {
	errs := v.validate.Struct(i).(validator.ValidationErrors)

	if len(errs) == 0 {
		return nil
	}

	var returnErrs []ValidationError
	for _, err := range errs {
		// cast the FieldError into our ValidationError and append to the slice
		ve := ValidationError{err.(validator.FieldError)}
		returnErrs = append(returnErrs, ve)
	}

	return returnErrs
}

handlers/post.go :

func (p *Products) Create(rw http.ResponseWriter, r *http.Request) {
	prod := r.Context().Value(KeyProduct{}).(data.Product) // Panic originate here. Check below for struct definiton

	p.l.Printf("[DEBUG] Inserting product: %#v\n", prod)
	data.AddProduct(prod)
}

data/products.go

// data.Product
type Product struct {

	ID int `json:"id"` // Unique identifier for the product

	Name string `json:"name" validate:"required"`

	Description string `json:"description"`

	SKU string `json:"sku" validate:"sku"`
}

Stack Trace:

products-api 2020/07/04 17:04:27 http: panic serving 127.0.0.1:47468: interface conversion: interface {} is *data.Product, not data.Product
goroutine 50 [running]:
net/http.(*conn).serve.func1(0xc0005fe0a0)
        /usr/local/go/src/net/http/server.go:1772 +0x139
panic(0x951fe0, 0xc00030a480)
        /usr/local/go/src/runtime/panic.go:973 +0x3e3
github.com/nicholasjackson/building-microservices-youtube/product-api/handlers.(*Products).Create(0xc0003192c0, 0xaa9320, 0xc00046a000, 0xc000400400)
        /tmp/building-microservices-youtube/product-api/handlers/post.go:20 +0x349
net/http.HandlerFunc.ServeHTTP(0xc000319510, 0xaa9320, 0xc00046a000, 0xc000400400)
        /usr/local/go/src/net/http/server.go:2012 +0x44
github.com/nicholasjackson/building-microservices-youtube/product-api/handlers.(*Products).MiddlewareValidateProduct.func1(0xaa9320, 0xc00046a000, 0xc000400200)
        /tmp/building-microservices-youtube/product-api/handlers/middleware.go:42 +0x553
net/http.HandlerFunc.ServeHTTP(0xc00027c0a0, 0xaa9320, 0xc00046a000, 0xc000400200)
        /usr/local/go/src/net/http/server.go:2012 +0x44
github.com/gorilla/mux.(*Router).ServeHTTP(0xc000156300, 0xaa9320, 0xc00046a000, 0xc000400000)
        /home/ayman/go/pkg/mod/github.com/gorilla/[email protected]/mux.go:212 +0xe2
net/http.serverHandler.ServeHTTP(0xc00055c1c0, 0xaa9320, 0xc00046a000, 0xc000400000)
        /usr/local/go/src/net/http/server.go:2807 +0xa3
net/http.(*conn).serve(0xc0005fe0a0, 0xaaa460, 0xc00021c040)
        /usr/local/go/src/net/http/server.go:1895 +0x86c
created by net/http.(*Server).Serve
        /usr/local/go/src/net/http/server.go:2933 +0x35c

Could it be the version 10 from go validator has broken the implementation? I have cloned your repo and checked out episode_7 branch and it throws the above error for valid requests.

Validation logs not showing up using cURL

Hi, and thank you for the great series!

I'm currently after Episode 6, where a JSON validator was introduced to handle validation in PUT and POST requests. It does work for me as in your video but with a one difference. If I use cURL I don't see validation errors while making those requests. It returns only correct HTTP status (Bad Request) but logs only something like parse error: Invalid numeric literal at line 1, column 6. In this case I can only see a printed log message inside the terminal, where the server runs. Oddly enough it does work if using Postman or any other HTTP Client, which display that error in the response from the server.

// products.go
err = prod.Validate()
		if err != nil {
                         // get logged in terminal
			t.l.Printf("[ERROR] validating product: %s\n", err)
			// get returned in response while using Postman but not in cUrl
                         http.Error(
				rw,
				fmt.Sprintf("Error validating product: %s", err),
				http.StatusBadRequest)
			return
		}

It was enough jq to delete from the request.

issue in currency dependancy

F:\gocode\pkg\mod\github.com\nicholasjackson\building-microservices-youtube\[email protected]\data\products.go:55:11: undefined: "github.com/nicholasjackson/building-microservices-youtube/currency/protos/currency".Currency_SubscribeRatesClient

Hi, will the series continue, also will you cover deployment??

Hi Nic,

I hope you are well ๐Ÿ‘

Two important questions:

  1. Can you cover Docker and cloud deployment for microservices written in Go?
  2. Will the service continue?

Microservice deployment is super important would be great to finish this tutorial with full deployment with docker etc..

Bests,

Sean.

Delete product doesn't work properly

Given a list of products like:

โŸฉ curl localhost:9090/products | jq
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   630  100   630    0     0   615k      0 --:--:-- --:--:-- --:--:--  615k
[
  {
    "id": 1,
    "name": "Latte",
    "description": "Frothy milky coffee",
    "price": 2.45,
    "sku": "abc323"
  },
  {
    "id": 2,
    "name": "Esspresso",
    "description": "Short and strong coffee without milk",
    "price": 1.99,
    "sku": "fjd34"
  },
  {
    "id": 3,
    "name": "Esspresso",
    "description": "Short and strong coffee without milk",
    "price": 1.99,
    "sku": "fjd34"
  },
  {
    "id": 4,
    "name": "Esspresso",
    "description": "Short and strong coffee without milk",
    "price": 1.99,
    "sku": "fjd34"
  },
  {
    "id": 5,
    "name": "Esspresso",
    "description": "Short and strong coffee without milk",
    "price": 1.99,
    "sku": "fjd34"
  },
  {
    "id": 6,
    "name": "Esspresso",
    "description": "Short and strong coffee without milk",
    "price": 1.99,
    "sku": "fjd34"
  }
]

After deleting one of them the list is not updated as expected:

โŸฉ curl localhost:9090/products/2 -XDELETE
โŸฉ curl localhost:9090/products | jq
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   198  100   198    0     0  66000      0 --:--:-- --:--:-- --:--:-- 99000
[
  {
    "id": 1,
    "name": "Latte",
    "description": "Frothy milky coffee",
    "price": 2.45,
    "sku": "abc323"
  },
  {
    "id": 3,
    "name": "Esspresso",
    "description": "Short and strong coffee without milk",
    "price": 1.99,
    "sku": "fjd34"
  }
]

7-Swagger refactor

Just saying that there're some flaws in the Swagger branch. The Documenting RESTful APIs with Swagger.
But it's quite helpful actually since it really makes me dive deep into the code and read everything three, four times.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.