Code Monkey home page Code Monkey logo

distributed-system-using-mqtt's Introduction

NestJS Microservices with RabbitMQ / Grafana / Helm Charts

Fully managed Microservices starter pack using NestJS / RabbitMQ. created with SQL, NoSQL databases, Kong API Gateway, Grafana Logging stack and Helm charts.

For Linux, I would recommend this tool for docker container inspection: https://github.com/jesseduffield/lazydocker

Dependencies & Services

Get started Notes:

  • Use git submodule update --init --recursive command to update/fetch submodules.
  • Use .env.local file while working on local environment and use .env for production.
  • Make sure to change service name to localhost in .env.local while running single service only in terminal.
  • kong.yml from kong/conf/kong.yml file is configured for api gateway.
  • Kong development server endpoint will start on port 8000.
  • Health endpoint: host:port/api/health
  • Swagger docs endpoint: host:port/api/docs

Run in local

start core services first (postgres, rabbitmq, mongodb, redis)

npm run up 

Now, first place .env file in service folder then go to service folder cd auth

npm run dev

to stop core services, run

npm run down

Run in local with docker-compose

start core services

npm run up

docker-compose command - this will take env variables from .env.local

docker-compose up "service name" 

rebuild

docker-compose up "service name" --build

Run all services in Local Env using docker-compose

docker-compose up 

Setup Grafana Dashboard in Local Env

To see the logs on Grafana dashboard, follow below steps.

  1. Open the browser and go to http://localhost:3000, use default credentials username "admin" and password "admin" to login.
  2. Now, go to http://localhost:3000/datasources and select Loki from Logging and document databases section.
  3. Enter http://loki:3100 in URL under HTTP section. We can do this because we are running Loki and Grafana in the same network.
  4. loki else you have to enter host IP address and port here, click on Save and Test button from the bottom of the page.
  5. Now, go to 3rd tab "Explore" from the left sidebar or http://localhost:3000/explore
  6. select containers and run the query. you will see below view on your screen. image

Deployment in ECR

sh deploy.sh

Notes:

  • change "aws_region" and "aws_account_id" as per your AWS account.

Build in Local

sh build.sh

K8s Deployment with Helm charts:

NOTE: I would recommend tool k9s for better understanding of a cluster. To start with helm deployment in local machine, make sure that you have started Minikube. service images should be available in your local docker env and core service should be up and running in local env.

If you're unable to use local images in minikube cluster then refer this blog

minikube start

Now, install helm charts

helm install nestjs-ms helm 

To get services endpoints

minikube service list
# check loki stack installed in current namespace
helm list
# NAME     	NAMESPACE	REVISION	UPDATED                                	STATUS  	CHART           	APP VERSION
# nestjs-ms	default  	1       	2023-02-27 19:11:14.537649934 +0530 IST	deployed	nestjs-ms-0.1.0 	1.0.0  

# now get loki IP from k8s cluster
kubectl get service nestjs-ms-loki
# NAME             TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)    AGE
# nestjs-ms-loki   ClusterIP   10.102.161.158   <none>        3100/TCP   28m

# use this ClusterIP to add datasource Loki in grafana dashboard

# at the end minikube service list will look like this
minikube service list
# |-------------|-----------------------------------|--------------------|---------------------------|
# |  NAMESPACE  |               NAME                |    TARGET PORT     |            URL            |
# |-------------|-----------------------------------|--------------------|---------------------------|
# | default     | auth-service                      | No node port       |                           |
# | default     | files-service                     | No node port       |                           |
# | default     | kubernetes                        | No node port       |                           |
# | default     | nestjs-ms-grafana                 | service/80         | http://192.168.49.2:31679 |
# | default     | nestjs-ms-kong-proxy              | kong-proxy/80      | http://192.168.49.2:32083 |
# |             |                                   | kong-proxy-tls/443 | http://192.168.49.2:31251 |
# | default     | nestjs-ms-kong-validation-webhook | No node port       |                           |
# | default     | nestjs-ms-loki                    | No node port       |                           |
# | default     | nestjs-ms-loki-headless           | No node port       |                           |
# | default     | nestjs-ms-loki-memberlist         | No node port       |                           |
# | default     | notification-service              | No node port       |                           |
# | default     | post-service                      | No node port       |                           |
# | kube-system | kube-dns                          | No node port       |                           |
# |-------------|-----------------------------------|--------------------|---------------------------|
# To access grafana : http://192.168.49.2:30353
# To access kong gateway: http://192.168.49.2:31208

Clean up the local env

helm uninstall nestjs-ms

distributed-system-using-mqtt's People

Contributors

hmake98 avatar harsh-simform avatar alishasimform avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.