Code Monkey home page Code Monkey logo

openshift-microservices's Introduction

THIS REPO IS NOW PART OF A HANDS ON WORKSHOP RUN BY RED HAT

Active development here has stopped, find maintenance updates happening here

  • Working services: app-ui, boards, context-scraper, profile service, and Auth via SSO
  • Working platform components integrated: Istio

Latest tested OpenShift version: OpenShift Version

Microservices

Microservices, also known as the microservice architecture, is a software development technique that structures an application as a collection of loosely coupled services. Microservice architectures enable the continuous delivery/deployment/scaling of complex applications.

This git repo showcases an app built using the microservice architecture with several intentionally simple components. The goal is to showcase an example way to develop and manage microservices using a container platform. However, this example is not meant to be prescriptive - obviously your team and business goals will drive your specific architecture and environment. The technology should translate and hopefully you will find this repo helpful.

Why microservices?

Agility. Deliver application updates faster. Isolate and fix bugs easier. Done right, a microservices architecture will you help to meet several important non-functional requirements for your software:

  • scalability
  • performance
  • reliability
  • resiliency
  • extensibility
  • availability

This Repo

This repo is an example microservices based application. It's core functionality is a paste board. It's an intentionally simple example that could be the prototype for something bigger like a Pinterest, a PasteBin, a CodePen, or even a Ranker.

Current screenshot

Screenshot

Here's the initial design for the architecture:

Diagram In the above diagram web app users are accessing the APP UI service which in turns calls chains of microservices on the backend. Optionally the backend services have calls to API services managed via 3scale (and future access to the services from mobile apps could go through the 3scale API management capability as well). A single sign on capability provides security around user access to the application via OpenID Connect (OIDC) and OAuth2. The Istio service mesh is shown too - it provides core capabilities for traffic management and security of the services as well as detailed observation into the application's operational status. All of this is running on top of an OpenShift cluster. (Additional service interactions and deployment details are in other diagrams).

Diagram The above diagram shows how the services are related and additionally how they are abstracted from the underlying infrastructure (compute and storage) when deployed on top of an OpenShift cluster. (The abstraction means this can be run in AWS, GCP, Azure, on-prem, or in some hybrid combination).

ℹī¸ This example is based on OpenShift Container Platform version 4.3.

How to run this?

First off, you need access to an OpenShift cluster. Don't have an OpenShift cluster? Download Code Ready Containers (CRC) for free here: https://developers.redhat.com/products/codeready-containers/overview. You will need a fairly beefy machine to run everything on the same machine, so I'd recommend you expose extra CPUs and Mem to CRC when you start your cluster - I actually haven't tried it so YMMV.

You will also need to install/configure the additional dependencies you plan to leverage. We assume you want to run these in the cluster, so you might need to tweak the install scripts to move things outside or to use already existing shared services in different namespaces.

  • Service Mesh (Istio)
  • 3Scale - FUTURE
  • Kafka - FUTURE
  • Caching - FUTURE

To install everything:

Once you have the basic app up and running, how about trying out some demos

About the code / software architecture

The parts in action here are:

  • A set of microservices that together provide full application capability for a cut and paste board (in code folder)
  • Key platform components that enable this example:
    • container building via s2i
    • service load balancing
    • service autoscaling
    • service health checks and recovery
    • dynamic storage allocation and persistent volume mapping
    • Kubernetes operators to manage middleware components (e.g. Kafka)
    • advanced service traffic management via Istio
    • additional service observability via Istio
  • Middleware components in this example:
    • API management and metrics (on the external facing APIs)
    • authorization and application security via SSO
    • Kafka for scalable messaging

This demo is now part of a hands-on workshop from Red Hat!

Click here to check it out

References, useful links, good videos to check out

Microservices

Istio Service Mesh

Single Sign On

Contributing

See guidelines here. But in general I would love for help with 2 things.

  1. Use this repo and report issues, even fix them if you can.
  2. Have discussions on topics related to this repo. Things like "Securing services", or "Microservices vs SOA"...
    • currently doing this in issues (and tagging them as a discussion).

License

Apache 2.0.

openshift-microservices's People

Contributors

dudash avatar gbengataylor avatar theckang avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar

openshift-microservices's Issues

Why Kafka? And how would that work on OpenShift?

A question I've gotten informally when discussing this with colleague was "Why use Kafka and how does that run on OpenShift/Kubernetes?"

Specifically, the "Why Kafka" was targeted at other options for microservices to communicate with each other (like REST, gRPC, etc.). This is probably the more important question of the two for this repo.

The "How does it work in K8s/OpenShift?" was looking to understand how a multi-broker with zookeeper would work in a managed container platform like OpenShift.

So I'm starting that discussion here.

Task: Build/Push Images to a Container Registry

Write a script to build/push container images into quay.io or dockerhub.

Update deploy installation docs to cover deployment using tagged images. Option 2 is to build code and deploy via s2i from git tagged source code.

CI/CD automatic build of services on master branch

Is your feature request related to a problem? Please describe.
No

Describe the solution you'd like
I'd like service updates to trigger building their containers and pushing that to quay. Potentially let quay build the containers for me. Or maybe via GitHub Actions. This should happen on the master branch only.

Describe alternatives you've considered
CircleCI, TravisCI

Additional context
Currently keeping containers in the quay registry here:
https://quay.io/user/jasonredhat

New feature - popular items (for shared board)

Add something like Claps or up/down voting.
This could be a new microservice and updates to the app-ui service

From demo POC - Can also use this as a way to showcase CI/CD, versioning and updates in MSA

Delete boards and items

Is your feature request related to a problem? Please describe.
I can't delete anything

Describe the solution you'd like
Delete buttons for boards and items you created

Describe alternatives you've considered
None

Additional context
None

Search Everything

Is your feature request related to a problem? Please describe.
It'd be nice to be able to search through users, boards, items, comments, etc...

Describe the solution you'd like
Something akin to what you see in github

Describe alternatives you've considered
NA

Additional context
Elasticsearch is a strong contender

Watch a board

Is your feature request related to a problem? Please describe.
no - just a new idea

Describe the solution you'd like
As a users I want to watch changes to shared or public boards - I'd like to be notified when things change

Describe alternatives you've considered
None

Additional context
advanced watch might allow me to specify "watch all changes" or "just new items"

Make Istio Required?

Discussion Topic / Overview
Let's considering making Istio required for deploying this example.

Kick it off
Initially, I wanted to have the service mesh be an optional platform capability so folks could choose to deploy with or without it. Now I'm thinking that being able to assume the capability of Istio is available would make it easier to implement and run this app. Opening it up for discussion.

3scale - automate creation of management artifacts for external APIs

The external APIs for the microservices will be managed via 3scale. It would be helpful to automate the creation of the 3scale management artifacts for the external APIs. 3scale includes a toolbox, https://github.com/3scale/3scale_toolbox, to manage services via the command line.

  1. Manage via OpenAPI spec
    Using Apicurio, the external APIs can be created and pushed to the git repo.

  2. Manage via a csv

The toolbox can generate the artifacts from the input source. Need to investigate which of these is more manageable and also generates more of the artifacts

Another alternative would be to write a script/playbook that uses the 3scale Management REST API to create all the management artifacts.

Debating whether unique clients are needed for each microservice within an SSO realm

A security realm can be established in Keycloak and it is possible to setup multiple clients within that realm. For example, we could create a realm for every frontend and backend app in the microservices demo .This documents conversations about how to go about creating clients

cutting and pasting discussion with Mike Echevarria

[08:33, 5/29/2019] Gbenga: Good morning Mike. When you get a chance I'd like to get your opinion about setting clients within a realm in keycloak. Imagine you have a microservices app with multiple backend apps and one or two front-end apps. How would you recommend setting up the clients within the sso realm. Would you have a client for every backend app, or just one for all the backend using a bearer token. Would you have a client for each frontend? I'm leaning towards one client for all backend apps and one for each frontend but wanted a sanity check.. thanks
[08:38, 5/29/2019] Mike Echevarria: Ideally you have a client for each service. That way you can more easily audit activities (see which sessions are active)
[08:39, 5/29/2019] Mike Echevarria: You can also revoke client access without breaking all the other microservices
[08:39, 5/29/2019] Mike Echevarria: That said, in my ocp-sso project I use one client for all the backends, lol. Because yeah, it's easier
[08:40, 5/29/2019] Gbenga: haha
[08:41, 5/29/2019] Mike Echevarria: I keep a separate client for frontends because those are usually public/confidential and have redirects. So it's actually more work to try and maintain all those possibilities instead of just separate clients
[08:42, 5/29/2019] Gbenga: so in terms of the flow, if i have an client for every microservice then the front end has to retrieve a separate token for each one...and if a backend needs to call another backend, it can't propagate the token it also needs to generate a new one? is that correct?
[08:46, 5/29/2019] Mike Echevarria: The clients themselves refresh tokens. It doesn't need to go all the way back up
[08:47, 5/29/2019] Mike Echevarria: Backends can move tokens along, that's no issue
[08:49, 5/29/2019] Mike Echevarria: If a backend needs to change to a different account, then that's where you start getting into service accounts. Or you have public methods that call a microservice that needs to get authenticated stuff
[09:01, 5/29/2019] Gbenga: thanks for the info

for now will have one client for all the backend microservices. Will reconsider as we expand on this demo

Should we enhance user profile service without SSO?

Discussion Topic / Overview
Discuss whether to enhance user profile service with or without SSO given the time and effort we have before an Istio workshop.

Kick it off
In the workshop, we would like the audience to deploy multiple versions of the user profile service to test traffic routing and other Istio features. We could update the app-ui to use the user profile service as-is, but it would mean using a "fake" user in the flow (currently "anonymous").

We could instead try to get SSO working in the app, but it would block this effort until SSO works.

User private boards

Is your feature request related to a problem? Please describe.
No

Describe the solution you'd like
As a user I'd like to be able to create board that only I can see (when logged in)

Describe alternatives you've considered
Related to #41 which limits users to 1 private board unless they have a PRO plan (API Mgmt)

Additional context
None

Write up: About the API Mgmt + API Gateway Layer

Write up a MD on what features the API gateway provides and why it's important in a microservices architecture. And

e.g.

  • A place to rate limit ingress (throttling)
  • A place to secure external facing published APIs and service access points (OpenIDConnect)
  • A place to do collection of metrics on the ingress of the application
  • Dynamic capability for non-coders to manage APIs

SSO Roles to allow more than 1 private board (i.e. PRO plan)

Is your feature request related to a problem? Please describe.
No

Describe the solution you'd like
Demo that showcases the ability to create multiple private boards only if you are in a certain SSO role/group. Default users would only get 1 private board.

Additional context
Also, maybe write up a something in the demos section to show how it works and showcase the components/code involved for others to learn from.

refactor the installation process for 4.x

Describe the bug
Now that OLM is in 4.x we need to re-write all the installation scripts and purge old junk and leverage newer easier operator based installations.

To Reproduce
NA

Expected behavior
Simplified installation which makes it as easy as possible to get this up and running - simple in both steps and in docs.

Screenshots
NA

Additional context
This will be in the deployment/install/* folders

Thoughts on CQRS vs CRUD for these services

I want to document decisions made in the architecture and provide a spot to debate - thus this issue.

CRUD was a simple v1 choice because it's easy to implement but as we grow some services might be better served by a CQRS model. I'm hesitant initially because of the added complexity. There are no plans to add eventing (KNative) into this demo repo.

Microservices.io Pattern ref
Martin Fowler link

History service

Is your feature request related to a problem? Please describe.
No

Describe the solution you'd like
History of viewed boards

Describe alternatives you've considered
None

Additional context
New service + UI updates

Default port usage

Discussion Topic / Overview
Default ports must be avoided

Kick it off
Greetings,

We are security researchers and we are looking for insecure coding patterns and configurations in the microservice architecture repositories. In your repository, we have found instances of default port usage. According to a recent report default port usages must be avoided: https://www.bleepingcomputer.com/news/security/most-cyber-attacks-focus-on-just-three-tcp-ports/#:~:text=According%20to%20the%20report%2C%20the,(Hypertext%20Transfer%20Protocol%20Secure)

Source: https://github.com/dudash/openshift-microservices/blob/master/deployment/install/microservices/openshift-configuration/auth-sso73-x509.yaml

Suggested fix: #64

I am interested to know if you agree with the findings. Any feedback is appreciated.

Write up: Dealing with Partial and Cascading Failures

Write up on how to use Istio to mitigate cascading failures

  • circuit breaker - outlierDetection
  • bulkheading - connectionPool

Also, mention service health and load balancing via K8s.

Can DRAFT some thoughts in here. Eventually move to it's own docs/XXXX.md

Envoy fails to start properly if Deployment Config / Services names and labels do not match

Describe the bug

If the DeploymentConfig name or labels do not match the corresponding Service name or labels, Envoy proxy will fail to start. This is ok for most services but when we're iterating on an app for a workshop (e.g. user profile), we create additional deployments (v2, v3) and the names might not match. Envoy fails to start for those deployments.

It can also be a problem if a different template is used from the service, because a template will inject labels into the API object.

Expected behavior
We should be able to create multiple DeploymentConfigs on an app where the "names" don't have to match the Service. We should further be able to use a template to generate those deployment configs and have it launch with envoy starting properly. The current workaround is to use the apps/Deployment object without templates.

Additional context
This is not a problem with the app per se, but logging this here for posterity. Will log into appropriate repo (probably OKD) and link it here.

Related: istio/istio#11979

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤ī¸ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.