Code Monkey home page Code Monkey logo

smilr's Introduction

Smilr - Microservices Reference Project

Smilr is a multi component application & reference architecure. It has been designed to showcase microservices design patterns & deployment architectures. It consists of a front end single page application (SPA), two lightweight services, supporting database and back end data enrichment functions.

The Smilr app is simple, it allows users to provide feedback on events or sessions they have attended via a simple web & mobile interface. The feedback consists of a rating (scored 1-5) and supporting comments.

  • The user interface is written in Vue.js and is completely de-coupled from the back end, which it communicates with via REST. The UI is fully responsive and will work on on both web and mobile.
  • The two microservices are both written in Node.js using the Express framework. These have been containerized so can easily be deployed & run as containers
  • The database is a NoSQL document store holding JSON, provided by MongoDB and/or Azure Cosmos DB

The app has been designed to be deployed to Azure, but the flexible nature of the design & chosen technology stack results in a wide range of deployment options and compute scenarios, including:

  • Containers: Azure Kubernetes Service (AKS) or Azure Container Instances
  • Platform services: Regular Windows Azure App Service (Web Apps) or Linux Web App for Containers
  • Serverless compute: Azure Functions
  • Virtual Machines: Sub-optimal but theoretically possible

This application supports a range of demonstration, and learning scenarios, such as:

  • A working example of microservices design
  • Use of containers, Docker & Kubernetes
  • NoSQL and document stores over traditional relational databases
  • Development and deployment challenges of single page applications
  • RESTful API design
  • Platform services for application hosting
  • Using serverless technology to support or provide services
  • DevOps practices, automated CI build, test & release pipelines
  • Use of an open source application stack such as Node.js
  • The Actor model as an alternative to a traditional data model
  • CQRS (Command & Query Responsibility Segregation) as a possible pattern to separate read and write actions and stores

Table of Contents


Architecture & Core App Components

arch{: .framed .padded}

Logical Components

The app consists of four logical components

  • Client Vue.js SPA - The primary web user interface, as a single page application (SPA)
  • Frontend Service - Serves the above client SPA and acts as runtime config API
  • Data API Service - Main REST API used by the app for all data access
  • Database - Backend MongoDB database & persistent store
  • Sentiment Service - Optional service for sentiment scoring of user feedback comments. Only used when deploying to Kubernetes. Read more

Screenshot

screenshot{: .framed .padded}

Getting Started

There are a number of ways to get started with this project depending on your goals. Below are some suggestions and pointers

  • Running Smilr locally with Node.js - If you're a developer interested in the building and running Node.js and/or Vue.js components locally, this is a good place to start.
  • Deploying to Azure from pre-built images - Probably the quickest way to deploy Smilr is using ARM templates & Azure Container Instances running from pre-built images
  • Deploying to Kubernetes - This project is an ideal candidate for running in Kubernetes. Step by step documentation is provided to walk through deployment to Kubernetes and Azure Kubernetes Service

Repository Structure

This is a monorepo so contains multiple discreet but loosely dependant projects. The top levels of the repository directory tree are laid out as follows

/
├── archive            Older and experimental code archive 
│   ├── mobile             Nativescript + Vue.js mobile client 
│   └── go                 Work in progress re-write of the main services in Go
├── azure              Supporting files for Azure deployment etc
│   ├── functionsv2      Azure Functions serverless implementation
│   ├── pipelines        Azure Pipelines for CI/CD
│   └── templates        Example ARM templates
├── docs               Documentation
├── docker             Docker compose files, Note. Dockerfiles are elsewhere 
├── etc                Supporting files, pictures and other artefacts 
├── kubernetes         Docs and files to support deployment to Kubernetes & AKS
│   ├── helm             Helm chart for deploying Smilr with Helm
│   ├── advanced         Deployment YAML for use with Ingress, SSL and persistence
│   └── basic            Simple deployment without Ingress or persistence
├── node               Main microservices, written in Node.js
│   ├── data-api         Data API service source code
│   └── frontend         Frontend service source code
├── orleans            Orleans actor based implementation of the Data API
├── testing            Mock test data, API tests and load test scripts
└── vue                The main app frontend, a Vue.js SPA
    ├── src              Source code of Vue.js app
    └── mock-api         Provides a fake API and database for local testing

Reference Implementation

The reference implementation of the Smilr app consists of the frontend server & data-api written in Node.js and the client as a SPA written in Vue.js. There are alternative versions (detailed below) but currently the core of the project is represented this implementation:

Notes on Security

As Smilr is commonly used in demos, workshops/labs and for self learning, it defaults to being unsecured and open. This makes it easier to deploy without additional configuration or authentication complications.

Should you want to secure Smilr, typically because you want to have a permanent "live" version hosted and accessible, this is also possible. Securing both single page applications and REST APIs presents some interesting challenges. The approach taken was to use Azure Active Directory v2 for identity, the OAuth2 Implicit Grant flow for authentication and OAuth2 bearer tokens for validation

Fundamentals

These docs covers some of the fundamental aspects of Smilr, which are independent of any particular implementation

Runtime Configuration & Settings

The primary configuration mechanism for this project is system environmental variables (or env vars). These env vars are used with the Node.js services, container runtime, Docker tools and helper scripts. This gives a flexible and cross platform way to inject runtime settings, it is also widely supported in Azure (e.g. Azure App Service Settings) and Docker/Kubernetes.

There are numerous ways to set & override environmental variables; in the operating system, the user profile or from command line tools. For local development purposes it is strongly recommended you create & use .env files. These are simple text files containing var=value pairs. Sample files named .env.sample are provided within the project, which you can rename and use. Note. .env files can often contain secrets so they are prevented from being committed to Git

Containers & Kubernetes

As containers and Kubernetes represents such an important build & deployment scenario, they have dedicated guides and documentation

Public Images

Pre built images are available on Dockerhub. Using these will naturally mean you can get started deploying/running Smilr without needing to do anything. However building your own is still recommended, especially if you want to understand the build process in more detail.

Images for the Node.js data API and frontend (serving the Vue.js client) are available.

  • The stable tag is only pushed after some manual validation and testing,
  • latest is auto-pushed with latest code every night

Deploying to Azure

See provided Azure Resource Manager (ARM) templates to allow you to deploy the complete Smilr app and all components to Azure. As all deployment scenarios are container based you also need to refer to the containers section described above

DevOps & CI/CD

Automated CI/CD pipelines have been created using Azure Pipelines. These automatically build the various components as containers and releases them to Azure for testing. The current status of these builds & releases is shown below

Automated Build Status
Linux Images & Tests Build Status
Windows Images & Dockerhub Build Status

You can visit the Azure Devops Public Project where these pipelines reside, although keep in mind the source code repo remains here on GitHub.

If you want to try using Azure DevOps to build Smilr, the pipelines are provided as YAML for your own use. An Azure DevOps account/organisation is required to use these, but can be setup for free.

Sub Projects

There are several sub-projects and re-implementations of the Smilr architecture. Either at the backend, providing an API compatible with the data-api REST specification. Also the frontend has been implemented as a mobile app. All of these alternatives are considered experimental and for technology demo use cases

Changelog

High level project changes and overall history are recorded here:

  • Jan 2020 - Rewrote Node.js data API, using MVCS pattern
  • Jul 2019 - Start of Go implementation. Archived old sub-projects
  • Jun 2019 - Security improvements, MSAL and token validation
  • Dec 2018 - Total rewrite of SPA in Vue.js and proper AAD security
  • May 2018 - Major updates for Kubernetes deployment & Angular 6
  • Mar 2018 - Switched to MongoDB for database
  • Jan 2018 - Simple onetime password security added to API
  • Dec 2017 - Name changed to Smilr
  • Nov 2017 - Switched to CosmosDB SQL API (aka DocumentDB)
  • Oct 2017 - Project created, using Azure Table storage

smilr's People

Contributors

arjendev avatar benc-uk avatar davidgristwood avatar dependabot[bot] avatar ross-p-smith avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

smilr's Issues

Failing

It is failing while pushing it to ACR at
Step 8/13 : RUN apk update && apk add bash
---> Running in 32a3d839e951
/bin/sh: 1: apk: not found

It worked fine until yesterday, something changed today.

Need data model documentation

by that I mean fully document the data that flows in and out the middle tier (topics, etc) that are totally abstracted from any underlying store (e.g. plug in azure table storage, reddis, cosmos, etc)

Orleans

Implement the API using Orleans actor model, fun project!

Refactor mobile sidebar as a component

The side bar used when in mobile view is a DOM styling kludge using getElementById

This really should be a child nested Angular component under the AppComponent

Error while running command on cloud shell

I get the following error when I run this command from module 2 of the AKS lab

az acr build --registry $ACR_NAME -g kube-lab --context . --file node/data-api/Dockerfile --image smilr/data-api

No module named 'pytz'
Traceback (most recent call last):
File "/opt/az/lib/python3.6/site-packages/knack/cli.py", line 197, in invoke
cmd_result = self.invocation.execute(args)
File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/init.py", line 251, in execute
self.commands_loader.load_arguments(command)
File "/opt/az/lib/python3.6/site-packages/azure/cli/core/init.py", line 244, in load_arguments
self.command_table[command].load_arguments() # this loads the arguments via reflection
File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/init.py", line 140, in load_arguments
super(AzCliCommand, self).load_arguments()
File "/opt/az/lib/python3.6/site-packages/knack/commands.py", line 76, in load_arguments
cmd_args = self.arguments_loader()
File "/opt/az/lib/python3.6/site-packages/azure/cli/core/init.py", line 425, in default_arguments_loader
op = handler or self.get_op_handler(operation)
File "/opt/az/lib/python3.6/site-packages/azure/cli/core/init.py", line 469, in get_op_handler
op = import_module(mod_to_import)
File "/opt/az/lib/python3.6/importlib/init.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 978, in _gcd_import
File "", line 961, in _find_and_load
File "", line 950, in _find_and_load_unlocked
File "", line 655, in _load_unlocked
File "", line 678, in exec_module
File "", line 205, in _call_with_frames_removed
File "/home/vishesh/.azure/cliextensions/acrbuildext/azext_acrbuildext/build.py", line 18, in
import pytz
ModuleNotFoundError: No module named 'pytz'

The python prompt gives version 3.5 and I have pip installed pytz
Please let me know where I am going wrong and what to try

Error visibility in UI/Frontend

When unable to contact the REST API or get results the error is only logged to the browser console, should (nicely) make this situation visible in the UI

Dapr?

Dapr-ize this app?
It's not really microservices

So many embedded styles

Sort the CSS out.
Move embedded styles to CSS files and work out what is happening with global vs component styles!

events and topics

As disucssed, can we have an "event" being the over arching aitivity that is unique and exits for a period time, and which is only open for feedback with that duration (to stop data colected being updated after or before the event). It then has zero or more "topic" (?) that data is collected against (happy, unhappy, etc)

.NET Core Re-implementation

Re-implement the two services in .NET Core using ASP.NET Core MVC, run in Linux containers much the same as the existing Node.js ones

Need to fully separate tier projects

I think we need to fully separate the different tier projects and their folders and git projects, so its possible to choose a UI and middle tier service in any arbitrary fashion eg swap out node api for service fabric api. I feel there seems to much "leakage" between the different projects

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.