azure-samples / serverless-microservices-reference-architecture Goto Github PK

This reference architecture walks you through the decision-making process involved in designing, developing, and delivering a serverless application using a microservices architecture through hands-on instructions for configuring and deploying all of the architecture's components along the way. The goal is to provide practical hands-on experience in working with several Azure services and the technologies that effectively use them in a cohesive and unified way to build a serverless-based microservices architecture.

License: MIT License

C# 70.39% PowerShell 3.58% JavaScript 5.93% HTML 0.48% Vue 13.61% Bicep 6.00%

serverless microservices microservices-architecture serverless-architectures azure-functions azure-app-service azure-devops azure-storage azure-active-directory azure-api-management

serverless-microservices-reference-architecture's Introduction

languages

products

page_type

description

csharp

javascript

nodejs

azure

azure-functions

azure-logic-apps

azure-event-grid

azure-cosmos-db

azure-sql-database

azure-storage

azure-app-service

sample

This architecture walks you through the process involved in developing the Rideshare by Relecloud application.

Serverless Microservices reference architecture

This sample has been upgraded to Azure Functions v3. An upgrade to Azure Functions v4 is in progress. See PR #43 for details.

The reference architecture

This reference architecture walks you through the decision-making process involved in designing, developing, and delivering the Rideshare by Relecloud application (a fictitious company). You will be given hands-on instructions for configuring and deploying all of the architecture's components, with helpful information about each component along the way. The goal is to provide you with practical hands-on experience in working with several integral Azure services and the technologies that effectively use them in a cohesive and unified way to build a serverless-based microservices architecture. We hope you will leave with a newfound appreciation for how serverless can be leveraged for both large-scale and small solutions, and the confidence to apply what you learned in your own projects.

Customer scenario

Relecloud is a new kind of company, not unlike many cloud services-focused organizations popping up around the world today. Born of the modern digital age that is focused on capitalizing on the fast-paced growth of the booming cloud industry, Relecloud has a track record of building lean startups with little operational overhead, owing to its distributed and remote workforce. Their latest endeavor is a ride share application that seeks to carve its place in the popular industry with competitive rates and improved communication between passengers and drivers.

The company's technical leadership thrives on keeping tabs on the rapidly growing technology industry and new innovations built on top of fully-managed services provided by cloud providers. They are not interested in maintaining infrastructure, as they feel as though their company's time is best spent on their core strength: rapidly developing new and innovative applications that can scale to meet global demand. Like many other technical leaders, they are keen on learning new buzzwords and leaving it up to their developers to do something useful with them. The hottest trend these days seems to be serverless. The promise of consumption-based pricing, where you only pay for what you use and nothing more, is enticing. Furthermore, they have heard many good things about how serverless platforms help you prototype and develop faster by reducing the amount of code and configuration required by more traditional web platforms. The cherry on top is the fact that serverless platforms also tend to manage scaling resources to meet demand automatically, giving them dreams of releasing a wildly popular new ride share app and enjoying near-instantaneous customer growth.

During their initial research phase consisting of comparing serverless offerings and creating rapid prototypes, Relecloud's team has decided to build their ride share application on Azure's serverless components, given the breadth of options and unique capabilities for orchestrating serverless activities, such as Durable Functions. They also want to investigate using the microservices pattern in their solution design, as it seems like a good fit alongside Azure functions, API Management, Event Grid, and other key components and services. Being able to monitor the solution as a whole is an important capability they want to put in place from the start, especially since they are relying on so many components. Finally, they wish to simplify the lifecycle management of all these pieces of the puzzle by applying DevOps practices to automate continuous integration and deployment, end-to-end.

Explore Relecloud's solution using serverless and microservices

Read about Relecloud's solution and overall architecture design and decisions. The article will briefly explain the concepts around both serverless and microservices, and how they can be used together to build solutions with little to no infrastructure overhead. It will then walk you through the sample solution you will deploy in the lab, broken down into its architectural components.

Deploy Relecloud's solution today with a hands-on lab

After learning about Relecloud's serverless microservices architecture, deploy the companion solution by following the step-by-step hands-on lab, or take the shortcut and deploy with a few clicks using our templates.

Each section of the lab will briefly explain what you are trying to accomplish and why. It will also link you to the relative portion of the architecture document.

Detailed documentation

Use the table of contents below for detailed documentation of each component of the reference architecture.

serverless-microservices-reference-architecture's People

Contributors

Stargazers

Watchers

Forkers

onebluecubedev quinntynebrown cloudmelon mananchoksi007 retrywith daverendon dankyungu matrixdekoder rafsanulhasan onebluecube janahyldvang jitendriyag2 glaucia86 yonet appcoreopc aheneghan carlfranklin rathic pacohams ric-ulctech devopsdemoapps david-yaari schunduea nateduff ericforgy pchenga aemorales bobsourced codestellar dmelgarejopdesgo naveengarla erwingeirnaert mcgriffsoftware alexma1017 ganesh-chavan rafallopatka winsome5557 poppywood mraipsec-mra xpouyat ajmal744 calebjenkins adraffin waiphyooo dileepgudivada josemarcosit continue-nature ruthbacci alexandros-lykourgos msajidirfan raima71 hariguru shaikhanas1993 sobo123 sarthakvijayvergiya lubna-thomas ssijbabu vechkar baluprasathsbp techiechap good-faith liemfrans arvindmits boye0040 momzo swiftarrow01 vinaybhumireddy hkumarigithub ashcherbinin ayushicloud cedricsteenbeke vany0114 llenroc karthikmakam leomedeiros-azure swimlanebuilder shimii leosimoez ajaysingh77 vandzaxe olafkfreund eagleeyethinker anantakjena vpadegaonkar indrajithbalasubramanian niloshima-srivastav emilverwoerd prosenjitd-84 gh-todorov originalobjective danlister aravinda-kumar ckinect dnyandeobharambe jabolt helios1989 sandeepsenguttuvan simplesoftmx gershon-tadepalli bapic

serverless-microservices-reference-architecture's Issues

=> Static Web Apps

docker images used by azure-functions-dotnet-core2.0 are outdated

@khaledhikmat, When I was trying to run dockerfiles it shows error as below on both arm64 and amd64 architectures.

ubuntu@ip-172-31-19-230:~Serverless-microservices-reference-architecture/dotnet/dockerfiles$  sudo docker build -t dotent:drivers -f drivers . 
[+] Building 0.2s (4/4) FINISHED 
=> [internal] load build definition from drivers                                                                                                                                          0.0s 
=> => transferring dockerfile: 195B                                                                                                                                                       0.0s 
=> [internal] load .dockerignore                                                                                                                                                          0.0s 
=> => transferring context: 2B                                                                                                                                                            0.0s 
=> ERROR [internal] load metadata for docker.io/microsoft/azure-functions-dotnet-core2.0:v2.0.11961-alpha                                                                                 0.2s 
=> [auth] microsoft/azure-functions-dotnet-core2.0:pull token for registry-1.docker.io                                                                                                    0.0s 
------ 
> [internal] load metadata for docker.io/microsoft/azure-functions-dotnet-core2.0:v2.0.11961-alpha: 
------ 
drivers:1 
-------------------- 
   1 | >>> FROM microsoft/azure-functions-dotnet-core2.0:v2.0.11961-alpha 
   2 | 
   3 |     COPY ./ServerlessMicroservices.FunctionApp.Drivers/bin/Debug/netstandard2.0 /home/site/wwwroot 
-------------------- 
ERROR: failed to solve: microsoft/azure-functions-dotnet-core2.0:v2.0.11961-alpha: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed

I have tried to check this docker image microsoft/azure-functions-dotnet-core2.0 on docker hub, but it does not show any result for the image.

Kindly share your feedback regarding this. I will be happy to contribute if required.

Upgrade Functions v2 => v3

ARM => Bicep

docker images used by azure-functions-dotnet-core2.0 are outdated

@khaledhikmat, When I was trying to run dockerfiles it shows error as below on both arm64 and amd64 architectures.

ubuntu@ip-172-31-19-230:~Serverless-microservices-reference-architecture/dotnet/dockerfiles$  sudo docker build -t dotent:drivers -f drivers . 
[+] Building 0.2s (4/4) FINISHED 
=> [internal] load build definition from drivers                                                                                                                                          0.0s 
=> => transferring dockerfile: 195B                                                                                                                                                       0.0s 
=> [internal] load .dockerignore                                                                                                                                                          0.0s 
=> => transferring context: 2B                                                                                                                                                            0.0s 
=> ERROR [internal] load metadata for docker.io/microsoft/azure-functions-dotnet-core2.0:v2.0.11961-alpha                                                                                 0.2s 
=> [auth] microsoft/azure-functions-dotnet-core2.0:pull token for registry-1.docker.io                                                                                                    0.0s 
------ 
> [internal] load metadata for docker.io/microsoft/azure-functions-dotnet-core2.0:v2.0.11961-alpha: 
------ 
drivers:1 
-------------------- 
   1 | >>> FROM microsoft/azure-functions-dotnet-core2.0:v2.0.11961-alpha 
   2 | 
   3 |     COPY ./ServerlessMicroservices.FunctionApp.Drivers/bin/Debug/netstandard2.0 /home/site/wwwroot 
-------------------- 
ERROR: failed to solve: microsoft/azure-functions-dotnet-core2.0:v2.0.11961-alpha: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed

I have tried to check this docker image microsoft/azure-functions-dotnet-core2.0 on docker hub, but it does not show any result for the image.

Kindly share your feedback regarding this. I will be happy to contribute if required.

docker images used by azure-functions-dotnet-core2.0 are outdated

@khaledhikmat, When I was trying to run dockerfiles it shows error as below on both arm64 and amd64 architectures.

ubuntu@ip-172-31-19-230:~Serverless-microservices-reference-architecture/dotnet/dockerfiles$  sudo docker build -t dotent:drivers -f drivers . 
[+] Building 0.2s (4/4) FINISHED 
=> [internal] load build definition from drivers                                                                                                                                          0.0s 
=> => transferring dockerfile: 195B                                                                                                                                                       0.0s 
=> [internal] load .dockerignore                                                                                                                                                          0.0s 
=> => transferring context: 2B                                                                                                                                                            0.0s 
=> ERROR [internal] load metadata for docker.io/microsoft/azure-functions-dotnet-core2.0:v2.0.11961-alpha                                                                                 0.2s 
=> [auth] microsoft/azure-functions-dotnet-core2.0:pull token for registry-1.docker.io                                                                                                    0.0s 
------ 
> [internal] load metadata for docker.io/microsoft/azure-functions-dotnet-core2.0:v2.0.11961-alpha: 
------ 
drivers:1 
-------------------- 
   1 | >>> FROM microsoft/azure-functions-dotnet-core2.0:v2.0.11961-alpha 
   2 | 
   3 |     COPY ./ServerlessMicroservices.FunctionApp.Drivers/bin/Debug/netstandard2.0 /home/site/wwwroot 
-------------------- 
ERROR: failed to solve: microsoft/azure-functions-dotnet-core2.0:v2.0.11961-alpha: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed

I have tried to check this docker image microsoft/azure-functions-dotnet-core2.0 on docker hub, but it does not show any result for the image.

Kindly share your feedback regarding this. I will be happy to contribute if required.

Build & integration test Actions

Let's create simple tests that assert build and deployment artefacts aren't broken by our refactoring

Upgrade Functions v3 => v4

Script to Publish to Azure and smoke test

Fix build action annotations

When most of the refresh has been done, fix the GH action warning and error annotations and add an audit build to assert that dependencies are still current.

Fix dotnet build warnings
Fix npm install warnings
Fix npm build (pack) warnings
Create audit actions to assert dependencies are still up to date

docker images used by azure-functions-dotnet-core2.0 are outdated

@khaledhikmat, When I was trying to run dockerfiles it shows error as below on both arm64 and amd64 architectures.

ubuntu@ip-172-31-19-230:~Serverless-microservices-reference-architecture/dotnet/dockerfiles$  sudo docker build -t dotent:drivers -f drivers . 
[+] Building 0.2s (4/4) FINISHED 
=> [internal] load build definition from drivers                                                                                                                                          0.0s 
=> => transferring dockerfile: 195B                                                                                                                                                       0.0s 
=> [internal] load .dockerignore                                                                                                                                                          0.0s 
=> => transferring context: 2B                                                                                                                                                            0.0s 
=> ERROR [internal] load metadata for docker.io/microsoft/azure-functions-dotnet-core2.0:v2.0.11961-alpha                                                                                 0.2s 
=> [auth] microsoft/azure-functions-dotnet-core2.0:pull token for registry-1.docker.io                                                                                                    0.0s 
------ 
> [internal] load metadata for docker.io/microsoft/azure-functions-dotnet-core2.0:v2.0.11961-alpha: 
------ 
drivers:1 
-------------------- 
   1 | >>> FROM microsoft/azure-functions-dotnet-core2.0:v2.0.11961-alpha 
   2 | 
   3 |     COPY ./ServerlessMicroservices.FunctionApp.Drivers/bin/Debug/netstandard2.0 /home/site/wwwroot 
-------------------- 
ERROR: failed to solve: microsoft/azure-functions-dotnet-core2.0:v2.0.11961-alpha: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed

I have tried to check this docker image microsoft/azure-functions-dotnet-core2.0 on docker hub, but it does not show any result for the image.

Kindly share your feedback regarding this. I will be happy to contribute if required.

docker images used by azure-functions-dotnet-core2.0 are outdated

@khaledhikmat, When I was trying to run dockerfiles it shows error as below on both arm64 and amd64 architectures.

ubuntu@ip-172-31-19-230:~Serverless-microservices-reference-architecture/dotnet/dockerfiles$  sudo docker build -t dotent:drivers -f drivers . 
[+] Building 0.2s (4/4) FINISHED 
=> [internal] load build definition from drivers                                                                                                                                          0.0s 
=> => transferring dockerfile: 195B                                                                                                                                                       0.0s 
=> [internal] load .dockerignore                                                                                                                                                          0.0s 
=> => transferring context: 2B                                                                                                                                                            0.0s 
=> ERROR [internal] load metadata for docker.io/microsoft/azure-functions-dotnet-core2.0:v2.0.11961-alpha                                                                                 0.2s 
=> [auth] microsoft/azure-functions-dotnet-core2.0:pull token for registry-1.docker.io                                                                                                    0.0s 
------ 
> [internal] load metadata for docker.io/microsoft/azure-functions-dotnet-core2.0:v2.0.11961-alpha: 
------ 
drivers:1 
-------------------- 
   1 | >>> FROM microsoft/azure-functions-dotnet-core2.0:v2.0.11961-alpha 
   2 | 
   3 |     COPY ./ServerlessMicroservices.FunctionApp.Drivers/bin/Debug/netstandard2.0 /home/site/wwwroot 
-------------------- 
ERROR: failed to solve: microsoft/azure-functions-dotnet-core2.0:v2.0.11961-alpha: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed

I have tried to check this docker image microsoft/azure-functions-dotnet-core2.0 on docker hub, but it does not show any result for the image.

Kindly share your feedback regarding this. I will be happy to contribute if required.

v4 Update

This issue is for a: (mark with an `x`)

- [ ] bug report -> please search issues before submitting
- [ ] feature request
- [ ] documentation issue or request
- [ X ] regression (a behavior that used to work and stopped in a new release)

Mention any other details that might be useful

The node example is using v3. It would be nice to have a v4 implementation

Can´t find the function EVGH_TripExternalizations2CosmosDB

In the documentation we must attach an eventgrid topic to this function, but it doesn´t exist.

In other things, how do I test this solution?

Please provide us with the following information:

This issue is for a: (mark with an `x`)

- [ ] bug report -> please search issues before submitting
- [ ] feature request
- [ ] documentation issue or request
- [ ] regression (a behavior that used to work and stopped in a new release)

Minimal steps to reproduce

Any log messages given by the failure

Expected/desired behavior

OS and Version?

Windows 7, 8 or 10. Linux (which distribution). macOS (Yosemite? El Capitan? Sierra?)

Versions

Mention any other details that might be useful

Thanks! We'll be in touch soon.

Bring reference architecture up to date

This issue is for a: (mark with an `x`)

- [ ] bug report -> please search issues before submitting
- [X] feature request
- [X] documentation issue or request
- [ ] regression (a behavior that used to work and stopped in a new release)

Minimal steps to reproduce

This reference architecture is really interesting but as with the ever increase in cadence of functionality in Azure it has fallen behind in services, SDK versions, best practices etc. and not been updated for 2 years. It would be really helpful if this reference architecture was updated to work with the latest functionality in Azure Functions etc. and best practices as required.

Thanks!

Bicep: Add app settings to Functions and Static web app

@wviriya please populate Key Vault secrets and set App Settings in Functions and Static web app at deployment time. The App settings for each Function are documented here.

Note: Only Cosmos key, connection string and AAD Client Secret need to be stored in Key Vault IMO. The rest of the settings can be ordinary App Settings, as they are not secrets. Application Insights iKey is also not a secret.

Create Cosmos Key, Connection string secret in Key Vault
Create Graph Client Secret in Key Vault
Set Key Vault references in App Settings for all Functions and Static web app
Set other settings in App Settings for all Functions and Static web app

Ensure Idempotency in A_TO_RecycleTripDriver Function

Description:
The A_TO_RecycleTripDriver function in the TripMonitorOrchestrator does not currently handle idempotency. This can lead to processing the same event multiple times, which may cause unwanted side effects, such as incorrect driver status updates. To address this issue, we need to store a unique identifier for each trip's recycling event in a persistent storage and ensure that each event is processed only once.

Steps to reproduce:

Run the TripMonitorOrchestrator.
Simulate an error exit after the ServiceFactory.GetLoggerService().Log api but before the function completes
Trigger the A_TO_RecycleTripDriver function through by invoking the orchestrator. Let the first run fail.
Let the function retry automatically.

Expected behavior:
The A_TO_RecycleTripDriver function should process the trip recycling event only once and skip any subsequent calls with the same trip.

Actual behavior:
The A_TO_RecycleTripDriver function will invoke the ServiceFactory.GetLoggerService().Log api more than once on error exit. Thus, there will be duplicated messages.

Proposed Solution:
We should have the front end deduplicate the recycling event for the trip has already been processed. This can be done by maintaining the last processed id in some queryable storage or memory. If the event id matches the last processed id. skip the processing to maintain idempotency.

Thank you for your contribution to the Github community and I really appreciate your effort in following through this issue.

Merge dependabots

Upgrade Logic App to Standard and add to Bicep and documentation

Consider:

moving the Logic App to Logic App Standard
automating the resource and logic app creation
updating the deployment documentation where Logic Apps is referenced

StorageService improvements

Upgrade Storage Nugets
Not thread safe
Swallowing exceptions
trip monitors queue not used

Error: error:0308010C:digital envelope routines::unsupported

In Node.js v17, the Node.js developers closed a security hole in the SSL provider. This fix was a breaking change that corresponded with similar breaking changes in the SSL packages in NPM. When you attempt to use SSL in Node.js v17 or later without also upgrading those SSL packages in your package.json, then you will see this error.

https://stackoverflow.com/a/73027407/610731

Error: error:0308010C:digital envelope routines::unsupported
    at new Hash (node:internal/crypto/hash:71:19)
    at Object.createHash (node:crypto:133:10)
    at module.exports (C:\Users\dalars\r\odidev\Serverless-microservices-reference-architecture\web\serverless-microservices-web\node_modules\webpack\lib\util\createHash.js:135:53)
    at NormalModule._initBuildHash (C:\Users\dalars\r\odidev\Serverless-microservices-reference-architecture\web\serverless-microservices-web\node_modules\webpack\lib\NormalModule.js:417:16)
    at handleParseError (C:\Users\dalars\r\odidev\Serverless-microservices-reference-architecture\web\serverless-microservices-web\node_modules\webpack\lib\NormalModule.js:471:10)
    at C:\Users\dalars\r\odidev\Serverless-microservices-reference-architecture\web\serverless-microservices-web\node_modules\webpack\lib\NormalModule.js:503:5
    at C:\Users\dalars\r\odidev\Serverless-microservices-reference-architecture\web\serverless-microservices-web\node_modules\webpack\lib\NormalModule.js:358:12
    at C:\Users\dalars\r\odidev\Serverless-microservices-reference-architecture\web\serverless-microservices-web\node_modules\loader-runner\lib\LoaderRunner.js:373:3
    at iterateNormalLoaders (C:\Users\dalars\r\odidev\Serverless-microservices-reference-architecture\web\serverless-microservices-web\node_modules\loader-runner\lib\LoaderRunner.js:214:10)
    at Array.<anonymous> (C:\Users\dalars\r\odidev\Serverless-microservices-reference-architecture\web\serverless-microservices-web\node_modules\loader-runner\lib\LoaderRunner.js:205:4)
    at Storage.finished (C:\Users\dalars\r\odidev\Serverless-microservices-reference-architecture\web\serverless-microservices-web\node_modules\enhanced-resolve\lib\CachedInputFileSystem.js:55:16)
    at C:\Users\dalars\r\odidev\Serverless-microservices-reference-architecture\web\serverless-microservices-web\node_modules\enhanced-resolve\lib\CachedInputFileSystem.js:91:9
    at C:\Users\dalars\r\odidev\Serverless-microservices-reference-architecture\web\serverless-microservices-web\node_modules\graceful-fs\graceful-fs.js:78:16
    at FSReqCallback.readFileAfterClose [as oncomplete] (node:internal/fs/read_file_context:68:3) {
  opensslErrorStack: [ 'error:03000086:digital envelope routines::initialization error' ],
  library: 'digital envelope routines',
  reason: 'unsupported',
  code: 'ERR_OSSL_EVP_UNSUPPORTED'
}

Node.js v18.13.0

azure-samples / serverless-microservices-reference-architecture Goto Github PK

serverless-microservices-reference-architecture's Introduction

Serverless Microservices reference architecture

The reference architecture

Customer scenario

Explore Relecloud's solution using serverless and microservices

Deploy Relecloud's solution today with a hands-on lab

Detailed documentation

serverless-microservices-reference-architecture's People

Contributors

Stargazers

Watchers

Forkers

serverless-microservices-reference-architecture's Issues

This issue is for a: (mark with an x)

Mention any other details that might be useful

Please provide us with the following information:

This issue is for a: (mark with an x)

Minimal steps to reproduce

Any log messages given by the failure

Expected/desired behavior

OS and Version?

Versions

Mention any other details that might be useful

This issue is for a: (mark with an x)

Minimal steps to reproduce

Recommend Projects

Recommend Topics

Recommend Org

This issue is for a: (mark with an `x`)

This issue is for a: (mark with an `x`)

This issue is for a: (mark with an `x`)