Code Monkey home page Code Monkey logo

mcw-serverless-architecture's Introduction

Serverless architecture

This workshop is archived and no longer being maintained. Content is read-only.

Contoso is rapidly expanding its toll booth management business to operate in a much larger area. As this is not their primary business, which is online payment services, they are struggling with scaling up to meet the upcoming demand to extract license plate information from many new tollbooths, using photos of vehicles uploaded to cloud storage. Currently, they have a manual process where they send batches of images to a 3rd-party who manually transcodes the license plates to CSV files that they send back to Contoso to upload to their online processing system.

They want to automate this process in a way that is cost-effective and scalable. They believe serverless is the best route for them but do not have the expertise to build the solution.

November 2021

Target Audience

Application developers

Abstracts

Workshop

In this workshop, you learn about setting up and configuring a serverless architecture within Azure using a combination of Azure Functions, Azure Logic Apps, Azure Event Grid, Azure Cosmos DB, and Azure Data Lake Storage. The focus is on removing server management from the equation, breaking down the solution into smaller components that are individually scalable, and allowing the customer to only pay for what they use.

At the end of this workshop, you will be able to break business logic down into discrete components that can independently scale and leverage vision algorithms to detect objects and extract text. You’ll know how to utilize Cosmos DB as a highly available NoSQL data store, build workflows using Azure Logic Apps, and conditionally send alerts based on operations. Finally, you’ll have the knowledge to monitor a serverless topology and implement a Continuous Deployment DevOps process to publish changes automatically.

Whiteboard Design Session

In this whiteboard design session, you work with a group to design a solution for processing vehicle photos in near real-time, as they are uploaded to a data lake, using serverless technologies on Azure. The license plate data must be extracted and stored in a highly available NoSQL data store for exporting. The data export process is orchestrated by a serverless Azure component that coordinates exporting new license plate data to file storage and sending notifications as needed. You will also configure a Continuous Deployment process to publish new changes to Function Apps automatically. Finally, the entire processing pipeline will need to be monitored, with particular attention paid to components scaling to meet processing demand.

At the end of this whiteboard design session, you will have greater insight into how best to take advantage of serverless architectures. You will understand better how to design highly scalable and cost-effective solutions that require very little code and virtually no infrastructure compared to traditional hosted web applications and services.

Hands-on Lab

In this hands-on lab, you implement an end-to-end solution using a supplied sample based on Microsoft Azure Functions, Azure Cosmos DB, Azure Event Grid, and related services. The scenario will include implementing compute, storage, workflows, and monitoring using various components of Microsoft Azure. You can implement the hands-on lab on your own. However, it is highly recommended to pair up with other members at the lab to model a real-world experience and to allow each member to share their expertise for the overall solution.

At the end of the hands-on-lab, you will have confidence in designing, developing, and monitoring a serverless solution that is resilient, scalable, and cost-effective.

Azure services and related products

Azure solution

Cloud-Native Apps

Related references, resources, and material

Help & Support

We welcome feedback and comments from Microsoft SMEs & learning partners who deliver MCWs.

Having trouble?

  • First, verify you have followed all written lab instructions (including the Before the Hands-on lab document).
  • Next, submit an issue with a detailed description of the problem.
  • Do not submit pull requests. Our content authors will make all changes and submit pull requests for approval.

If you are planning to present a workshop, review and test the materials early! We recommend at least two weeks prior.

Please allow 5 - 10 business days for review and resolution of issues.

mcw-serverless-architecture's People

Contributors

bala1074 avatar ciwchris avatar codingbandit avatar daronyondem avatar dawnmariedesjardins avatar drmanhatin avatar emilysaeli avatar envera avatar givenscj avatar hopero929 avatar ishammohamed avatar jinssi avatar joelhulen avatar kylebunting avatar microsoftopensource avatar msftgits avatar mwasham avatar rbmathis avatar sadukie avatar saimachi avatar timahenning avatar waltermyersiii avatar zoinertejada avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

mcw-serverless-architecture's Issues

Verify links in HOLs

Folder and document names have been updated. Please check your HOL documents for links that use folder names in their path and make sure they are still valid and working.

Please work on following issues:

In Exercise 1- Task 2.4 need to change screenshot for Create Function App blade. Now, we have to select Runtime Stack(Language) during create Function App. For TollBoothEvents function need to select Java Script for Runtime Stack.
snag_1dd9e9c5
For the second function TollBoothFunctionApp choose .NET for Runtime Stack option.
In Exercise 3- Task 1: UI for adding Event Grid Trigger is changed. Please add new screenshots.
Also please re-validate the lab and update necessary changes.
Please update the lab guide and let me know when it will be ready.

TollBothFunctionApp BlobTrigger not firing

Files are uploaded to the blob storage but function Process Image is not triggered, I tried to update Microsoft.Azure.Webjobs to version 3.0.0 as documentation indicates, I get this error

serverlesserror

Thank you in advance for all your help.

cosmosdb query requires a partition key in the URI

By default when creating a cosmosdb it is required to enable partition. The provided db URI gives the following error. How do I correct this ?
2018-12-31T12:30:53.872 [Error] Critical error: One or more errors occurred.
2018-12-31T12:30:53.872 [Error] Cross partition query is required but disabled. Please set x-ms-documentdb-query-enablecrosspartition to true, specify x-ms-documentdb-partitionkey, or revise your query to avoid this exception.

ActivityId: 5d1538fe-4235-40c0-9700-e7cd4a9f00bf, Microsoft.Azure.Documents.Common/2.1.0.0, Windows/10.0.14393 documentdb-netcore-sdk/1.7.1

Uploadimage.exe file is missing

Exercise 4:- Task 3 :- step 5
Showing an error , i.e file name "Uploadimage.exe" is missing.
I have attached screenshot for reference
001

Exercise 2: Task 5:Not able to add Event Grid Subscription

Exercise 2: Task 5: Step 11: The Integration flow is not similar for me as it is in the lab guide. Also there's a warning highlighted in the image below. Not sure why the warning message is there.

image

Step 12: Not able to add Event Grid Subscription. I tried the way shown in the image below but this also didn't work.

image

Can you please look into this?

Thanks,
Seerat

January 2020 - content update

This workshop is scheduled for a January 2020 content update. Please review open issue #56 and provide additional recommended changes for SME review.

Minor: screen capture of creating a storage account is not up to date anymore

at Task 1: Provision the storage account in section Exercise 1: Azure data, storage, and serverless environment setup the screen caputre of step 4.g. doesn't reassemble the current UI in the Azure Portal anymore. The end user has to select the Hot or Cold tier at a different place than shown in the screen capture.

Missing resource group

Hi

The first step is to create a Storage account which require a resource group. But that is not part of the steps.

Automation

Thanks to the authors for this intro to Azure serverless architecture.

From a "cloud engineering" perspective, it would be nice to see this setup automated so it could live in a "gitops-spirit" environment.

I will work on this a bit and hope to have a first version by the end of the year.
Never really worked with ARM, so this will take some time.
And it's a sparetime project to me.

Thanks again

Creation of Second Function App should use "node.js" Runtime Stack instead of Java script

Exercise 1 Task 2 while creating the second Function App Lab guide says select java script for Runtime Stack but Currently we see that javascript is not available in Azure Portal, avaiable is only java but if we're selecting java then we're facing issues In Exercise 3 Task 1 Create function to save license plate data to Azure Cosmos DB
Step3 : In the blade menu, select Functions, the click + New Function.

Here we need to create new Function but due to wrong Runtime stack selection while create of second Function App i.e. app whose name ends with Events, we're not able to view the option which is mentioned in Step 4: Enter event grid into the template search form, then select the Azure Event Grid trigger template.

When we're creating the function app with node.js Runtime stack then we're able to create new function as per the instructions mentioned in Step 4: Enter event grid into the template search form, then select the Azure Event Grid trigger template.

@DawnmarieDesJardins Could you please assign this ASAP, as we have a workshop schedued on Aug 15th.

Following needs to be changed:

  • Exercise 1 Task 2 while creating the second Function App users should select node.js Runtime stack instead of java/ as javascript is not available in Azure portal.

  • Instructions for Create function to save license plate data to Azure Cosmos DB needs additional steps:

  • Before Step 4 need instructions as mentioned below:
    4 (a). After clicking on + New Function, select In-Portal then click continue as shown in below image:
    image

4 (b). Select More templates
image

4 (c). Click on Finish and view template

Then users can continue from Step 4 mentioned in lab guide.

Thanks in Advance
Abhishek Pathania

Outdated workshop?

Hi, I am planning to deliver this workshop to one of my customers next week but it seems that some of the instructions and the screenshots are outdated - they no longer reflect what the portal looks like or the steps that need to be followed to accomplish certain tasks. So far I have been able to figure out most tasks but I got stuck on Exercise 2 - Task 5 (Adding EventGrid Subscription to the Function App). Are there any plans to update this workshop soon? Can I get in touch with someone internally (I am MSFT FTE) to get this running in my environment so I can deliver it to my customer next week?

Thank you!

The OCR is not reading the plate number

Does anybody know why the ocr is not reading the plate number from the same pics attached to the project files?
all the pictures are moving to NeedsManualReview.

Images to be updated in the lab guide

Exercise 1: Task 2: Step 10: Here Version needs to be added as shown below:

image

Exercise 1: Task 3: Step 4: Now Event Schema is on the separate page from Basics:

image

Exercise 1: Task 4: Step 3: There are new options available while creating Azure Cosmos DB Account, they are - Apply Free Tier Discount, Account Type and Availability Zones.

image

Exercise 1: Task 4: Step 7: Indexing is added as shown below:

image

Exercise 1: Task 4: Step 10 (d): To add throughput firstly you need to check ** Provision dedicated throughput for this container** box and then only you can add throughput.

image

Exercise 1: Task 6: Step 4: Screenshot needs to be updated.

image

Missing alt-text

HOL Unguided - Solution architecture - line 165 in .md is missing alt-text

Issue regarding Function App

Exercise 3: Task 4: Step 7: On saving the code it shows the following log.

image

Exercise 3: Task 5: Step 1: There's no button to Add Event Grid subscription as now we need to do it separately.

Step 2: When tried to separately add Event Grid Subscription, new point is that for:
a. Endpoint Type: Azure Function
b. Endpoint: Click on select endpoint. Now Select Azure Function window will appear, add following configurations there:

  • Select your subscription from the drop-down.
  • Select you Resource Group serverless-architecture from the drop-down.
  • Function App: Select TollBoothFunctionApp from the drop-down.
  • Slot: Production
  • Function: Select ProcessImage from the drop-down.
  • Then select Confirm Selection and Create.

Exercise 4: Task 3: Step 10: Not getting the same output as shown in lab guide for Live Metrics in Application Insights.

image

Exercise 5: Task 1: No documents fetched under Proccessed and NeedsManualReview collections.

image

Can you please check?

Thanks,
Seerat

Misleading HOL step-by-step instruction

Bullets "Task 2: Finish the ProcessImage function" are out of order.
In addition it sould be added a note clarifying that TODO 5,6,7 will be addressed later in the guide.
Many thanks

Screenshots are very old. Need to update.

  1. In Exercise 1- Task 2.4 need to change screenshot for Create Function App blade. Now, we have to select Runtime Stack(Language) during create Function App. For TollBoothEvents function need to select Java Script for Runtime Stack.
    snag_1dd9e9c5
  2. For the second function TollBoothFunctionApp choose .NET for Runtime Stack option.
  3. In Exercise 3- Task 1: UI for adding Event Grid Trigger is changed. Please add new screenshots.
  4. Also please re-validate the lab and update necessary changes.
  5. Please update the lab guide and let me know when it will be ready.

How to deploy the function app directly from VS

Currently, the function app project seems a bit old at https://github.com/Microsoft/MCW-Serverless-architecture/tree/master/Hands-on%20lab/starter

If you directly deploy from VS, you will get deployment failure with below detail:
Web deployment task failed. (Web Deploy experienced a connection problem with the server and had to terminate the connection. Contact your server administrator if the problem persists. Learn more at: http://go.microsoft.com/fwlink/?LinkId=221672#ERROR_CONNECTION_TERMINATED

How to deploy the function app directly from VS

the project’s with runtime v1, so you’d need to create/deploy to v1 Azure function app. And add the function runtime version in the csproj file:

  1. Right click the project and edit the csproj file.
  2. Add <AzureFunctionsVersion>v1</AzureFunctionsVersion> under <PropertyGroup>

To enable “Run from package file” option in Publish blade,

  1. Right click the project and manage Nuget packages
  2. Update Microsoft.NET.Sdk.Functions to latest version (1.0.24).
  3. Then you will see “Run from package file” in Publish blade.

Then you'll be able to deploy the project to Azure with “Run from package file” enabled.

Couldn't publish to function app from VS code (mac)

On VS Code (Mac) publish didn't work.

  1. dotnet restore and dotnet build worked, as well as build from VS Code
  2. But deploy didn't work:

image

  1. I also tried from Visual Studio on Mac (not code) but have the same error.

I tried to publish 3 other apps and working ok.

Possible something with path.

Custom topic sub error

When i do as in lab and subscribe savePlateData and queuePlateForManualCheckup, savePlateData documents are not written to cosmos. When i subscribe to all Events it is working. queuePlateForManualCheckup has no problem. But savePlateData event not working.

StatusCode: 404: 'Resource Not Found'. Wrong Vision API OCR URI.

Exercise 2->Task 4->Step 7.
uriBase = Environment.GetEnvironmentVariable("computerVisionApiUrl"); is wrong (response is StatusCode: 404: 'Resource Not Found'.)

var uriBase = Environment.GetEnvironmentVariable("computerVisionApiUrl") + "vision/v2.1/ocr";

It should contain also "vision/v2.1/ocr".

Update serverless architecture MCW

This package is scheduled for a November update. Please fork this repo, make your updates, then reference this issue when you submit your pull request.

Update Header image

Need new MCW headers on all docs please.
Need to rename folders HOL & WDS

PPT

Not all images are in the same style, per the Microsoft style guide. Some are "live" and some are clip art.

CosmosDB Processed Table Empty

There seems to be an issue with the lab. I have tried executing it multiple times and I always end up with an empty "Processed" table - no items at all. Can someone review the steps to make sure nothing is missing and/or ping me internally to discuss this? Thank you.

MicrosoftTeams-image (1)

September 2019 - Content update

This MCW is scheduled for content update.
Solliance - please review current content and recommended updates for SME review.

HOL Issue: Storage Acct Creation

In the very first step in the pre-requisites it says to build a Storage account with the Blog Type and Hot Tier specified. However, further down it says after copying down the access keys, it says to go to the Containers setting under General Settings, however there is no such place within any of the available Storage Account types. This needs to be resolved before the Ready event coming up.

This should actually be using the "Browse Blobs" settings under the Blobs heading.

Brian

Update Readme file

Please add:
Introduction
Update abstracts to reflect new style
Add services/products
Related links (if any)

September 2020 update suggestions

I recommend the following updates. Please provide feedback on the following:

  • Update all images used in the lab to correspond with the latest portal UI changes
  • Update NuGet packages and SDKs to latest versions
  • Update to Azure Functions 3.x runtime and .NET Core 3.1
  • Add GitHub Actions to the WDS and architecture diagram
  • Add Durable Functions to WDS and use case scenario

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.