Code Monkey home page Code Monkey logo

ibm / ibm-cloud-functions-serverless-apis Goto Github PK

View Code? Open in Web Editor NEW
30.0 17.0 40.0 1.59 MB

Create a serverless, event-driven application with Apache OpenWhisk on IBM Cloud Functions that executes code in response to HTTP REST API calls.

Home Page: https://developer.ibm.com/code/patterns/build-serverless-api-handlers/

License: Apache License 2.0

JavaScript 50.73% Shell 32.70% Python 16.56%
openwhisk serverless bluemix ibm-developer-technology-cloud openwhisk-getting-started openwhisk-hands-on-demo openwhisk-sample ibm-cloud-functions ibmcode

ibm-cloud-functions-serverless-apis's Introduction

Serverless APIs with IBM Cloud Functions (powered by Apache OpenWhisk)

Read this in other languages: 한국어, 日本語, 繁體中文.

This project shows how serverless, event-driven architectures can execute code that scales automatically in response to demand from HTTP REST API calls. No resources are consumed until the API endpoints are called. When they are called, resources are provisioned to exactly match the current load needed by each HTTP method independently.

It shows four IBM Cloud Functions (powered by Apache OpenWhisk) actions (written in JavaScript) that write and read data in a MySQL database. This demonstrates how actions can work with supporting data services and execute logic in response to HTTP requests.

One action is mapped to HTTP POST requests. It inserts the supplied cat name and color parameters into the database. A second action is mapped to PUT requests to update those fields for an existing cat. A third action is mapped to GET requests that return specific cat data. A fourth action deletes a given cat data.

The Node.js runtime on the IBM Cloud provides a built-in list of approved npm modules. This demo also highlights how additional Node.js dependencies – such as the MySQL client – can be packaged in a ZIP file with custom actions to provide a high level of extensibility.

Sample Architecture

Included components

  • IBM Cloud Functions (powered by Apache OpenWhisk)
  • ClearDB or Compose (MySQL)

Prerequisite

You should have a basic understanding of the OpenWhisk programming model. If not, try the action, trigger, and rule demo first.

Also, you'll need an IBM Cloud account and the latest OpenWhisk command line tool (ibmcloud fn) installed and on your PATH.

As an alternative to this end-to-end example, you might also consider the more basic "building block" version of this sample.

Steps

  1. Provision MySQL
  2. Create OpenWhisk actions and mappings
  3. Test API endpoints
  4. Delete actions and mappings
  5. Recreate deployment manually

1. Provision MySQL

Log into the IBM Cloud and provision a ClearDB or a Compose for MySQL database instance. ClearDB has a free tier for simple testing, while Compose has tiers for larger workloads.

  • For ClearDB, log into the ClearDB dashboard, and select the default database created for you. Get the user, password and host information under "Endpoint Information".

  • For Compose, get the information from the Service Credentials tab in the IBM Cloud console.

Copy template.local.env to a new file named local.env and update the MYSQL_HOSTNAME, MYSQL_USERNAME, MYSQL_PASSWORD and MYSQL_DATABASE for your MySQL instance.

2. Create OpenWhisk actions and mappings

deploy.sh is a convenience script reads the environment variables from local.env and creates the OpenWhisk actions and API mappings on your behalf. Later you will run these commands yourself.

./deploy.sh --install

Note: If you see any error messages, refer to the Troubleshooting section below. You can also explore Alternative deployment methods.

3. Test API endpoints

There are four helper scripts that simulate HTTP API clients to create, get, update and delete entities against the /v1/cat endpoint.

# POST /v1/cat {"name": "Tarball", "color": "Black"}
client/cat-post.sh Tarball Black

# GET /v1/cat?id=1
client/cat-get.sh 1 # Or whatever integer ID was returned by the command above

# PUT /v1/cat {"id": 1, "name": "Tarball", "color": "Gray"}
client/cat-put.sh 1 Tarball Gray

# DELETE /v1/cat?id=1
client/cat-delete.sh 1

4. Delete actions and mappings

Use deploy.sh again to tear down the OpenWhisk actions and mappings. You will recreate them step-by-step in the next section.

./deploy.sh --uninstall

5. Recreate deployment manually

This section provides a deeper look into what the deploy.sh script executes so that you understand how to work with OpenWhisk triggers, actions, rules, and packages in more detail.

5.1 Create OpenWhisk actions to modify cat data

Create four actions to manage cat data, one for each method (POST, PUT, GET, and DELETE) of our API. The code for the actions is located in /actions. Let's start with the action action that creates a cat record first.

Note: There are a number of built-in packages available in the OpenWhisk Node.js runtime environment. If you need additional packages, you can upload them in a ZIP file along with your action file. More information on the single file versus zipped archive approaches is available in the getting started guide.

5.1.1 The cat package

Because all of the actions rely on the MySQL database service, it's convenient to set the credentials once at the package level. This makes them available to all the actions in the package so we don't need to define them for each action at creation and run time.

source local.env
ibmcloud fn package create cat \
  --param "MYSQL_HOSTNAME" $MYSQL_HOSTNAME \
  --param "MYSQL_PORT" $MYSQL_PORT \
  --param "MYSQL_USERNAME" $MYSQL_USERNAME \
  --param "MYSQL_PASSWORD" $MYSQL_PASSWORD \
  --param "MYSQL_DATABASE" $MYSQL_DATABASE

5.1.2 The cat create action

The JavaScript code for the POST action is in /actions/cat-post-action/index.js. This function depends on the mysql client npm package which we need to connect to the database. Install the package using npm install (which parses package.json) and create a ZIP file that includes both your application and its dependencies.

cd actions/cat-post-action
npm install
zip -rq action.zip *

Next use the OpenWhisk CLI to create an action from action.zip.

# Create
ibmcloud fn action create cat/cat-post \
  --kind nodejs:6 action.zip \
  --web true

Then manually invoke the action using the ibmcloud fn CLI to test.

# Test
ibmcloud fn action invoke \
  --blocking \
  --param name Tarball \
  --param color Black \
  cat/cat-post

Repeat the steps above to create and test the corresponding GET, PUT, and DELETE actions.

Note: Replace the number 1 in your tests below to reflect the actual id returned from the POST action result above.

5.1.3 The cat read action

# Create
cd ../../actions/cat-get-action
npm install
zip -rq action.zip *
ibmcloud fn action create cat/cat-get \
  --kind nodejs:6 action.zip \
  --web true

# Test
ibmcloud fn action invoke \
  --blocking \
  --param id 1 \
  cat/cat-get
5.1.4 The cat update action
# Create
cd ../../actions/cat-put-action
npm install
zip -rq action.zip *
ibmcloud fn action create cat/cat-put \
  --kind nodejs:6 action.zip \
  --web true

# Test
ibmcloud fn action invoke \
  --blocking \
  --param name Tarball \
  --param color Gray \
  --param id 1 \
  cat/cat-put

ibmcloud fn action invoke \
  --blocking \
  --param id 1 \
  cat/cat-get

5.1.5 The cat delete action

# Create
cd ../../actions/cat-delete-action
npm install
zip -rq action.zip *
ibmcloud fn action create cat/cat-delete \
  --kind nodejs:6 action.zip \
  --web true

# Test
ibmcloud fn action invoke \
  --blocking \
  --param id 1 \
  cat/cat-delete

ibmcloud fn action invoke \
  --blocking \
  --param id 1 \
  cat/cat-get

5.2 Create REST API endpoints

Now map a resource endpoint (/cat) to the GET, DELETE, PUT, and POST HTTP methods, associate them with the corresponding OpenWhisk actions, and use the client scripts to test.

# Create
ibmcloud fn api create -n "Cats API" /v1 /cat post cat/cat-post
ibmcloud fn api create /v1 /cat put cat/cat-put
ibmcloud fn api create /v1 /cat get cat/cat-get
ibmcloud fn api create /v1 /cat delete cat/cat-delete

# Test

# POST /v1/cat {"name": "Tarball", "color": "Black"}
client/cat-post.sh Tarball Black

# GET /v1/cat?id=1
client/cat-get.sh 1 # Replace 1 with the id returned from the POST action above

# PUT /v1/cat {"id": 1, "name": "Tarball", "color": "Gray"}
client/cat-put.sh 1 Tarball Gray

# DELETE /v1/cat?id=1
client/cat-delete.sh 1

5.3 Clean up

Remove the API mappings and delete the actions.

ibmcloud fn api delete /v1
ibmcloud fn action delete cat/cat-post
ibmcloud fn action delete cat/cat-put
ibmcloud fn action delete cat/cat-get
ibmcloud fn action delete cat/cat-delete
ibmcloud fn package delete cat

Troubleshooting

Check for errors first in the OpenWhisk activation log. Tail the log on the command line with ibmcloud fn activation poll or drill into details visually with the monitoring console on the IBM Cloud.

If the error is not immediately obvious, make sure you have the latest version of the ibmcloud fn CLI installed. If it's older than a few weeks, download an update.

ibmcloud fn property get --cliversion

Alternative deployment methods

deploy.sh will be replaced with wskdeploy in the future. wskdeploy uses a manifest to deploy declared triggers, actions, and rules to OpenWhisk.

You can also use the following button to clone a copy of this repository and deploy to the IBM Cloud as part of a DevOps toolchain. Supply your OpenWhisk and MySQL credentials under the Delivery Pipeline icon, click Create, then run the Deploy stage for the Delivery Pipeline.

Deploy to the IBM Cloud

License

This code pattern is licensed under the Apache Software License, Version 2. Separate third party code objects invoked within this code pattern are licensed by their respective providers pursuant to their own separate licenses. Contributions are subject to the Developer Certificate of Origin, Version 1.1 (DCO) and the Apache Software License, Version 2.

Apache Software License (ASL) FAQ

ibm-cloud-functions-serverless-apis's People

Contributors

coryma avatar dokun1 avatar dolph avatar hongjsk avatar imgbotapp avatar jthomas avatar jzaccone avatar kant avatar krook avatar ljbennett62 avatar stevemar avatar yamachan avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

ibm-cloud-functions-serverless-apis's Issues

Complete IBM Developer Technology checklist

Guidelines

  • Source Code + Documentation on Github   (github.com/ibm)
  • Apache 2.0 License
  • Detailed documentation in README.md
  • Insured by peer reviews, one each from technology and advocacy team as applicable
  • Code updates/fixes/PRs
  • CONTRIBUTING. md and MAINTAINERS. md file to be added
  • Github repo owner has commit/merge privileges
  • Scenario issues are filed and triaged through Github. 
  • Issues have to be triaged and responded to (not solved) within 72 (working) hours.
  • A successful Travis CI build should be a requirement for successful merge

README elements

  • Overview: 
  • Detailed installation/deployment steps. E.g.: 
    Setting up the environment (provisioning services)
    Setting up Component A
    Setting up Component B
    Setting up Component C
  • Runtime instructions (end user documentation)
  • Accessing/running the scenario
  • End to end testing of the scenario
  • Troubleshooting

Cat id varies

Running through the tutorial I noticed that my cat id=1 always returned a blank result. When adding a new cat entry the id varied.

I am not sure if there are clean up steps that would ensure the user is always using id=1, or if it should be stated that the user make note of the id when they do a POST.

Update to use Cloudant instead of MySQL

When I first created this, I used a relational database to show the REST CRUD mapping for a system many developers were already familiar with. That made sense to me from an API point of view.

But, to make this truly serverless and auto-scalable, the backend should be as well. Updates will be a little more complex in Cloudant, but that's the only downside. It also has the benefit of being Lite account compatible.

FYI @dokun1 @jthomas

Tutorial pre-reqs

  • Add steps showing users how to add the wsk CLI to their PATH
  • Add step for git cloning the repo

wsk api-experimental | wsk api failing on test script

Just wiped all OW from my Mac and tried the Serverless API demo from the “getting started” page on bluemix… but it appears the binary whisk CLI (as downloaded via the “Download Whisk CLI” from https://console.ng.bluemix.net/openwhisk/ (Mac) does not support either the “wsk api” or “wsk api-experimental” commands.

The result is Matt$ wsk api
usage: wsk [-h] [-v] [--apihost hostname] [--apiversion version]

      {package,list,activation,namespace,rule,trigger,action,property,sdk}
      ...

wsk: error: argument cmd: invalid choice: ‘api’ (choose from ‘package’, ‘list’, ‘activation’, ‘namespace’, ‘rule’, ‘trigger’, ‘action’, ‘property’, ‘sdk’)

same for api-experimental...

specifically the script to test results in: Matt$ client/cat-post.sh Tarball Black
usage: wsk [-h] [-v] [--apihost hostname] [--apiversion version]

      {package,list,activation,namespace,rule,trigger,action,property,sdk}
      ...

wsk: error: argument cmd: invalid choice: ‘api-experimental’ (choose from ‘package’, ‘list’, ‘activation’, ‘namespace’, ‘rule’, ‘trigger’, ‘action’, ‘property’, ‘sdk’)
curl: no URL specified!
curl: try ‘curl --help’ or ‘curl --manual’ for more information

So it appears that MAYBE api-experimental is supported but the script invocation is wrong (yet to look)?

at the very least the HELP for the wsk command does not acknowledge either command as valid.

Update for new API Gateway and Web Actions approach

Refactor this sample (or add a parallel repository) to both simplify the example and introduce new features that are now generally available.

  • Simplify to use non-zipped actions (which were needed for the MySQL client package) and use Cloudant as the data store
  • Update to use a combination of API Gateway + Web Actions.
  • Provide a linkage to an alternate approach with the Serverless Framework.
  • Support a wskdeploy manifest to replace deploy.sh as it matures.
  • Incorporate cost/performance metrics against existing ways to provide serverful web APIs.
  • Include a linkage to the why behind this approach.

what about web actions?

Should we extend this example to start with "basic" web actions, and then extend it with the richer functionality of API Gateway?

Or should we alternatively create a new sample app for web actions?

Connections not closing immediately with destory()

The current connection limit for the trial clearDB is 4 connections. While this works consistently for one time tests, the connections are not closing out immediately with destroy which causes the following error:

{
"code": "ER_USER_LIMIT_REACHED",
"errno": 1226,
"sqlState": "42000",
"fatal": true
}

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.