Code Monkey home page Code Monkey logo

btp-resource-consumption-monitor's Introduction

REUSE status

BTP Resource Consumption Monitor

Description

Application to monitor and alert on commercial and technical consumption of BTP services. See also SAP Blog: Prototype: Keep Track of those BTP Credits.

Screenshot of a service page

Disclaimer: This tool is provided as-is and is not covered by SAP Support. This is not a replacement of the official billing documents you receive from SAP. The information provided by this tool is purely indicative.

Requirements

The following Subscriptions are required to deploy and use this application:

  • SAP HANA Cloud (you can re-use an existing instance)
  • SAP Work Zone (Standard edition is sufficient)
  • SAP Business Application Studio

The following Entitlements need to be available to use this application:

  • Alert Notification: standard
  • Authorization and Trust Management Service: application
  • Destination service: lite
  • HTML5 Application Repository Service: app-host
  • Job Scheduling Service: standard
  • SAP HANA Schemas & HDI Continers: hdi-shared
  • Usage Data Management Service: reporting-ga-admin
  • Application Logging Service: standard (optional service)

Download and Installation

1. CF Application

In Business Application Studio, make sure to have a Development Space of kind Full Stack Cloud Application with the additional Development Tools for SAP Build Work Zone extension enabled.

Clone this repository in your environment and open the project.

Important - First change the following code lines:

Option A. Deploy with Alert Notification configuration (existing configuration will be overwritten):

cd cf
npm install
mbt build
cf deploy ./mta_archives/btp-resource-consumption_1.0.0.mtar -e mtaext_notifications.mtaext

Option B. Deploy without changing any Alert Notification configuration:

cd cf
npm install
mbt build
cf deploy ./mta_archives/btp-resource-consumption_1.0.0.mtar

2. Work Zone Content

Step 1. Build

cd workzone
npm install
npm run build

This will create a /workzone/package.zip file which you can download to your local machine to use in step 2 below.

Step 2.Deploy the package

In the Work Zone Site Manager, open the Channel Manager and:

  1. Synchronize your HTML5 Repository
  2. Click on + New, Content Package and upload the generated /workzone/package.zip file, specifying btprc-srv as Runtime Destination
  3. Navigate to the Site Directory and make sure you have a site with the view setting set to Spaces and Pages - New Experience, and which has the BTP Resource Consumption Role role assigned.

3. Role Assignments

In the BTP Cockpit, go to the Security settings of your subaccount and assign the below 2 Role Collections to your user:

  • ~btprc.cpkg_access_role to access the Work Zone content (front-end)
  • BTPResourceConsumption Viewer to access the CF Application (back-end)

Access the application

Open the Work Zone site and navigate to the BTP Credits page.

It is suggested to configure the application and modify the 'Forecasting Configuration' individually for each of the services to make sure the forecast is calculated correctly. Until this is set correctly, the forecasted costs might be unrealisticly high.

Run Locally

To run the application locally, you need to bind to the cloud service instances and run the application in hybrid mode.

cds bind -2 btprc-uas,btprc-db,btprc-dest
cds-ts watch --profile hybrid

Open your browser to http://localhost:4004 where you will find the following Web Applications:

  • /btprcreport/webapp
  • /managealerts/webapp

Tip: This will connect to the HANA Cloud database. In case you want to use a local sqlite database for testing, run cds deploy -2 sqlite, and remove the binding to btprc-db from the .cdsrc-private.json file.

Architecture

BTP Architecture

Initial Data and Recurring Jobs

When started for the first time, the application creates multiple jobs in the Job Scheduling Service:

  • Default_UpdateDailyUsage: a recurring job to retrieve the consumption information for that day. In its default configuration, this job will run every 3 hours (*/3 UTC). Typically, the Usage Data Management API provides new data points around 6am UTC but this can vary.
  • Default_UpdateMonthlyUsage: a recurring job to retrieve the consumption information from the past month. In its default configuration, this job will run every 1st day of the month, at 1:00 AM and 3:00 AM UTC. It runs twice, in case the first run is not successful.
  • Default_UpdateHistoricUsage: a one-time job which runs after creation of the job. It is used to retrieve the consumption information of that day + the past months (default: from October 2023, see settings.ts). This makes sure that after deploying the application you immediately have some information in the dashboards before the scheduled jobs run.

The initial data that this last job retrieves can be refreshed/extended from the UI: in the application open the Data Management menu and click on Load historic data. Here you can specify how far back you want to retrieve the consumption information.

The other menu option (Load consumption for current month) is exactly the same as what the first job runs every 3 hours, so it is not needed to be used and is mainly intented for testing purposes.

You can freely edit/change the created jobs, or create other jobs. When the application (re)starts it will re-create the jobs if they don't exist already, but it won't overwrite them if they do exist.

Note: Jobs have a default expected completion time of 15 seconds. Depending on the query, the application and/or API can take longer to process a request. In that case, the job monitor will classify the run as failed even though it ran successfully.

Forecasting Configuration

Each commercial metric of a service has a Forecasting Configuration. The following settings are available:

  • Excluded: the metric will not be forecasted/propagated. The current consumption for today is what is expected for the entire month. This should be used for 'stable services'.
    • examples: SAP Integration Suite tenants, SAP Business Application Studio users, ...
  • Time Linear: the metric will be forecasted with a linear trend, based on time. The consumption will be divided by the current day in the month and multiplied by 30 to come to a full month's consumption. This should be used for services that have a steady consumption throughout the month.
    • examples: SAP HANA Cloud capacity units, Cloud Foundry GB memory, ...
  • Time Degressive: the metric will be forecasted with a degressive/progressive trend, based on time. This is a variant of the linear approach, but you can give a degression factor to specify the decrease/increase in consumption through the month. This should be used for services that have a free allowance and charges only apply above this allowance (progressive), or for services that have a baseline consumption + some small uptake throughout the month (degressive)
    • Degression factor values:
      • < 1: the usage of the remaining days will be lower than the usage of the past days (degressive)
      • = 1: the usage of the remaining days will be similar to the usage of the past days (linear)
      • > 1: the usage of the remaining days will be higher than the usage of the past days (progressive)
    • examples: SAP Integration Suite transactions (has free allowance), Mobile Services users (most users' first access will be early in the month), ...

The forecasting settings can be all reset to the default value (see settings.ts) from the UI: menu Forecast Management: Revert all forecast settings to default.

Switching Global Accounts

The application can be connected to a different Global Account to monitor that consumption instead of the Global Account where the application is deployed in (default).

To do so:

  • Manually create an instance of the Usage Data Management Service service of plan reporting-ga-admin in the other Global Account. Create a Service Key on that service instance, and copy its contents.
  • In the Global Account where the application is deployed, create a new User Provided Service in which you paste the service key contents.
  • Adapt the mta.yaml on lines 35, 36 and 176, 183 to swap the bound standard service instance for the user-provided instance.

Using Free Tier

For demo and test purposes it is possible to implement this solution using Free Tier services. Be aware there are restrictions when using the Free Tier service plans.

  • Subscriptions:

    • SAP HANA Cloud: free tier plan is available, but note your database will be stopped every night causing jobs to fail. Restart your database in the mornings. See restrictions.
    • SAP Work Zone, standard edition: free tier plan is available and can be used. See restrictions.
    • SAP Business Application Studio: free tier plan is available and can be used. See restrictions.
  • Resources:

    • Cloud Foundry runtime: free tier plan is available and can be used. Create your CF Organisation with the 'free' option before deploying the application.
    • SAP Alert Notification service: free tier plan is available and can be used. Switch your MTA to plan 'free' instead of 'standard' here.
    • SAP Application Logging service: there is no free plan, but this is an optional service, so you can just remove it from your MTA, both here and here.
    • SAP Job Scheduling service: no free plan available. For an indicative pricing, see here.

Known Issues

  • Dynamic Tile: In case the dynamic tile of the Report application does not show your forecasted CPEA credit for this month (but shows 3 dots instead), you will need to manually create a system mapping. In the Work Zone Site Manager, navigate to the Settings menu and go to Alias Mapping. Add a new alias with the following settings: Aliases = sid(BTPRC.CPKG) and Runtime Destination = btprc-srv. If this mapping already exists and the error is still present, edit and save this mapping without making changes to re-trigger it.
  • Cachebuster error 500: In case your applications do not open from the launchpad because of a failed call to /cachebusterTokens.json, you can correct this from the Work Zone Channel Manager. Use the Update content button of the HTML5 Apps entry 2 times (refresh the content twice consecutively), which will create the cache records in the background and then your apps should open correctly.

How to obtain support

Create an issue in this repository if you find a bug or have questions about the content.

For additional support, ask a question in SAP Community.

Contributing

If you wish to contribute code, offer fixes or improvements, please send a pull request. Due to legal reasons, contributors will be asked to accept a DCO when they create the first pull request to this project. This happens in an automated fashion during the submission process. SAP uses the standard DCO text of the Linux Foundation.

Code of Conduct

See Our Code of Conduct.

License

Copyright (c) 2024 SAP SE or an SAP affiliate company. All rights reserved. This project is licensed under the Apache Software License, version 2.0 except as noted otherwise in the LICENSE file.

btp-resource-consumption-monitor's People

Contributors

dependabot[bot] avatar mauriciolauffer avatar ospo-bot[bot] avatar sap-ospo-bot avatar willemi069808 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

btp-resource-consumption-monitor's Issues

Error while deploying

Hi,

I'm getting this below error while deploying the code. I've a subaccount with HANA Cloud and SAP build Workzone, also I made sure all the entitlements are available in this subaccount.

Also one suggestion, mar file name is btp-resource-consumption_1.0.0.mtar not BTPResourceConsumption_1.0.0.mtar so please change that accordingly.

Error with creating a client for content service "btprc-dest": Error retrieving required property "uaa" for service key "btprc-dest-key" for service "btprc-dest"

Thanks
Alok Varma

Understanding the forecasting and alert functionality

Awesome app. Just got it up and running. I notice the 3 hourly job is continually sending me alerts due to the forecast values being 200 - 300% over current actual values. I just have all the default values at this stage for the forecasts. Why would it be forecasting so high. Am I supposed to go through each service and metric and individually set these? Even if I change this to Linear forecasting the percentage forecasts are very high - over 400%.

Also - if I set all forecasts to EXCLUDED - does that now mean I will only receive alerts for actual cost increases?

What I am tying to work out is how to get an alert only when there is a cost blowout. I.e. someone turns on an additional integration suite tenant and triggers a $6k charge. What settings would be necessary to get an alert for that?

Unable to deploy - error create jobscheduler service key

Hi. I am following the instructions to deploy this project but the mta deployment fails with:

Error monitoring creation or update of services: Error creating service "btprc-auth" from offering "xsuaa" and plan "application": Service broker error: Service broker xsuaa failed with: Error creating application null (Error parsing xs-security.json data: Could not find any xsuaa service for instance ID 6c03df46-64dd-418c-a774-3549457b93b5 and service name btprc-scheduler) 
Creating service key "btprc-scheduler-key" for service instance "btprc-scheduler"...
Creating service key "btprc-uas-key" for service instance "btprc-uas"...
Proceeding with automatic retry... (3 of 3 attempts left)
Creating service "btprc-auth" from MTA resource "btprc-auth"...
Async operation for service key of service instance "btprc-scheduler" failed with "10009 CF-UnableToPerform bind could not be completed: Service broker error: Service broker jobscheduler-broker failed with: The BTP Job Scheduling Service does currently not support the creation of service keys. Instead you have to bind an app for accessing this service instance."
A step of the process has failed. Retrying it may solve the issue.

I am trying to test this in a BTP trial account and have set the service plan to "lite" for the app logging service and the job scheduler service.

Any ideas?

Only Homepage loads - other sites won't open

Hey guys,
the homepage loads successfully, but all other links won't open:
Homepage;
image

Configure Alerts (same error for other sites)
image
Any idea what's happening or missing here?

Any help is appreciated, thank you very much,
Christopher

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.