Code Monkey home page Code Monkey logo

aws-lambda-env-modeler's Introduction

AWS Lambda Environment Variables Modeler (Python)

license PythonSupport PyPI version PyPi monthly downloads codecov version OpenSSF Scorecard issues

alt text

AWS-Lambda-Env-Modeler is a Python library designed to simplify the process of managing and validating environment variables in your AWS Lambda functions.

It leverages the power of Pydantic models to define the expected structure and types of the environment variables.

This library is especially handy for serverless applications where managing configuration via environment variables is a common practice.

📜Documentation | Blogs website

Contact details | [email protected]

The Problem

Environment variables are often viewed as an essential utility. They serve as static AWS Lambda function configuration.

Their values are set during the Lambda deployment, and the only way to change them is to redeploy the Lambda function with updated values.

However, many engineers use them unsafely despite being such an integral and fundamental part of any AWS Lambda function deployment.

This usage may cause nasty bugs or even crashes in production.

This library allows you to correctly parse, validate, and use your environment variables in your Python AWS Lambda code.

Read more about it here

Features

  • Validates the environment variables against a Pydantic model: define both semantic and syntactic validation.
  • Serializes the string environment variables into complex classes and types.
  • Provides means to access the environment variables safely with a global getter function in every part of the function.
  • Provides a decorator to initialize the environment variables before executing a function.
  • Caches the parsed model for performance improvement for multiple 'get' calls.

Installation

You can install it using pip:

pip install aws-lambda-env-modeler

Getting started

Head over to the complete project documentation pages at GitHub pages at https://ran-isenberg.github.io/aws-lambda-env-modeler

Usage

First, define a Pydantic model for your environment variables:

from pydantic import BaseModel, HttpUrl

class MyEnvVariables(BaseModel):
    DB_HOST: str
    DB_PORT: int
    DB_USER: str
    DB_PASS: str
    FLAG_X:  bool
    API_URL: HttpUrl

Before executing a function, you must use the @init_environment_variables decorator to validate and initialize the environment variables automatically.

The decorator guarantees that the function will run with the correct variable configuration.

Then, you can fetch the environment variables using the global getter function, 'get_environment_variables,' and use them just like a data class. At this point, they are parsed and validated.

from aws_lambda_env_modeler import init_environment_variables

@init_environment_variables(MyEnvVariables)
def my_handler_entry_function(event, context):
    # At this point, environment variables are already validated and initialized
    pass

Then, you can fetch and validate the environment variables with your model:

from aws_lambda_env_modeler import get_environment_variables

env_vars = get_environment_variables(MyEnvVariables)
print(env_vars.DB_HOST)

Disabling Cache for Testing

By default, the modeler uses cache - the parsed model is cached for performance improvement for multiple 'get' calls.

In some cases, such as during testing, you may want to turn off the cache. You can do this by setting the LAMBDA_ENV_MODELER_DISABLE_CACHE environment variable to 'True.'

This is especially useful in tests where you want to run multiple tests concurrently, each with a different set of environment variables.

Here's an example of how you can use this in a pytest test:

import json
from http import HTTPStatus
from typing import Any, Dict
from unittest.mock import patch

from pydantic import BaseModel
from typing_extensions import Literal

from aws_lambda_env_modeler import LAMBDA_ENV_MODELER_DISABLE_CACHE, get_environment_variables, init_environment_variables


class MyHandlerEnvVars(BaseModel):
    LOG_LEVEL: Literal['DEBUG', 'INFO', 'ERROR', 'CRITICAL', 'WARNING', 'EXCEPTION']


@init_environment_variables(model=MyHandlerEnvVars)
def my_handler(event: Dict[str, Any], context) -> Dict[str, Any]:
    env_vars = get_environment_variables(model=MyHandlerEnvVars)  # noqa: F841
    # can access directly env_vars.LOG_LEVEL as dataclass
    return {
        'statusCode': HTTPStatus.OK,
        'headers': {'Content-Type': 'application/json'},
        'body': json.dumps({'message': 'success'}),
    }


@patch.dict('os.environ', {LAMBDA_ENV_MODELER_DISABLE_CACHE: 'true', 'LOG_LEVEL': 'DEBUG'})
def test_my_handler():
    response = my_handler({}, None)
    assert response['statusCode'] == HTTPStatus.OK
    assert response['headers'] == {'Content-Type': 'application/json'}
    assert json.loads(response['body']) == {'message': 'success'}

Code Contributions

Code contributions are welcomed. Read this guide.

Code of Conduct

Read our code of conduct here.

Connect

License

This library is licensed under the MIT License. See the LICENSE file.

aws-lambda-env-modeler's People

Contributors

dependabot[bot] avatar heitorlessa avatar ran-isenberg avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

aws-lambda-env-modeler's Issues

@init_environment_variables decorator TypeError when handler has additional keyword arguments

Describe the bug
A clear and concise description of what the bug is.

To Reproduce
Steps to reproduce the behavior:

Add an additional keyword argument to a Lambda handler, and use the decorator

@init_environment_variables(model=PrcStreamVars)
def process_stream(
    event: dict[str, Any],
    context: LambdaContext,
    event_handler: BaseEventHandler | None = None,
) -> dict:
    ....

Call the handler with an additional keyword argument like so and it'll result in TypeError:

    process_stream(event=dynamodb_stream_events, context=generate_context(), event_handler=event_store)

Expected behavior
A clear and concise description of what you expected to happen.

Decorator should preserve the function signature so to not break at runtime when using additional keyword arguments.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

  • OS: [e.g. iOS]
  • Browser [e.g. chrome, safari]
  • Version [e.g. 22]

Smartphone (please complete the following information):

  • Device: [e.g. iPhone6]
  • OS: [e.g. iOS8.1]
  • Browser [e.g. stock browser, safari]
  • Version [e.g. 22]

Additional context
Add any other context about the problem here.

Feature request: provide a Python function to load lambda env vars in local IDE for tests

I'm looking for feedback for a feature where:

  1. In your integration tests you setup env vars since the decorator 'init_environment_variables' will require them to exist when you call the handler function.
  2. instead of manually setting values, a better way would be to load the "real" env vars deployed to your function. That way, you know for sure they are correct even before running your e2e tests and you can remove the manual code setup from conftest.py
import os

import pytest

from cdk.service.constants import (
    CONFIGURATION_NAME,
    ENVIRONMENT,
    IDEMPOTENCY_TABLE_NAME_OUTPUT,
    POWER_TOOLS_LOG_LEVEL,
    POWERTOOLS_SERVICE_NAME,
    SERVICE_NAME,
    TABLE_NAME_OUTPUT,
)
from tests.utils import get_stack_output


@pytest.fixture(scope='module', autouse=True)
def init():
    os.environ[POWERTOOLS_SERVICE_NAME] = SERVICE_NAME
    os.environ[POWER_TOOLS_LOG_LEVEL] = 'DEBUG'
    os.environ['REST_API'] = 'https://www.ranthebuilder.cloud/api'
    os.environ['ROLE_ARN'] = 'arn:partition:service:region:account-id:resource-type:resource-id'
    os.environ['CONFIGURATION_APP'] = SERVICE_NAME
    os.environ['CONFIGURATION_ENV'] = ENVIRONMENT
    os.environ['CONFIGURATION_NAME'] = CONFIGURATION_NAME
    os.environ['CONFIGURATION_MAX_AGE_MINUTES'] = '5'
    os.environ['AWS_DEFAULT_REGION'] = 'us-east-1'  # used for appconfig mocked boto calls
    os.environ['TABLE_NAME'] = get_stack_output(TABLE_NAME_OUTPUT)
    os.environ['IDEMPOTENCY_TABLE_NAME'] = get_stack_output(IDEMPOTENCY_TABLE_NAME_OUTPUT)


@pytest.fixture(scope='module', autouse=True)
def table_name():
    return os.environ['TABLE_NAME']

I can provide a function that loads the env vars for you. It can accept one or more of the following parameters:

  1. stack name
  2. function name
  3. function ARN

Usage:
before running an integration test for a specific lambda with pytest as show here:
you will create a pytest fixture that will load the env vars for that function.
An easy way to do in to use stack outputs for the names or provide stack name and lambda regex/name to look for.

What do you think of the feature? how would you like to use it? provide stack name and lambda name? or just ARN?

import pytest

@pytest.fixture
def init():
    load_env_vars_from(stack_name=..., lambda_arn=...)

Disable `lru_caching` when needed in testing context

Is your feature request related to a problem? Please describe.

AS A

Developer

I WANT TO

Disable lru_caching when I'm testing an AWS Lambda handler, decorated with a init_environment_variables and a different set of provided env vars (validated with a given model AppEnvVars).

SO THAT

I can ensure that the AWS Lambda handler validates the given set of env vars against the model

Describe the solution you'd like
Below is a quite addition to the existing decorator:

def init_environment_variables(model: Type[Model], cache: bool = True):
    ...
    def decorator(lambda_handler_function: Callable):
        @wraps(lambda_handler_function)
        def wrapper(event: Dict[str, Any], context, **kwargs):
-            __parse_model(model)
+           __effective_parse_model = __parse_model if cache else __parse_model.__wrapped__
+           __effective_parse_model(model)
            return lambda_handler_function(event, context, **kwargs)

        return wrapper

    return decorator

When used in a test:

from unittest import mock
from aws_lambda_env_modeler import init_environment_variables
from ui.awslambda import certificate_renewal_handler


@mock.patch.dict(
    os.environ,
    {
        "ENV1": fake_username(), # required
        "ENV2": fake_password(), # required
    },
    clear=True,
)
def test_ok():
  _ = init_environment_variables(model=AppEnvVars)(
                certificate_renewal_handler.handle
            )(event, context)

def test_fail():
  with pytest.raises(ValueError):
    _ = init_environment_variables(model=AppEnvVars, cache=False)(
                certificate_renewal_handler.handle
            )(event, context)

Describe alternatives you've considered
None

Additional context
None

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.