Code Monkey home page Code Monkey logo

python-lambda's People

Contributors

applechief avatar asaolabs avatar carhartl avatar conor-f avatar corbinsimpsonacquia avatar cristianoliveira avatar cyriac avatar dependabot-preview[bot] avatar franciscohanna92 avatar frnsys avatar gmyers-amfam avatar iamjohnford avatar jamrizzi avatar javamonkey79 avatar jmeekhof avatar johnclyde avatar loide avatar mccartney avatar mpgo13 avatar mwbenowitz avatar nficano avatar philchristensen avatar rashadmoarref avatar rishabh-bhargava avatar rzwck avatar simplyahmazing avatar slapula avatar soapergem avatar timgates42 avatar waldyrious avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

python-lambda's Issues

Local Test - Modify service.py file did not reflect the changes

I am new to lambda and Bash script. When I modify the service.py under the root folder, e.g. change e + pi to e - pi, it did not return the result after the changes. Anyone could help? Or do I need to recompiled the service.pyc file everytime I make the changes?

Thanks for the help.

aws api invocation failing

hi folks,
having one more issue here :
when i test the function on the lambda console - it works fine - but when I try to call the API , the following error occurs and the cloudwatch log suggests no parameters were sent.

Traceback (most recent call last):

File "/var/task/service.py", line 11, in handler
"body": e + pi
TypeError: unsupported operand type(s) for +: 'NoneType' and 'NoneType'

called like so:

curl https://<endpoint>/prod/my_lambda_function  --data '{"pi" : 3.14, "e" : 2.718}' -H 'Content-Type: application/json' -v
*   Trying 54.192.136.115...
* Connected to l3rj7tq81d.execute-api.us-west-2.amazonaws.com (54.192.136.115) port 443 (#0)
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
*   CAfile: //anaconda/ssl/cacert.pem
  CApath: none
* TLSv1.2 (OUT), TLS header, Certificate Status (22):
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server accepted to use http/1.1
* Server certificate:
*  subject: C=US; ST=Washington; L=Seattle; O=Amazon.com, Inc.; CN=*.execute-api.us-west-2.amazonaws.com
*  start date: May 30 00:00:00 2017 GMT
*  expire date: Feb 28 23:59:59 2018 GMT
*  subjectAltName: host "l3rj7tq81d.execute-api.us-west-2.amazonaws.com" matched cert's "*.execute-api.us-west-2.amazonaws.com"
*  issuer: C=US; O=Symantec Corporation; OU=Symantec Trust Network; CN=Symantec Class 3 Secure Server CA - G4
*  SSL certificate verify ok.
> POST /prod/my_lambda_function HTTP/1.1
> Host: l3rj7tq81d.execute-api.us-west-2.amazonaws.com
> User-Agent: curl/7.49.0
> Accept: */*
> Content-Type: application/json
> Content-Length: 26
>
* upload completely sent off: 26 out of 26 bytes
< HTTP/1.1 502 Bad Gateway
< Content-Type: application/json
< Content-Length: 36
< Connection: keep-alive
< Date: Mon, 10 Jul 2017 15:58:50 GMT
< x-amzn-RequestId: aa719f09-6588-11e7-a0b0-657c544d8223
< X-Cache: Error from cloudfront
< Via: 1.1 941cbd1049a1cc3d6d633dce8d55cf36.cloudfront.net (CloudFront)
< X-Amz-Cf-Id: KbnVOMc8Fwkm3cLO_pU2X8R5Sr54XIeUdpq9qtgq77QZF7zww65_zw==
<
* Connection #0 to host l3rj7tq81d.execute-api.us-west-2.amazonaws.com left intact
{"message": "Internal server error"}

No such file or directory: 'requirements.txt'

When following the steps in Getting Started, I see this error when I run pip install python-lambda:

Collecting python-lambda
  Using cached python-lambda-0.5.1.tar.gz
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/private/var/folders/br/bx8tyrg91fx1pgqwvnwc315w0000gn/T/pip-build-l7DEkV/python-lambda/setup.py", line 16, in <module>
        pip_requirements = [str(r.req) for r in requirements]
      File "/Users/tmh/virtualenvs/pylambda/lib/python2.7/site-packages/pip/req/req_file.py", line 84, in parse_requirements
        filename, comes_from=comes_from, session=session
      File "/Users/tmh/virtualenvs/pylambda/lib/python2.7/site-packages/pip/download.py", line 425, in get_file_content
        'Could not open requirements file: %s' % str(exc)
    pip.exceptions.InstallationError: Could not open requirements file: [Errno 2] No such file or directory: 'requirements.txt'
    
    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /private/var/folders/br/bx8tyrg91fx1pgqwvnwc315w0000gn/T/pip-build-l7DEkV/python-lambda/

Not pip installing a local package

I am deploying psycopg2 with my lambda build. Because psycopg2 relies on the statically linked libpq.so library, pip install doesn't work (so neither does --local-package).

What I've done in the past is to pre-compile the psycopg2 package and copy the folder directly to lambda dist (no pip install). Seems like a useful feature.

https://github.com/jkehler/awslambda-psycopg2

Support for version in Amazon API and filenames

Currently filenames are coded only with a timestamp... ideally there would be provisions for creating the filename based on a version string. The same input could then in turn be used to set the version as per the AWS Lambda API:

https://docs.aws.amazon.com/lambda/latest/dg/versioning-intro.html

This could be implemented by adding a "version: x.x.x" line to the config.yaml file, if present then the version provided will be used, otherwise the code will exhibit the current behavior for backward compatibility. Another option would be to add a '--version=x.x.x' command line parameter to 'lambda build' and the pending S3 change 'lambda upload'.

It would break 100% compatibility with current outputs, but it would also likely be more intuitive to have the filenames generated be in the format -<version|timestamp>.zip rather than the current -.zip.

Package libraries with native dependencies

I am developing on macOS and want to package Pillow. I have installed and downloaded pillow from an ec2 instance and want it to be packaged instead of the one in site-packages. Right now only way to do is to include PIL in source_directories and uninstall it via pip before packaging.
Is there a better way to do it?

Error when using 'lambda deploy_s3': Unexpected keyword argument 'use_requirements'

When I try to deploy my lambda function with lambda deploy_s3 I receive the following error:

TypeError: deploy_s3() got an unexpected keyword argument 'use_requirements'

I've installed the dependencies of my project trough a requirements.txt file with the command pip install -r requirements.txt. The error doesn't arise if I use lambda deploy.

Enhancement: Environment Variables

What do you think about adding an .env file into every lambda folder? Sending environment variables can be accomplished after uploading the zip file.

# http://docs.aws.amazon.com/lambda/latest/dg/env_variables.html
aws lambda create-function \
    --region us-east-1
    --function-name myTestFunction
    --zip-file fileb://path/package.zip
    --role role-arn
    --environment Variables={LD_LIBRARY_PATH=/usr/bin/test/lib64}
    --handler index.handler
    --runtime nodejs4.3
    --profile default

Logging configuration

Hi, When running invoke locally it appears it does not configure logging..

Should it ?

No handlers could be found for logger "root"

function_exists requires list_functions permission

The current implementation of function_exists requires the calling account to have list_functions permissions. It also iterates through the functions. A better implementation would be to use the get_function function.

Error: IsADirectoryError on init command

Hi there.

We are getting the follow errors using python3:

Traceback (most recent call last):
  File "/Users/cristian.oliveira/.virtualenv/test/bin/lambda", line 53, in <module>
    cli()
  File "/Users/cristian.oliveira/.virtualenv/test/lib/python3.6/site-packages/click/core.py", line 716, in __call__
    return self.main(*args, **kwargs)
  File "/Users/cristian.oliveira/.virtualenv/test/lib/python3.6/site-packages/click/core.py", line 696, in main
    rv = self.invoke(ctx)
  File "/Users/cristian.oliveira/.virtualenv/test/lib/python3.6/site-packages/click/core.py", line 1060, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/cristian.oliveira/.virtualenv/test/lib/python3.6/site-packages/click/core.py", line 889, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/cristian.oliveira/.virtualenv/test/lib/python3.6/site-packages/click/core.py", line 534, in invoke
    return callback(*args, **kwargs)
  File "/Users/cristian.oliveira/.virtualenv/test/bin/lambda", line 20, in init
    aws_lambda.init(CURRENT_DIR)
  File "/Users/cristian.oliveira/.virtualenv/test/lib/python3.6/site-packages/aws_lambda/aws_lambda.py", line 149, in init
    copy(dest_path, src)
  File "/Users/cristian.oliveira/.virtualenv/test/lib/python3.6/shutil.py", line 243, in copy
    copyfile(src, dst, follow_symlinks=follow_symlinks)
  File "/Users/cristian.oliveira/.virtualenv/test/lib/python3.6/shutil.py", line 122, in copyfile
    with open(src, 'rb') as fsrc:
IsADirectoryError: [Errno 21] Is a directory: '/Users/cristian.oliveira/.virtualenv/test/lib/python3.6/site-packages/aws_lambda/project_templates/__pycache__'

It is trying to copy a folder, in this case __pycache__, which I think is a wrong behavior. I have already fixed that here. I going to open a PR with the fix.

Unknown parameter in input: "Environment", must be one of: FunctionName, Runtime, Role, Handler, Code, Description, Timeout, MemorySize, Publish, VpcConfig

when i follow the instructions in the README and
lambda deploy , I get the following stack trace:

(pylam) $ lambda deploy
Gathering pip packages
Installing boto3==1.4.1
Installing botocore==1.4.61
Installing click==6.6
Installing docutils==0.12
Installing futures==3.0.5
Installing jmespath==0.9.0
Installing pip==9.0.1
Installing pyaml==15.8.2
Installing python-dateutil==2.5.3
Installing PyYAML==3.11
Installing s3transfer==0.1.10
Installing setuptools==36.0.1
Installing six==1.10.0
Installing wheel==0.30.0a0
Installing wsgiref==0.1.2
Bundling: 'event.json'
Bundling: 'service.py'
Bundling: 'service.pyc'
Starting new HTTPS connection (1): lambda.us-west-2.amazonaws.com
Creating your new Lambda function
Starting new HTTPS connection (1): sts.amazonaws.com
Creating lambda function with name: my_lambda_function
Traceback (most recent call last):
  File "/pylam/bin/lambda", line 62, in <module>
    cli()
  File "/pylam/lib/python2.7/site-packages/click/core.py", line 716, in __call__
    return self.main(*args, **kwargs)
  File "/pylam/lib/python2.7/site-packages/click/core.py", line 696, in main
    rv = self.invoke(ctx)
  File "/pylam/lib/python2.7/site-packages/click/core.py", line 1060, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/pylam/lib/python2.7/site-packages/click/core.py", line 889, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/pylam/lib/python2.7/site-packages/click/core.py", line 534, in invoke
    return callback(*args, **kwargs)
  File "/pylam/bin/lambda", line 47, in deploy
    aws_lambda.deploy(CURRENT_DIR, use_requirements, local_package)
  File "/pylam/lib/python2.7/site-packages/aws_lambda/aws_lambda.py", line 89, in deploy
    create_function(cfg, path_to_zip_file)
  File "/pylam/lib/python2.7/site-packages/aws_lambda/aws_lambda.py", line 371, in create_function
    Publish=True
  File "/pylam/lib/python2.7/site-packages/botocore/client.py", line 159, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/pylam/lib/python2.7/site-packages/botocore/client.py", line 470, in _make_api_call
    api_params, operation_model, context=request_context)
  File "/pylam/lib/python2.7/site-packages/botocore/client.py", line 523, in _convert_to_request_dict
    api_params, operation_model)
  File "/pylam/lib/python2.7/site-packages/botocore/validate.py", line 270, in serialize_to_request
    raise ParamValidationError(report=report.generate_report())
botocore.exceptions.ParamValidationError: Parameter validation failed:
Unknown parameter in input: "Environment", must be one of: FunctionName, Runtime, Role, Handler, Code, Description, Timeout, MemorySize, Publish, VpcConfig

"lambda invoke" failed without environment_variables in config

when environment_variables are commented out, lambda_invoke failed with the following exception:

Traceback (most recent call last):
  File "/home/ubuntu/Envs/env1/bin/lambda", line 67, in <module>
    cli()
  File "/home/ubuntu/Envs/env1/local/lib/python2.7/site-packages/click/core.py", line 716, in __call__
    return self.main(*args, **kwargs)
  File "/home/ubuntu/Envs/env1/local/lib/python2.7/site-packages/click/core.py", line 696, in main
    rv = self.invoke(ctx)
  File "/home/ubuntu/Envs/env1/local/lib/python2.7/site-packages/click/core.py", line 1060, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/ubuntu/Envs/env1/local/lib/python2.7/site-packages/click/core.py", line 889, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/ubuntu/Envs/env1/local/lib/python2.7/site-packages/click/core.py", line 534, in invoke
    return callback(*args, **kwargs)
  File "/home/ubuntu/Envs/env1/bin/lambda", line 40, in invoke
    aws_lambda.invoke(CURRENT_DIR, event_file, verbose)
  File "/home/ubuntu/Envs/env1/local/lib/python2.7/site-packages/aws_lambda/aws_lambda.py", line 147, in invoke
    for key, value in cfg.get('environment_variables').items():
AttributeError: 'NoneType' object has no attribute 'items'

This happened on a project where I didn't need env vars and can't deploy them due to KMS issue when encrypting env vars.

lambda function gets enviroment variable name

When calling invoke, if you try to get a env var (os.environ['host']) set in the config file like so:
host: $HOST it will print out $HOST instead of the actual value of the envar $HOST

Function already exists error after initial lambda deploy

Hi,

My deployment completes successfully the first time, but errors out on subsequent deployments with this error:

botocore.exceptions.ClientError: An error occurred (ResourceConflictException) when calling the CreateFunction operation: Function already exist: es_get_channel_http_microservice

Deploying demo app - "Unknown parameter in input: "Environment"

I'm attempting to deploy the basic demo app to our account, just to get familiar with the workflow. However, when attempting to initialize the deploy, I receive this error, which is traced to line 371 of the aws_lambda.py code (the section dealing with "Environment" variables):

  File "/Users/x/Envs/pylambda_slack/bin/lambda", line 62, in <module>
    cli()
  File "/Users/x/Envs/pylambda_slack/lib/python2.7/site-packages/click/core.py", line 716, in __call__
    return self.main(*args, **kwargs)
  File "/Users/x/Envs/pylambda_slack/lib/python2.7/site-packages/click/core.py", line 696, in main
    rv = self.invoke(ctx)
  File "/Users/x/Envs/pylambda_slack/lib/python2.7/site-packages/click/core.py", line 1060, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/x/Envs/pylambda_slack/lib/python2.7/site-packages/click/core.py", line 889, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/x/Envs/pylambda_slack/lib/python2.7/site-packages/click/core.py", line 534, in invoke
    return callback(*args, **kwargs)
  File "/Users/x/Envs/pylambda_slack/bin/lambda", line 47, in deploy
    aws_lambda.deploy(CURRENT_DIR, use_requirements, local_package)
  File "/Users/x/Envs/pylambda_slack/lib/python2.7/site-packages/aws_lambda/aws_lambda.py", line 89, in deploy
    create_function(cfg, path_to_zip_file)
  File "/Users/x/Envs/pylambda_slack/lib/python2.7/site-packages/aws_lambda/aws_lambda.py", line 371, in create_function
    Publish=True
  File "/Users/x/Envs/pylambda_slack/lib/python2.7/site-packages/botocore/client.py", line 159, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/Users/x/Envs/pylambda_slack/lib/python2.7/site-packages/botocore/client.py", line 470, in _make_api_call
    api_params, operation_model, context=request_context)
  File "/Users/x/Envs/pylambda_slack/lib/python2.7/site-packages/botocore/client.py", line 523, in _convert_to_request_dict
    api_params, operation_model)
  File "/Users/x/Envs/pylambda_slack/lib/python2.7/site-packages/botocore/validate.py", line 270, in serialize_to_request
    raise ParamValidationError(report=report.generate_report())
botocore.exceptions.ParamValidationError: Parameter validation failed:
Unknown parameter in input: "Environment", must be one of: FunctionName, Runtime, Role, Handler, Code, Description, Timeout, MemorySize, Publish, VpcConfig

This appears to be the function being called - any thoughts? I'm not looking to push any environment variable through in this deploy, just get a working app up.

    client.create_function(
        FunctionName=func_name,
        Runtime=cfg.get('runtime', 'python2.7'),
        Role=role,
        Handler=cfg.get('handler'),
        Code={'ZipFile': byte_stream},
        Description=cfg.get('description'),
        Timeout=cfg.get('timeout', 15),
        MemorySize=cfg.get('memory_size', 512),
        Environment={
            'Variables': {
                key.strip('LAMBDA_'): value
                for key, value in os.environ.items()
                if key.startswith('LAMBDA_')
            }
        },
        Publish=True
    )

`lambda deploy_s3` not working without changes to aws_lambda.py

When I tried to deploy through S3 using lambda deploy_s3 I encountered three issues.

  1. Setting role: basic_s3_upload in config.yaml did not work, since it then tried to give that role to the lambda function.

I solved this issue by removing the role definition in config.yaml, and then everything worked.

  1. The filename on S3 was not properly passed along to the update/create_function function

From what I can see the argument list for update/create_function (https://github.com/nficano/python-lambda/blob/master/aws_lambda/aws_lambda.py#L462) wants use_s3 to be a list of arguments and s3_file to be a dictionary of keyword arguments. These two values should be a boolean and a string, and when I changed the code to use them as such everything is working fine for me.

  1. Getting data from the event parameter in the handler function did not work

I had to change the code to something like this:

url = json.loads(event.get('body')).get('url')

Are the changes I had to do due to something strange on my system, or are these actually bugs?

I'm using Python 3.6, and have not tested this on any other Python versions.

Unable to import sub-directory module in AWS (but worked locally)

I extended my python code to include a main lambda handler file and a module in a subdirectory:

  • main lambda handler: service.py

  • newly created module: utils/slotutils.py where it contains the get_slot() and elicit_slot() functions.

  • In service.py, get_slot() and elicit_slot() were imported and referenced

  • I was able to run the code locally with the command "lambda invoke -v"

  • However, when the package was uploaded to AWS Lambda, executing the same python code resulted the following error message:
    "Unable to import module 'service': No module named utils.slotutils"

Is there an extra setting needed to tell AWS to search for my newly created sub-directory "utils"?
Note: when I moved slotutils.py one directory to co-locate with service.py, my AWS Lambda worked fine. Thank you so much for sharing this great tool.

Deploy only packages necessary for your code

Great project @nficano, thanks for making it.

I noticed that lambda deploy zips up and deploys all the packages in the virtualenv, including ones that are only needed to run python-lambda itself locally (e.g. boto, pip, pyaml, etc.) and therefore unneeded on AWS. It would be nice to be able to restrict this, maybe an additional config variable where you can indicate package names needed for your code?

Client Role Error

Hi,

After building a simple function and running lambda deploy I receive this error:

botocore.exceptions.ClientError: An error occurred (InvalidParameterValueException) when calling the CreateFunction operation: The role defined for the function cannot be assumed by Lambda.

Has anyone else had this issue? Is there a config file somewhere specifying the role that I can edit?

Lots of packages cannot be installed when using `lambda deploy`

Lots of packages cannot be installed when using lambda deploy

for example,
Installing screen-resolution-extra==0.0.0
Could not find a version that satisfies the requirement screen-resolution-extra==0.0.0 (from versions: )
No matching distribution found for screen-resolution-extra==0.0.0

README supplement

Thanks for this project and I followed the README very easily, but I think README needs some supplement for new users of AWS (like me).

While I was following that README, Before running of lambda deploy, I should create IAM users for getting aws_access_key_id and aws_secret_access_key. Nextly, I should create ~/.aws/credentials files for saving both key or just input to config.yaml (don't need to share). Finally, I should attach two policies to my IAM users, AWSLambdaFullAccess (for creating lambda function) and IAMFullAccess (perfoming iam:getUser). (I think Amazon provides more narrow polices)

Summary,
Before run lambda deploy

  1. Create IAM users
  2. Get aws_access_key_id and aws_secret_access_key
  3. Save above keys to ~/.aws/credentials
  4. Attach AWSLambdaFullAccess and IAMFullAccess
  5. Run lambda deploy

Regards

locally installed packages fail to deploy

I have a locally install package, built like so:

pip install . --upgrade

I check disk, and its there in the virtualenv.

However, when I run lambda build I get:

  Could not find a version that satisfies the requirement apt-commons==1.0 (from versions: )
No matching distribution found for apt-commons==1.0

I've tried the local-package argument, and it works, but then the module is not picked up in intellij, so, I'd prefer to just have it picked up from virtualenv.

deployment fails

gettiing below error while deploying the function

botocore.exceptions.ClientError: An error occurred (InvalidParameterValueException) when calling the CreateFunction operation: The role defi
ned for the function cannot be assumed by Lambda.

deployment fails after this message
Starting new HTTPS connection (1): lambda.us-east-1.amazonaws.com
Creating your new Lambda function
Starting new HTTPS connection (1): sts.amazonaws.com
Starting new HTTPS connection (1): lambda.us-east-1.amazonaws.com

importing boto3 fails (python3.6)

To reproduce just

import boto3

somewhere in the code, for example in service.py
and deploy for python3.6

Removing concurrent directory from the zip package solves the problem.

Exception details:

s3_client = boto3.client('s3')
File "/var/task/boto3/__init__.py", line 83, in client
return _get_default_session().client(*args, **kwargs)
File "/var/task/boto3/session.py", line 263, in client
aws_session_token=aws_session_token, config=config)
File "/var/task/botocore/session.py", line 836, in create_client
client_config=config, api_version=api_version)
File "/var/task/botocore/client.py", line 65, in create_client
cls = self._create_client_class(service_name, service_model)
File "/var/task/botocore/client.py", line 89, in _create_client_class
base_classes=bases)
File "/var/task/botocore/hooks.py", line 227, in emit
return self._emit(event_name, kwargs)
File "/var/task/botocore/hooks.py", line 210, in _emit
response = handler(**kwargs)
File "/var/task/boto3/utils.py", line 61, in _handler
module = import_module(module)
File "/var/task/boto3/utils.py", line 52, in import_module
__import__(name)
File "/var/task/boto3/s3/inject.py", line 15, in <module>
from boto3.s3.transfer import create_transfer_manager
File "/var/task/boto3/s3/transfer.py", line 127, in <module>
from s3transfer.exceptions import RetriesExceededError as \
File "/var/task/s3transfer/__init__.py", line 134, in <module>
import concurrent.futures
File "/var/task/concurrent/futures/__init__.py", line 8, in <module>
from concurrent.futures._base import (FIRST_COMPLETED,
File "/var/task/concurrent/futures/_base.py", line 357
raise type(self._exception), self._exception, self._traceback
^
SyntaxError: invalid syntax

S3 Support

I'm getting an error when trying to deploy:

botocore.exceptions.ClientError: An error occurred (RequestEntityTooLargeException) when calling the UpdateFunctionCode operation: Request must be smaller than 69905067 bytes for the UpdateFunctionCode operation

It could be avoided by uploading the lambda function code first to S3 and then deploying it to lambda from there.

Do you plan to support this?

Tnx

Error while deploying...

I'm constantly getting this error while trying to deploy to AWS..

21:00 $ lambda deploy --use-requirements
No dependency packages installed!
Bundling: 'event.json'
Bundling: 'service.py'
Creating your new Lambda function
Traceback (most recent call last):
  File "/Users/R.Thomas/.pyenv/versions/3.6.0/bin/lambda", line 62, in <module>
    cli()
  File "/Users/R.Thomas/.pyenv/versions/3.6.0/lib/python3.6/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/Users/R.Thomas/.pyenv/versions/3.6.0/lib/python3.6/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/Users/R.Thomas/.pyenv/versions/3.6.0/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/R.Thomas/.pyenv/versions/3.6.0/lib/python3.6/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/R.Thomas/.pyenv/versions/3.6.0/lib/python3.6/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/Users/R.Thomas/.pyenv/versions/3.6.0/bin/lambda", line 47, in deploy
    aws_lambda.deploy(CURRENT_DIR, use_requirements, local_package)
  File "/Users/R.Thomas/.pyenv/versions/3.6.0/lib/python3.6/site-packages/aws_lambda/aws_lambda.py", line 89, in deploy
    create_function(cfg, path_to_zip_file)
  File "/Users/R.Thomas/.pyenv/versions/3.6.0/lib/python3.6/site-packages/aws_lambda/aws_lambda.py", line 341, in create_function
    byte_stream = read(path_to_zip_file)
  File "/Users/R.Thomas/.pyenv/versions/3.6.0/lib/python3.6/site-packages/aws_lambda/helpers.py", line 15, in read
    return fh.read()
  File "/Users/R.Thomas/.pyenv/versions/3.6.0/lib/python3.6/codecs.py", line 321, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa8 in position 11: invalid start byte

including executable binaries

Lambda supports execution of binary programs within a lambda function, and python-lambda is a convenient way to package and upload these wrapped in a python interface. This almost works perfectly as is, except that python zip doesn't preserve executable bits when creating the package, and AWS requires these to be set or else the binary will fail with a permissions error.

I worked around this for my own use case by hacking helpers.py based on code in this stackexchange question, hardcoding special handling for my executable file. A more complete solution might be to check the executable flags on the files when packaging and preserve when building the zip archive, or alternatively allow the config file to specify a list of executables for special handling during packaging.

Thanks for the useful package!

Too many packages included on `lambda deploy`

When I do lambda deploy it does include these packages:

boto3 (1.4.4)
botocore (1.5.62)
click (6.6)
docutils (0.12)
futures (3.0.5)
jmespath (0.9.0)
pip (9.0.1)
pyaml (15.8.2)
python-dateutil (2.5.3)
python-lambda (2.2.0)
PyYAML (3.11)
s3transfer (0.1.11)
setuptools (38.2.3)
six (1.10.0)
wheel (0.30.0)

They are all NOT part of my project but parts of python-lambda.
They are therefore not needed on AWS Lambda to run my project.

How can I exclude these from deployment?

Provide option for multiple environments

I would like to deploy lambda function in multiple environments, like development, staging, and production.

Is there is a way to setup config.yaml for different working environments?

Importing LXML fails (Python 3.6)

Hi everyone,

I'm working on a lambda that will makes use of LXML. When running the lambda in AWS I noticed the following error message: Unable to import module 'service': cannot import name 'etree'. It seems this is because "LXML must be built with C extensions for libxml2 and libxslt in a way that plays well with the Amazon Lambda execution environment."

Has anyone else run into this issue?

source: https://www.azavea.com/blog/2016/06/27/using-python-lxml-amazon-lambda/

Ability to pass path (relative/absolute) of requirements file

My current usage of python-lambda uses a common requirements.txt file for all my lambda functions. Instead of placing the requirements.txt file in the function directory. Can I just add the ability to pass my own requirements.txt file instead of being forced to put the requirements.txt file in the function directory.

Click commands should use correct keyword args for requirements

The build command in scripts/lambda uses keyword arguments that are inconsistent with the build function in aws_lambda.py
def build(use_requirements, local_package, config_file): aws_lambda.build( CURRENT_DIR, use_requirements=use_requirements, local_package=local_package, config_file=config_file, )
aws_lambda build function signature looks like this:
def build( src, requirements=False, local_package=None, config_file='config.yaml'):

Import of local modules from the main module doesn't work when invoking

I have a lambda that imports a local module (i.e. another Python file in the same folder as the main lambda module, as opposed to a pip installed package). Doing a lambda deploy works, and the resulting lambda on AWS works also. However, when I do lambda invoke for local testing, I get an ImportError: No module named foo where foo is the module I'm trying to import.

Jenkins build error

Hi,

I want to use Jenkins for packaging our AWS Lambda project. I use ShiningPanda for managing virtual environment. When I execute lambda build command, it gives error. It creates a zip but it does not have some dependencies. Lines with + are my commands.

[TEST] $ /usr/bin/python2.7 /var/lib/jenkins/shiningpanda/jobs/033bd94b/virtualenv.py /var/lib/jenkins/shiningpanda/jobs/033bd94b/virtualenvs/945f3fc4
New python executable in /var/lib/jenkins/shiningpanda/jobs/033bd94b/virtualenvs/945f3fc4/bin/python2.7
Also creating executable in /var/lib/jenkins/shiningpanda/jobs/033bd94b/virtualenvs/945f3fc4/bin/python
Installing setuptools, pip, wheel...done.
[TEST] $ /bin/sh -xe /tmp/shiningpanda3458767265880341612.sh
+ pip install -r requirements.txt
Collecting boto3==1.3.1 (from -r requirements.txt (line 1))
  Using cached boto3-1.3.1-py2.py3-none-any.whl
Collecting botocore==1.4.32 (from -r requirements.txt (line 2))
  Using cached botocore-1.4.32-py2.py3-none-any.whl
Collecting click==6.6 (from -r requirements.txt (line 3))
Collecting docutils==0.12 (from -r requirements.txt (line 4))
Collecting futures==3.0.5 (from -r requirements.txt (line 5))
  Using cached futures-3.0.5-py2-none-any.whl
Collecting jmespath==0.9.0 (from -r requirements.txt (line 6))
  Using cached jmespath-0.9.0-py2.py3-none-any.whl
Collecting Pillow==3.3.1 (from -r requirements.txt (line 7))
  Using cached Pillow-3.3.1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pyaml==15.8.2 (from -r requirements.txt (line 8))
Collecting python-dateutil==2.5.3 (from -r requirements.txt (line 9))
  Using cached python_dateutil-2.5.3-py2.py3-none-any.whl
Collecting python-lambda==0.4.0 (from -r requirements.txt (line 10))
  Using cached python_lambda-0.4.0-py2.py3-none-any.whl
Collecting PyYAML==3.11 (from -r requirements.txt (line 11))
Collecting six==1.10.0 (from -r requirements.txt (line 12))
  Using cached six-1.10.0-py2.py3-none-any.whl
Installing collected packages: futures, jmespath, six, python-dateutil, docutils, botocore, boto3, click, Pillow, PyYAML, pyaml, python-lambda
Successfully installed Pillow-3.3.1 PyYAML-3.11 boto3-1.3.1 botocore-1.4.32 click-6.6 docutils-0.12 futures-3.0.5 jmespath-0.9.0 pyaml-15.8.2 python-dateutil-2.5.3 python-lambda-0.4.0 six-1.10.0
+ which python
/var/lib/jenkins/shiningpanda/jobs/033bd94b/virtualenvs/945f3fc4/bin/python
+ which pip
/var/lib/jenkins/shiningpanda/jobs/033bd94b/virtualenvs/945f3fc4/bin/pip
+ pip list
boto3 (1.3.1)
botocore (1.4.32)
click (6.6)
docutils (0.12)
futures (3.0.5)
jmespath (0.9.0)
Pillow (3.3.1)
pip (8.1.2)
pyaml (15.8.2)
python-dateutil (2.5.3)
python-lambda (0.4.0)
PyYAML (3.11)
setuptools (26.1.1)
six (1.10.0)
wheel (0.29.0)
+ lambda build
Exception:
Traceback (most recent call last):
  File "/var/lib/jenkins/shiningpanda/jobs/033bd94b/virtualenvs/945f3fc4/local/lib/python2.7/site-packages/pip/basecommand.py", line 215, in main
    status = self.run(options, args)
  File "/var/lib/jenkins/shiningpanda/jobs/033bd94b/virtualenvs/945f3fc4/local/lib/python2.7/site-packages/pip/commands/install.py", line 357, in run
    for item in os.listdir(lib_dir):
OSError: [Errno 2] No such file or directory: '/tmp/tmpxo3fWo/lib/python'
Exception:
Traceback (most recent call last):
  File "/var/lib/jenkins/shiningpanda/jobs/033bd94b/virtualenvs/945f3fc4/local/lib/python2.7/site-packages/pip/basecommand.py", line 215, in main
    status = self.run(options, args)
  File "/var/lib/jenkins/shiningpanda/jobs/033bd94b/virtualenvs/945f3fc4/local/lib/python2.7/site-packages/pip/commands/install.py", line 357, in run
    for item in os.listdir(lib_dir):
OSError: [Errno 2] No such file or directory: '/tmp/tmpMeHaOI/lib/python'
Gathering pip packages
Installing boto3==1.3.1
Installing botocore==1.4.32
Installing click==6.6
Installing docutils==0.12
Installing futures==3.0.5
Installing jmespath==0.9.0
Installing Pillow==3.3.1
Installing pip==8.1.2
Installing pyaml==15.8.2
Installing python-dateutil==2.5.3
Installing python-lambda==0.4.0
Installing PyYAML==3.11
Installing setuptools==26.1.1
Installing six==1.10.0
Installing wheel==0.29.0
Installing wsgiref==0.1.2

Not deploying to AWS

Nice Lib. So much simpler than Zappa. It's really what I need, but I am having hard times to deploy. It got stuck when calling lambda deploy.

Found credentials in shared credentials file: ~/.aws/credentials
Starting new HTTPS connection (1): lambda.us-west-2.amazonaws.com
Creating your new Lambda function
Starting new HTTPS connection (1): sts.amazonaws.com
Starting new HTTPS connection (1): lambda.us-west-2.amazonaws.com
Starting new HTTPS connection (2): lambda.us-west-2.amazonaws.com
Starting new HTTPS connection (3): lambda.us-west-2.amazonaws.com

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.