denoland / deno-lambda Goto Github PK
View Code? Open in Web Editor NEWA deno runtime for AWS Lambda. Deploy deno via docker, SAM, serverless, or bundle it yourself.
License: MIT License
A deno runtime for AWS Lambda. Deploy deno via docker, SAM, serverless, or bundle it yourself.
License: MIT License
In node console.log(XXX)
is prefixed with:
2020-01-17T04:36:04.408Z 330af918-dcb6-448d-945a-60e6c9c0a63d INFO XXX
should this be the case in deno-lambda?
Note:
My opinion is the timestamp is unnecessary and noisy:
`${REQUEST_ID}\tINFO\tXXX`
is better...
Run deno fmt --check
in CI to check that all files have proper formatting. Currently not all files are formatted the same.
Perhaps don't print this warning if the file endswith bundle.js
?
The published application at arn:aws:serverlessrepo:us-east-1:390065572566:applications~deno was uses a SemanticVersion
1.7.2
.
This is a little problematic with all the serverless framework, SAM, and others examples resources in uses 1.10.2
.
I imagine it would be ideal if the application was updated rather than the examples corrected? Good time to bump to 1.10.3 ;)
How will deno permission be set in serverless? Will it always default to allow all?
At the moment you need to do:
# Compile the handler (and fetch dependencies into DENO_DIR).
DENO_DIR=.deno_dir deno fetch hello.ts
# This is the "remapping" step:
cp -R .deno_dir/gen/file/$PWD/ .deno_dir/LAMBDA_TASK_ROOT
# Note: We do the inverse of this operation in bootstrap.
zip lambda.zip -x '.deno_dir/gen/file/*' -r .deno_dir hello.ts # other source files
I don't think this works in windows (if it does then this can be done with Deno.run
).
Either
zip
)Add this as https://deno.land/x/lambda/zip.ts
(or something). Potentially support .npmignore (since this is respected by SAM it makes sense to emulate that).
I have an otherwise working application that fails if I try to use an import map; specifically the kind of import maps generated by trex. I have created a minimal repro of the issue with this repository. Here is the working version without the import map.
The failure it produces is
2022-05-30T13:22:56.089-07:00 | error: Module not found "file:///var/task/tmp/runtime.js".
-- | --
| 2022-05-30T13:22:56.133-07:00 | error: deno exited
| 2022-05-30T13:22:56.219-07:00 | START RequestId: ee82b3a3-6dc4-4f7b-ae89-d7646649862d Version: $LATEST
| 2022-05-30T13:22:56.822-07:00 | error: Module not found "file:///var/task/tmp/runtime.js".
| 2022-05-30T13:22:57.303-07:00 | error: deno exited
| 2022-05-30T13:22:57.443-07:00 | END RequestId: ee82b3a3-6dc4-4f7b-ae89-d7646649862d
| 2022-05-30T13:22:57.443-07:00 | REPORT RequestId: ee82b3a3-6dc4-4f7b-ae89-d7646649862d Duration: 1222.96 ms Billed Duration: 1223 ms Memory Size: 128 MB Max Memory Used: 12 MB
| 2022-05-30T13:22:57.443-07:00 | Unknown application error occurred Unhandled
| 2022-05-30T13:22:57.904-07:00 | error: Module not found "file:///var/task/tmp/runtime.js".
| 2022-05-30T13:22:57.945-07:00 | error: deno exited
I'm bundling my lambda with the recommended instructions
# Compile the handler (and cache dependencies and compile artifacts into DENO_DIR).
DENO_DIR=.deno_dir deno cache --import-map=import_map.json src/lambda.ts
# This is the "remapping" step:
cp -R .deno_dir/gen/file/$PWD/ .deno_dir/LAMBDA_TASK_ROOT
# Note: We do the inversew of this operation in bootstrap.
# First zip deno deps up
zip build/lambda.zip -x '.deno_dir/gen/file/*' -r .deno_dir
One interesting thing to note is that deno cache
is producing files with double extensions
I've tried setting HANDLER_EXT = "ts.js
, but I still encounter the following errors
2022-05-23T20:30:23.956-07:00 | error: Unable to load 'file:///var/task/import_map.json' import map
-- | --
| 2022-05-23T20:30:23.956-07:00 | Caused by:
| 2022-05-23T20:30:23.956-07:00 | No such file or directory (os error 2)
| 2022-05-23T20:30:24.113-07:00Copyerror: missing expected handler file 'src/lambda.ts.js' | error: missing expected handler file 'src/lambda.ts.js'
The failure to transform DENO_IMPORTMAP = "./import_map.json"
the way $LAMBDA_TASK_ROOT/$HANDLER_FILE
is possibly a bug, but I don't think it is related.
Prevent #53.
Should test this env var is equal to docker run test-runner --entrypoint deno version
(?)
Edit: mostly there in b914943 just need to add file to docker. 🤞
I'm just trying to understand why we need to remap the DENO_DIR? I have read through DENO_DIR Remapping but don't understand why $ deno bundle
would not work? Does $ deno bundle
not include external modules?
Thanks in advance for clarifying.
When I run the hello-world sample laid out in the README I'm getting bundle error on aws/now.
Details in the now logs here: https://zeit.co/phil/deno-tmp/f65urnf02
I wonder, has something changed upstream in AWS? Can you guys deploy using this builder to now
currently, or may I'm doing something stupid.
Thanks - and THANKS for putting this builder together for deno! This is super helpful to see. 🍰
I have the following configuration in Terraform:
resource "aws_lambda_function" "this" {
# ...
image_config {
working_directory = "/var/task/foo/bar"
command = ["baz.handler"]
}
}
but the bootstrap script seems to be ignoring the working_directory
parameter and tries to import /var/task/baz.js
instead of /var/task/foo/bar/baz.js
- as a workaround I commented out the parameter and changed the command to ["foo/bar/baz.handler"]
You could do this on amazon linux docker:
Somehow use this for tests.
Edit: I have a first pass of this.
hey! wondering if you're planning on releasing updated bits to SAR? I'd rather re-use your work and get rid of mine!
These should match AWS Lambda uses for the function and the layers.
This will allow a more robust test for layer + function (to test the artifacts*).
Should we be using $SCRIPT_DIR rather than $LAMBDA_TASK_ROOT everywhere in bootstrap?
https://deno.land/x/lambda
in tests but fill that into DENO_DIR from runtime (we likely should do this in bootstrap too actually).Remaining from original issue but will handle separately.
Hi,
Confused about how to get data from event,
I invoked the function as below:
aws lambda invoke --endpoint http://localhost:8002 --no-sign-request --cli-binary-format raw-in-base64-out \
--function-name handler --payload '{"field1":11, "tableName":"t1"}' output.json
i can log that event as: { field1: 11, tableName: "t1" }
, which is what expected
but having question about how to get it out
As mentioned, event is APIGatewayProxyEvent type
, while the data passed-in from above invoke seems not set in APIGatewayProxyEvent type
, it means all values of fields from APIGatewayProxyEvent type
are empty
At the moment each line is a separate event for each line (this is handled by the callee of the custom runtime who is piping stdout to cloudwatch logs).
The objective is to be able to push a single event e.g. console.log(1, '\n', 2)
It is expandable and copy/pastable (where it wouldn't be if it was across multiple events).
Since console.log can be overloaded the naive solution is to push each console.log to CW however the problem is that this is rate-limited:
There is a quota of 5 requests per second per log stream. Additional requests are throttled. This quota can't be changed.
What you want to do is queue the log events up and PutLogEvents every second. The problem is that if you POST the queue after each event you may hit the rate-limit (since each event can be < 50ms) BUT if you don't then the process after each event they may never be POSTed (if the process is terminated) https://github.com/hayd/deno-lambda#warning
Ideally custom runtimes will have a local endpoint which manages such a CW logs batch queue... but nothing is documented as far as I can see.
OR it's even simpler and the newline can be encoded somehow (and still passed to stdout).
cc @brianleroux (perhaps you know / or know someone how has an answer for this)!
I'm trying to give this lambda runtime a go on Pulumi, which should theoretically be similar to AWS-CDK but I'm not sure they support all the constructs necessary.
I can write something like:
const denoRuntime = await aws.serverlessrepository.getApplication({
applicationId: "arn:aws:serverlessrepo:us-east-1:390065572566:applications/deno",
semanticVersion: "1.16.0",
});
To get the runtime application, but it's not clear if there's a way to actually create/clone this application and produce a layer ARN.
(Apologies for this pretty open support request, I'm kind of new to all of these tools)
https://aws.amazon.com/blogs/compute/migrating-aws-lambda-functions-to-al2/
Hopefully it's easy to migrate (changing provided
to provided.al2
, and then simplifying the docker image used for tests.
This would make updating deno-lambda much easier/faster.
End-of-life of standard support for Amazon Linux (AL1 for simplicity in this post) is coming in December 2020.
yikes!! Hopefully they mean end of december 😳
hi,
I tried to play this with local lambda
, while getting the below error:
111 #### PATH=/var/task/bin:/usr/local/bin:/usr/bin/:/bin:/opt/bin
222 #### pwd=/var/task
/var/task/bootstrap: line 238: deno: command not found
line 238 is below line, i tried to echo out the path and pwd
https://github.com/hayd/deno-lambda/blob/master/runtime/bootstrap#L236
reckon if it's related to lambda image ? because that deno
can not run in Mac as well
BTW, what is https://github.com/hayd/deno-lambda/releases/download/1.0.2/amz-deno.gz and what it's for?
First of all thank you so much for this toolkit, having a deno lambda layer is awesome! 🎉
This is a feature request for users that want to build their own layer. Our company has strict policies against using external layers, so we need to bake and create out own. The instructions in the README assume we are working from the released ZIP. It would be awesome if this was extended to show how to create the layer zip from source.
This would also enable users finer control over the produced layer, in case they wanted to make small source changes before producing the layer zip.
Currently stdout and stderr cannot be read since the bootstrap script is killed.
This will be easier if the http server were closed properly (rather than killed).
Edit: This should be feasible now we're using 0.24.0.
Deno issue: denoland/deno#3416.
Note: stdout DOES get posted to CW logs as it stands, this simply isn't tested in CI. For more advanced console.log handling see #40.
AWS recently announced support for Docker runtimes for Lambda. They have provided base images, one of which is the "provided" image for custom runtimes.
Fundamentally the runtime still works the same, it just supports a new upload format on top of the current zip one. All that would need to be done is adding a Dockerfile to the repo. The current runtime and layer could still be published and used the same as they currently are, but the Dockerfile could potentially make it easier to test, and by publishing an image to a public ECR repo it would give yet another option for deploying Deno applications to Lambda.
I'm unable to import the lambda module when using deno run
or deno cache
due to export declarations to unavailable sources. See https://github.com/hayd/deno-lambda/blob/853879fa2f45a4b4b75240ba12b9870bf2a3f413/runtime/types.d.ts#L1434-L1444
This seems to have been introduce in https://github.com/DefinitelyTyped/DefinitelyTyped/blob/f7ec78508c6797e42f87a4390735bc2c650a1bfd/types/aws-lambda/trigger/cognito-user-pool-trigger/index.d.ts#L109-L119
Looks like v1.9-1 is fine, and was problematic in v1.10.2
Here's the log
Download https://deno.land/x/lambda/mod.ts
Warning Implicitly using latest version (1.10.2) for https://deno.land/x/lambda/mod.ts
Download https://deno.land/x/[email protected]/mod.ts
Download https://deno.land/x/[email protected]/types.d.ts
Download https://deno.land/x/[email protected]/create-auth-challenge
Download https://deno.land/x/[email protected]/custom-message
Download https://deno.land/x/[email protected]/post-authentication
Download https://deno.land/x/[email protected]/verify-auth-challenge-response
Download https://deno.land/x/[email protected]/custom-email-sender
Download https://deno.land/x/[email protected]/pre-token-generation
Download https://deno.land/x/[email protected]/post-confirmation
Download https://deno.land/x/[email protected]/user-migration
Download https://deno.land/x/[email protected]/define-auth-challenge
Download https://deno.land/x/[email protected]/pre-authentication
Download https://deno.land/x/[email protected]/pre-signup
error: Import 'https://deno.land/x/[email protected]/create-auth-challenge' failed: 404 Not Found
at https://deno.land/x/[email protected]/types.d.ts:1434:0
Uncommenting this crashes the error test for some reason:
This could also be a legitimate bug in the runtime... Edit: it was not.
Previously
import type {
Context,
APIGatewayProxyEvent
} from "https://deno.land/x/lambda/mod.ts";
etc.
the deps and gen were correctly cached (from the layer and the function code), this seems to no longer be the case.
It's unclear to me if it's a zipping issue or due the filenames being renamed (now there's hashes in the name).
I'm not 100% sure this is an issue, but have a suspicion... needs investigate (and ideally a test).
Deno has a flag to run without type-checking, which is useful for lambdas that have already been type-checked. There are already several DENO
ENV VARs to configure behavior, a DENO_NOCHECK
or more flexible DENO_FLAGS
would be useful here.
Currently these point to [email protected] but should be removed and replaced with something else.
see tests/pad.ts, tests/importmap.json and tests/importmap.ts.
Perhaps delay from https://deno.land/std@VERSION/util/async.ts ?
I am not able to invoke lambda locally with Serverless framework with sls invoke local -f ...
All works well when I deploy my lambda, but when I try to run it locally I have the following error:
Failed to run docker for provided.al2 image. Error: Couldn't find valid bootstrap(s): [/var/task/bootstrap /opt/bootstrap]
If it is possible, could you provide some instructions how to do it.
Thanks
Hi! Thank you for your great work!
I'm trying to build sample lambda with docker and I have a problem with extra file.
I tested a really simple example and I faced with following error.
❯ docker run -p 9000:8080 --rm deno-lambda:test
# ... after invocation via curl
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
time="2021-04-20T12:10:30.99" level=info msg="exec '/var/runtime/bootstrap' (cwd=/var/task, handler=)"
time="2021-04-20T12:10:32.738" level=info msg="extensionsDisabledByLayer(/opt/disable-extensions-jwigqn8j) -> stat /opt/disable-extensions-jwigqn8j: no such file or directory"
time="2021-04-20T12:10:32.74" level=warning msg="Cannot list external agents" error="open /opt/extensions: no such file or directory"
START RequestId: 2038c5d5-5185-4660-a33e-1eea3f6947f0 Version: $LATEST
warn: unable to import '.deno_dir/' as DENO_DIR
at async file:///tmp/runtime.js:77:19h/26_fetch.js:844:19)
END RequestId: 2038c5d5-5185-4660-a33e-1eea3f6947f0
REPORT RequestId: 2038c5d5-5185-4660-a33e-1eea3f6947f0 Init Duration: 3.67 ms Duration: 774.07 ms Billed Duration: 800 ms Memory Size: 3008 MBMax Memory Used: 3008 MB
The project has just 3 files and each files are very simple.
hello.ts
import {
APIGatewayProxyEventV2,
APIGatewayProxyResultV2,
Context,
} from "https://deno.land/x/lambda/mod.ts";
import { test } from "./lib.ts";
// deno-lint-ignore require-await
export async function handler(
event: APIGatewayProxyEventV2,
context: Context,
): Promise<APIGatewayProxyResultV2> {
console.log(test());
return {
statusCode: 200,
headers: { "content-type": "text/html;charset=utf8" },
body: `Welcome to deno ${Deno.version.deno} 🦕`,
};
}
lib.ts
export const test = () => {
return "Fugafuga";
};
Dockerfile
FROM hayd/deno-lambda
COPY . .
RUN deno cache hello.ts
CMD ["hello.handler"]
Any suggestion for this?
I used current latest version 1.9
for base image
Thanks for the work on this project, I burned a lot of time trying to compile deno for an AWS environment. I'm hoping projects like this will gain traction for deno.
I have used the template examples to build a serverless application using SAM. I then use it to deploy the custom layers. I am now trying to deploy a serverless function that uses the custom runtime however the "provided" is throwing the following error:
$ sam build --template packaged.yaml --region us-east-1
Build Failed
Error: 'provided' runtime is not supported
Here is an issue tracking this
edit: layers
These need to be defined in bootstrap script.
Hayd,
I followed your directions on using SAR as well as the ZIP based deploy for the deno-lambda hello.ts.
Each time I have this error where:
import {
APIGatewayProxyEventV2,
APIGatewayProxyResultV2,
Context,
} from "../runtime/mod.ts";
Is not resolving & is not found. Am I doing something wrong? Is the SAR deploy method not current? I was able to work around it when I loaded them into a new directory I created (copy / paste workaround), but I'm curious as to why I wasn't able to do it as you prescribed. Just want to make sure I am understanding your flow.
Most of us heard about that poor guy running that .NET environment, non-optimized for lambda with (supposedly) bloated with propagating lambda self executions leading to bloated price for this architecture on aws lambda.
So my question, - is there some differences in initialize time, for the Deno environment, comparing to Node environment? Maybe even some overhead?
Using the example from the read me. The request spits out okay. I am really just trying to use severless_oak. I am going through trying to debug it. On Line 22, it uses event.path
.
So I decided to take the example from your readme and try to access the path like so:
import {
APIGatewayProxyEvent,
APIGatewayProxyResult,
Context,
} from "https://deno.land/x/lambda/mod.ts";
export async function handler(
event: APIGatewayProxyEvent,
context: Context,
): Promise<APIGatewayProxyResult> {
console.log(event);
console.log(event.path);
console.log(context);
return await {
body: `Welcome to deno ${Deno.version.deno} 🦕`,
headers: { "content-type": "text/html;charset=utf8" },
statusCode: 200,
};
}
As I said before, the request comes out fine, event looks like it has a path sitting in the http
object but typescript prevents me from accessing it and event.path
is logging undefined
in this example. Any ideas on how to remedy this? This is running aws lambda with amazon linux 2. I followed the SAR guide to deploy. I added an api gateway so i can hit it in my browser instead of using the test function.
Update:
I think I got it. APIGatewayProxyEventV2
is the event type I should be using. I can use rawPath
for fetching the path. Then i just configure to API Gateway with a /
path so I can hit the server with "XXXX.execute-api.us-east-2.amazonaws.com" and event.rawPath
will return /
.
Is this the proper way to do this or should I handle this another way? First time deploying a lambda function manually. Usually use serverless with nextjs.
The boostrap investigate
function seems to handle pre-handler error's badly. I had uncaught exception during setup, and this showed up in the logs afterwards.
2020-11-17T13:27:33.673-08:00Copy�[0m�[1m�[31merror�[0m: Could not create TypeScript compiler cache location: "/home/sbx_user1051/.cache/deno/gen" | �[0m�[1m�[31merror�[0m: Could not create TypeScript compiler cache location: "/home/sbx_user1051/.cache/deno/gen"
-- | --
| 2020-11-17T13:27:33.673-08:00 | Check the permission of the directory.
| 2020-11-17T13:27:33.674-08:00 | error: bad deno executable
After fixing the error the lambda ran fine, so I don't think the deno executable is actually bad.
Howdy 👋 Just wondering if you had benchmarks for Deno performance on Lambda squirreled away you could share. Trying to convince a team to give this a shot and benchmarks was the first thing they asked for.
It's kinda hidden here: https://github.com/hayd/deno-lambda/blob/master/README.md#deno_dir-remapping
The reason I ported deno-zip was to do this remapping in deno, so I should write a a ts that creates such a zip.
Requires:
anything else?
deno
is in PATH. #45I'm unable to fix, though I could release as 0.34.1...
Edit: I should get my eyes tested, but the joke is I had just had my eyes tested and my pupils were still dilated 😆 (whoops), apologies for not checking the diff carefully enough.
I've been trying to get puppeteer running in this Dockerfile.
FROM hayd/deno-lambda:1.13.2
ENV PUPPETEER_PRODUCT=chrome
RUN deno run -A --unstable https://deno.land/x/[email protected]/install.ts
COPY hello.ts .
RUN deno cache --unstable hello.ts
CMD ["hello.handler"]
and hello.ts:
import {
APIGatewayProxyEventV2,
APIGatewayProxyResultV2,
Context,
} from "https://deno.land/x/lambda/mod.ts";
import puppeteer from "https://deno.land/x/[email protected]/mod.ts";
export async function handler(
event: APIGatewayProxyEventV2,
context: Context,
): Promise<APIGatewayProxyResultV2> {
console.log('start');
const browser = await puppeteer.launch({
headless: true,
args: [
"--no-sandbox",
"--disable-dev-shm-usage",
]
});
const page = (await browser.pages())[0];
await browser.close();
console.log('finish');
return {
statusCode: 200,
headers: { "content-type": "text/html;charset=utf8" },
body: `Welcome to deno ${Deno.version.deno} 🦕`,
};
}
Gets:
Error: Could not find browser revision 869685. Run "PUPPETEER_PRODUCT=chrome deno run -A --unstable https://deno.land/x/[email protected]/install.ts" to download a supported browser binary.
It seems that the RUN deno run -A --unstable https://deno.land/x/[email protected]/install.ts
doesn't persist chrome into the image, as RUN chrome
after it doesn't work.
I've also tried to install chrome by RUN curl https://dl.google.com/linux/direct/google-chrome-stable_current_x86_64.rpm -o ./google-chrome-stable_current_x86_64.rpm && yum install ./google-chrome-stable_current_x86_64.rpm -y && ln -s /usr/bin/google-chrome-stable /usr/bin/chrome
where RUN chrome
now works but still get the same error.
When using serverless to deploy bundled file dist.js
, it was reporting that
START RequestId: 39dff4dd-6def-48d7-9830-52d5d8406efc Version: $LATEST
warn: unable to import '.deno_dir/' as DENO_DIR
error: Cannot resolve module "file:///var/task/dist.ts" from "file:///tmp/runtime.js"
�[0m�[1m�[31merror�[0m: Could not create TypeScript compiler cache location: "/home/sbx_user1051/.cache/deno/gen"
Check the permission of the directory.
error: bad deno executable
END RequestId: 39dff4dd-6def-48d7-9830-52d5d8406efc
REPORT RequestId: 39dff4dd-6def-48d7-9830-52d5d8406efc Duration: 164.47 ms Billed Duration: 200 ms Memory Size: 1024 MB Max Memory Used: 11 MB
Unknown application error occurred
Unhandled
It looks like the module is assuming .ts files by default
This doesn't work in 1.0.0-rc2,atm it's tested on 0.41.0 only, requires dynamodb to be updated.
Pagic is a static site generator powered by Deno + React
Here are some demos:
If it is possible to use pagic to build the website, I would like to create the pull-request
The dependencies may need to be bumped...
deno-lambda might skip deno version 0.28.0 due to openssl dependency in that version.
This can probably be worked around and happy to take a workaround over at denoland/deno_docker#12 but this is likely to be fixed by denoland/deno#3586 in 0.29.0 (which may include rusty_v8 and hence necessitate a dockerfile rewrite anyway).
Apologies!
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.