rails-lambda / crypteia Goto Github PK

We are using this simple profile release from this repo (https://rustrepo.com/repo/johnthagen-min-sized-rust) which gives us a lot of savings. The binary is ~10MB and the shared object is ~300KB. This is not bad for Lambda Containers and we would rather optimize for speed.

[profile.release]
strip = true

That said, if the topic ever comes up and folks think we could do more while still being fast, please let me know what from here (https://rustrepo.com/repo/johnthagen-min-sized-rust) or elsewhere we should consider.

I have witnessed shutdown errors that does not cause any issues. It could be a Ruby Runtime Interface Client (https://github.com/aws/aws-lambda-ruby-runtime-interface-client) bug only. Maybe we need some invoke hook to make sure SSM finished work? Will investigate later.

Is it possible to load secrets from secrets manager via the special aws/reference path?

I would love to see loading secrets from secrets manager too

Might have to change our Dockerfile around a bit if not Tier 1. More details here. https://doc.rust-lang.org/nightly/rustc/platform-support.html

Running the setup command for the first time results in the following error:

error[E0463]: can't find crate for `core`
  |
  = note: the `x86_64-unknown-linux-gnu` target may not be installed
  = help: consider downloading the target with `rustup target add x86_64-unknown-linux-gnu`

error[E0463]: can't find crate for `compiler_builtins`

For more information about this error, try `rustc --explain E0463`.
error: could not compile `cfg-if` due to 2 previous errors
warning: build failed, waiting for other jobs to finish...
error[E0463]: can't find crate for `core`
    --> /Users/thomasstark/.cargo/registry/src/github.com-1ecc6299db9ec823/pin-project-lite-0.2.9/src/lib.rs:1503:9

I'm assuming this can easily be resolved with a few additional package installs, but I'm leaving this here in case someone wants to smooth out the setup.

Thanks to @cameronlynn for pointing this out. It is possible to use two paths where a value like SECRET would be in both.

X_CRYPTEIA_SSM1: x-crypteia-ssm-path:/myapp/1/env
X_CRYPTEIA_SSM2: x-crypteia-ssm-path:/myapp/2/env
SECRET: x-crypteia

In this case, there is no deterministic behavior on what the value of SECRET would be. Should we handle this? Document it? Etc?

Use this env var AWS_LAMBDA_RUNTIME_API to avoid to avoid running this https://github.com/customink/crypteia/blob/main/src/main.rs#L18 when not present. Thanks @tdstark

Expressed here, good answers in the replies too. devcontainers/ci#173

Right now the proof of concept will work as long as an existing ENV is set. For example, if HELLO=yall is set then the shared object hook into getenv can override that with WORLD as a value. But if HELLO were not set at all, then the current code will not return the override. Why is this important?

Let's say you want to batch fetch many parameters using ssm_parameters:/my/application/env as a path. You could have something like this in your template.yml file:

Globals:
  Function:
    Environment:
      Variables:
        X_CRYPTEIA_ENVIRONMENTS: ssm_parameters:/my/application/env

So this could pull in DATABASE_URL, SECRET_KEY_BASE, and many more. But would fail because there was no previous DATABASE_URL set. We could treat this as a feature and encourage placeholders like so, but that feels a bit superfluous?

Globals:
  Function:
    Environment:
      Variables:
        DATABASE_URL: placeholder
        SECRET_KEY_BASE: placeholder
        X_CRYPTEIA_ENVIRONMENTS: ssm_parameters:/my/application/env

Install node on the dev container and leverage it in the same libcrypteia test scripts using an argument on which binary (ruby or node, etc) to test with.

Should be a simple change to our workflow for publishing packages.

Hi,

First of all thank you very much for the work put into this, it was a crucial component of moving some of my workloads into lambda, and it works like a charm!
I've recently started experiencing intermittent lambda executions where my crypteia env variables don't get swapped out for the SSM secrets. I haven't dug into the root cause yet, but I assume something is failing when fetching the secrets from SSM, so I was wondering if there is some agreed upon way to handle these scenarios.
My use case is running lamby rails jobs that get triggered via an sqs queue (via lambdakiq). Sometimes the jobs will execute and I'll get errors because I'll just have x-crypteia-ssm:.. in my envs instead of the secrets. To make matters worse, I'm pretty sure that when facing high loads lambda copies over my jobs and I get more failures because they don't seem to startup and try to load the vars again.
I'm wondering if there's some way I can just kill the execution of the container straight away, and then my queue retry mechanisms will kick in and execute the job again?
Any help here is greatly appreciated, and apologies if this isn't the right place to post this question.

Thanks!

Test when someone does something like /myapp/envs/some/garbage/SECRET or something like that.

Expand on the https://github.com/customink/crypteia#development section with a new script and docs to show people how to publish a Lambda Layer to their own account.

Investigate if our installation guides will clobber an existing opt directory or merge things in. If it does clobber, account for this in a major version change and new guides.

The library here will be a simple .so file using redhook (https://crates.io/crates/redhook) with LD_PRELOAD like we saw done with https://github.com/mmanciop/opentelemetry-injector but now the question is how do we know when the extension's SSM parameter fetch is done so we can share the response data for the replacement ENV variables? Research list:

• https://crates.io/crates/inotify
• https://crates.io/crates/notify
• https://crates.io/crates/ipmpsc
• https://crates.io/crates/interprocess
• https://crates.io/crates/shared_memory
• https://crates.io/crates/signal-hook
• https://crates.io/crates/futures-signals
• https://crates.io/crates/signalbool
• https://github.com/servo/ipc-channel
• https://crates.io/crates/nix
• https://crates.io/crates/unix-named-pipe

Thoughts on changing the way this project works by updated each matching ENV variable with the returned value. So in stead of this:

echo $FOO_PARAM
ssm_parameter:/my/parameter

We would have this. And every runtime can simply use the environment vs constructing a fetch to localhost.

echo $FOO_PARAM
my-parameter

A friend shared this project as an idea on how to tap into LD_PRELOAD to update environments before the "real" lambda function/application starts. Thoughts?

https://github.com/mmanciop/opentelemetry-injector

The redhook crate will output the following on a base NodeJS image.

/lib64/libc.so.6: version `GLIBC_2.28' not found (required by /opt/lib/libcrypteia.so)

This is what I see on ruby:2.7, nodejs:14, and nodes:16. This version was released on 2017-08-02 :(

bash-4.2# find / -name "*glibc*"
/usr/sbin/glibc_post_upgrade.x86_64
/usr/share/licenses/glibc-2.26
/usr/share/doc/glibc-common-2.26
/usr/share/doc/glibc-2.26

So I tried learning if I could find a package (https://gist.github.com/metaskills/14bf3ddb65f7cc1e6156ee1609e64cb3) or build glibc from source (https://www.gnu.org/software/libc/), which both were failures.

Hi, I spent the afternoon trying to build this and I only managed to build this using cargo-lambda - because it is using zig build https://www.cargo-lambda.info/commands/build.html#compiler-backends

In Codespace and after modification of .devcontainer I've got an error + warnings in the newer rust

warning: flag `-Z sparse-registry` has been stabilized in the 1.68 release, and is no longer necessary
  The sparse protocol is now the default for crates.io

   Compiling autocfg v1.1.0
   Compiling proc-macro2 v1.0.50
   Compiling quote v1.0.23
   Compiling unicode-ident v1.0.6
   Compiling syn v1.0.107
error[E0635]: unknown feature `proc_macro_span_shrink`
  --> /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/proc-macro2-1.0.50/src/lib.rs:92:30
   |
92 |     feature(proc_macro_span, proc_macro_span_shrink)
   |                              ^^^^^^^^^^^^^^^^^^^^^^

   Compiling libc v0.2.139
For more information about this error, try `rustc --explain E0635`.
error: could not compile `proc-macro2` (lib) due to previous error
warning: build failed, waiting for other jobs to finish...

If you were to print out the full ENV object in Ruby it would not show the resolved values because we patch into getenv only which comes into play when you use ENV#[]. The same might be true in other languages.

Seems AWS is having a SSM issue in us-east-1 today. Perhaps we can have a CRYPTEIA_AWS_REGION config here https://github.com/customink/crypteia/blob/main/src/ssm.rs#L7 to force another region in case it is needed?