navarasu / serverless-ruby-layer Goto Github PK

A Serverless Plugin to deploy gems from Gemfile to AWS Layer

License: MIT License

JavaScript 100.00%

serverless serverless-plugin aws-lambda aws-lambda-ruby aws aws-layers aws-lambda-layer ruby bundler gemfile

serverless-ruby-layer's Introduction

A Serverless Plugin which bundles ruby gems from Gemfile and deploys them to the lambda layer automatically while running serverless deploy.

It auto-configures the AWS lambda layer and RUBY_PATH to all the functions.

Install

Install ⚡️ serverless. Refer here for serverless installation instructions.

Navigate to your serverless project and install the plugin

sls plugin install -n serverless-ruby-layer

This will add the plugin to package.json and the plugins section of serverless.yml.

Documentation

Check out the documentation here for,

Getting Started

Simple Usage

serverless.yml

service: basic

plugins:
  - serverless-ruby-layer

provider:
  name: aws
  runtime: ruby3.2

functions:
  hello:
    handler: handler.hello

Gemfile

  source 'https://rubygems.org'
  gem 'httparty'

Running sls deploy automatically deploys the required gems as in Gemfile to AWS lambda layer and make the gems available to the RUBY_PATH of the functions hello.handler

Refer example amd docs for more details

Customization

The plugin operation can be customized by specifying the custom configuration under rubyLayer.

For example, to use docker environment for packing gem, the below configuration is added to serverless.yml

custom:
  rubyLayer:
    use_docker: true

For more detailse refer the docs here

Usage

Using the custom configuration, the plugin can be utilized for below cases,

Using locally installed bundler for gems without any native extensions - Example - Docs
Using Docker for gems with OS native C extensions or system libraries like http, Nokogiri - Example - Docs
Preinstall OS packages (yum packages) for gems which requires OS native system libraries like pg, mysql, RMagick - PG Example - Docs
Using Dockerfile for gems which with other OS Linux image or system libraries and utilities - Example - Docs
Using Docker / Dockerfile with environment variable - Docker Example - DockerFile Example - Docs
Include / Exclude specific functions from layer configuration - Include Example , Exclude Example - Docs
Exclude test and development related gems from layer - Example - Docs
Using Bundler.require(:default) to require all gems in handler.rb by respecting Gemfile.lock - Example - Docs

Contributing

Pull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.

Please make sure to update the tests as appropriate.

Refer Guidelines for more information.

License

MIT

serverless-ruby-layer's People

Contributors

Stargazers

Watchers

Forkers

dengel29 njoerd114 jagthedrummer travisdahlke zaguiini agoratix thienhv scientist-softserv lukasz-horonziak joelcogen osdakira easantillan

serverless-ruby-layer's Issues

Docker command not found when run on Github Actions

Hi.

When i try to deploy using Github Actions, the docker command does't work inside serverless-ruby-layer container: "docker command not found."

Local works fine.

Error Stack Trace

Run serverless/github-action@master
/usr/bin/docker run --name a7afdfa700d0a34bd0a58de72731e56b87_4abc12 --label 9916a7 --workdir /github/workspace --rm -e AWS_ACCESS_KEY_ID -e AWS_SECRET_ACCESS_KEY -e SLS_DEBUG -e INPUT_ARGS -e INPUT_ENTRYPOINT -e HOME -e GITHUB_JOB -e GITHUB_REF -e GITHUB_SHA -e GITHUB_REPOSITORY -e GITHUB_REPOSITORY_OWNER -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_RETENTION_DAYS -e GITHUB_RUN_ATTEMPT -e GITHUB_ACTOR -e GITHUB_WORKFLOW -e GITHUB_HEAD_REF -e GITHUB_BASE_REF -e GITHUB_EVENT_NAME -e GITHUB_SERVER_URL -e GITHUB_API_URL -e GITHUB_GRAPHQL_URL -e GITHUB_REF_NAME -e GITHUB_REF_PROTECTED -e GITHUB_REF_TYPE -e GITHUB_WORKSPACE -e GITHUB_ACTION -e GITHUB_EVENT_PATH -e GITHUB_ACTION_REPOSITORY -e GITHUB_ACTION_REF -e GITHUB_PATH -e GITHUB_ENV -e RUNNER_OS -e RUNNER_ARCH -e RUNNER_NAME -e RUNNER_TOOL_CACHE -e RUNNER_TEMP -e RUNNER_WORKSPACE -e ACTIONS_RUNTIME_URL -e ACTIONS_RUNTIME_TOKEN -e ACTIONS_CACHE_URL -e GITHUB_ACTIONS=true -e CI=true --entrypoint "/bin/sh" -v "/var/run/docker.sock":"/var/run/docker.sock" -v "/home/runner/work/_temp/_github_home":"/github/home" -v "/home/runner/work/_temp/_github_workflow":"/github/workflow" -v "/home/runner/work/_temp/_runner_file_commands":"/github/file_commands" -v "/home/runner/work/ruby-serverless/ruby-serverless":"/github/workspace" 9916a7:afdfa700d0a34bd0a58de72731e56b87 -c "sls plugin install -n serverless-ruby-layer && sls deploy"
Serverless: Installing plugin "serverless-ruby-layer@latest" (this might take a few seconds...)
Serverless: Successfully installed "serverless-ruby-layer@latest"
Serverless: Load command interactiveCli
Serverless: Load command config
Serverless: Load command config:credentials
Serverless: Load command config:tabcompletion
Serverless: Load command config:tabcompletion:install
Serverless: Load command config:tabcompletion:uninstall
Serverless: Load command create
Serverless: Load command install
Serverless: Load command package
Serverless: Load command deploy
Serverless: Load command deploy:function
Serverless: Load command deploy:list
Serverless: Load command deploy:list:functions
Serverless: Load command invoke
Serverless: Load command invoke:local
Serverless: Load command info
Serverless: Load command logs
Serverless: Load command metrics
Serverless: Load command print
Serverless: Load command remove
Serverless: Load command rollback
Serverless: Load command rollback:function
Serverless: Load command slstats
Serverless: Load command plugin
Serverless: Load command plugin
Serverless: Load command plugin:install
Serverless: Load command plugin
Serverless: Load command plugin:uninstall
Serverless: Load command plugin
Serverless: Load command plugin:list
Serverless: Load command plugin
Serverless: Load command plugin:search
Serverless: Load command config
Serverless: Load command config:credentials
Serverless: Load command upgrade
Serverless: Load command uninstall
Serverless: Load command rubylayer
Serverless: Load command login
Serverless: Load command logout
Serverless: Load command generate-event
Serverless: Load command test
Serverless: Load command dashboard
Serverless: Load command output
Serverless: Load command output:get
Serverless: Load command output:list
Serverless: Load command param
Serverless: Load command param:get
Serverless: Load command param:list
Serverless: Load command studio
Serverless: Skipping variables resolution with old resolver (new resolver reported no more variables to resolve)
Serverless: Invoke deploy
Serverless: Invoke package
Serverless: Invoke aws:common:validate
Serverless: Invoke aws:common:cleanupTempDir
Serverless: Packaging service...
Serverless: Excluding development dependencies...
Serverless: [AWS sts 200 0.149s 0 retries] getCallerIdentity({})
Serverless: Clearing previous build ruby layer build
 
 Error ---------------------------------------------------
 
  Error: docker command not found. Please install docker https://www.docker.com/products/docker-desktop
      at ServerlessRubyBundler.bundleInstall (/github/workspace/node_modules/serverless-ruby-layer/lib/bundle.js:61:13)
      at ServerlessRubyBundler.tryCatcher (/github/workspace/node_modules/bluebird/js/release/util.js:16:23)
      at Promise._settlePromiseFromHandler (/github/workspace/node_modules/bluebird/js/release/promise.js:547:31)
      at Promise._settlePromise (/github/workspace/node_modules/bluebird/js/release/promise.js:604:18)
      at Promise._settlePromise0 (/github/workspace/node_modules/bluebird/js/release/promise.js:649:10)
      at Promise._settlePromises (/github/workspace/node_modules/bluebird/js/release/promise.js:729:18)
      at _drainQueueStep (/github/workspace/node_modules/bluebird/js/release/async.js:93:12)
      at _drainQueue (/github/workspace/node_modules/bluebird/js/release/async.js:86:9)
      at Async._drainQueues (/github/workspace/node_modules/bluebird/js/release/async.js:102:5)
      at Immediate.Async.drainQueues [as _onImmediate] (/github/workspace/node_modules/bluebird/js/release/async.js:15:14)
      at processImmediate (internal/timers.js:464:21)
 
  Get Support --------------------------------------------
     Docs:          docs.serverless.com
     Bugs:          github.com/serverless/serverless/issues
     Issues:        forum.serverless.com
 
  Your Environment Information ---------------------------
     Operating System:          linux
     Node Version:              14.18.3
     Framework Version:         2.71.0
     Plugin Version:            5.5.3
     SDK Version:               4.3.0
     Components Version:        3.18.1

Config files

serverless.yml


service: ruby-serverless

frameworkVersion: '2'

plugins:
  - serverless-ruby-layer
custom:
  rubyLayer:
    use_docker: true
    docker_yums:
      - postgresql-devel
    native_libs:
      - /usr/lib64/libpq.so.5
      - /usr/lib64/libldap_r-2.4.so.2
      - /usr/lib64/liblber-2.4.so.2
      - /usr/lib64/libsasl2.so.3
      - /usr/lib64/libssl3.so
      - /usr/lib64/libsmime3.so
      - /usr/lib64/libnss3.so
      
provider:
  name: aws
  runtime: ruby2.7
  lambdaHashingVersion: 20201221
  stage: dev
  region: us-east-1

package:
 patterns:
   - '!node_modules'

functions:
  hello:
    handler: handler.hello

Gemfile

source "https://rubygems.org"

git_source(:github) {|repo_name| "https://github.com/#{repo_name}" }

gem "pg"

deploy.yml

name: Deploy SLS

on:
  push:
jobs:
  build:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v2
      - name: serverless deploy
        uses: serverless/github-action@master
        with:
          args: -c "sls plugin install -n serverless-ruby-layer && sls deploy"
          entrypoint: /bin/sh
        env:
          AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
          AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          SLS_DEBUG: '*'

Desktop:

OS: Mac OS
NodeVersion: 16.13.1
Ruby version: 2.7.0
Bundler version: 2.1.2

Added test case for the pending case to improve the test coverage

Automatically exclude node_modules, Gemfile, vendor folder from function zip

Release 1.6.0 Milestone

Any chance for a 1.5.1 release to pick up some of the newest updates?

Building via docker on GitHub actions?

Currently when I try to run sls deploy on github actions it fails with:

Serverless: Zipping the gemfiles and native libs to /home/runner/work/repo/repo/.serverless/ruby_layer/gemLayer.zip
 
 Exception -----------------------------------------------
 
  [OperationalError: EACCES: permission denied, scandir '/home/runner/work/repo/repo/.serverless/ruby_layer/build/ruby/2.7.0/gems/pg-1.2.3/ext/.gem.20210317-1-q5o5ls'] {
    cause: [Error: EACCES: permission denied, scandir '/home/runner/work/repo/repo/.serverless/ruby_layer/build/ruby/2.7.0/gems/pg-1.2.3/ext/.gem.20210317-1-q5o5ls'] {
      errno: -13,
      code: 'EACCES',
      syscall: 'scandir',
      path: '/home/runner/work/repo/repo/.serverless/ruby_layer/build/ruby/2.7.0/gems/pg-1.2.3/ext/.gem.20210317-1-q5o5ls'
    },
    isOperational: true,
    errno: -13,
    code: 'EACCES',
    syscall: 'scandir',
    path: '/home/runner/work/repo/repo/.serverless/ruby_layer/build/ruby/2.7.0/gems/pg-1.2.3/ext/.gem.20210317-1-q5o5ls'
  }

My serverless config contains:

custom:
  rubyLayer:
    use_docker: true
    docker_file: Dockerfile
    native_libs:
      - /usr/lib64/libpq.so.5
      - /usr/lib64/libldap_r-2.4.so.2
      - /usr/lib64/liblber-2.4.so.2
      - /usr/lib64/libsasl2.so.3
      - /usr/lib64/libssl3.so
      - /usr/lib64/libsmime3.so
      - /usr/lib64/libnss3.so

OS: ubuntu-latest
NodeVersion 15.10.0
Ruby version 2.7.1
Bundler version 2.2.6

Added coverage report using coverall and istanbul

Clean up and test env pass through & build args with single parameter

Use generic name environment replacing docker_build_args and docker_env_variable

docker_env_variable and docker_build_args will use same environment option

custom:
  rubyLayer:
    use_docker: true
    environment:
      -ENVIRONMENT_VARIABLE
      -ENVIRONMENT_VARIABLE=env_value

Ability to pass build args to a Docker build

I'd like to use a gem which isn't a publicly available gem, but is installed via Gemfury. To make that happen I can use an environment variable that makes things magically work when I'm doing a local build. A Docker build doesn't have access to that ENV var, and the way to pass things like this into docker is with the --build-arg flag. I can make a custom Dockerfile but don't have a way to populate a build arg.

In the Dockerfile I'd like to do this:

ARG BUNDLE_GEM__FURY__IO
ENV BUNDLE_GEM__FURY__IO=${BUNDLE_GEM__FURY__IO}

It would be nice if I could specify a build arg to pass in the rubyLayer section of serverless.yml.

Something like this would work:

rubyLayer:
  dockerBuildArgs:
    ENV BUNDLE_GEM__FURY__IO: ${env:BUNDLE_GEM__FURY__IO}

Or it would be even smoother if I didn't need to have a custom Dockerfile at all and could just specify that I'd like that ENV var to be passed from my local env into the default Docker build context.

rubyLayer:
  dockerEnvironmentVariablesToPassFromLocalEnvIntoDocker:
    - ENV BUNDLE_GEM__FURY__IO

Trouble deploying `pg` due to `pg_ext.so`

Describe the bug

I'm trying to follow the example for deploying the pg gem, but I'm not having much luck. I have the gem in my Gemfile and I've activated Docker and have included libpq.so.5 as shown in the example.

When I require 'pg' I get this error:

libldap_r-2.4.so.2: cannot open shared object file: No such file or directory - /opt/ruby/2.7.0/gems/pg-1.2.3/lib/pg_ext.so

In trying to debug the problem I've added some code to show the files in /opt/ruby/2.7.0/gems/pg-1.2.3/lib and pg_ext.so does seem to be there.

Files in /opt/ruby/2.7.0/gems/pg-1.2.3/lib
/opt/ruby/2.7.0/gems/pg-1.2.3/lib/pg
/opt/ruby/2.7.0/gems/pg-1.2.3/lib/pg.rb
/opt/ruby/2.7.0/gems/pg-1.2.3/lib/pg_ext.so

I've also noticed that there seem to be three copies of the file in gemLayer.zip. (Not sure if that matters.)

$ unzip -l .serverless/ruby_layer/gemLayer.zip  | grep pg_ext
   567056  04-01-2021 02:53   ruby/2.7.0/gems/pg-1.2.3/ext/pg_ext.so
   567056  04-01-2021 02:53   ruby/2.7.0/gems/pg-1.2.3/lib/pg_ext.so
   567056  04-01-2021 02:53   ruby/2.7.0/extensions/x86_64-linux/2.7.0/pg-1.2.3/pg_ext.so

Error Stack Trace

Error is just at runtime, not deploy time.

Example Config files

Gemfile:

source 'https://rubygems.org'
git_source(:github) { |repo| "https://github.com/#{repo}.git" }

ruby '2.7.2'

gem 'aws-sdk-lambda', '~> 1.62'
gem 'aws-sdk-sqs', '~> 1.37'
gem 'aws-sdk-dynamodb', '~> 1.60'
gem 'aws-sdk-ssm', '~> 1.107'

gem 'pg', '~> 1.1'

gem 'activerecord', '~> 6.1', '>= 6.1.3.1'

gem 'pusher'

serverless.yml:

...
custom:
  rubyLayer:
    use_docker: true
    docker_yums:
      - postgresql-devel
      - postgresql-libs
    native_libs:
      - /usr/lib64/libpq.so.5
...

Desktop (please complete the following information):

OS: Mac OS
NodeVersion: 15.11.0
Ruby version 2.7.2
Bundler version 2.1.4

Additional context

None that seems relevant. Happy to provide additional info if there are specifics anyone would like to ask for.

Error while adding local path in Gemfile

Describe the bug

I added to my Gemfile a local gem using the standard format

gem 'test_gem', path: 'vendor/gems/test_gem'

However, when deploy it gives me the following error

Error Stack Trace

Error:
Error: The path
/<my-path>/.serverless/ruby_layer/vendor/gems/test_gem
does not exist.

Example Config files
This is my Gemfile:

source "https://rubygems.org"

gem 'rails'
gem 'json'
gem 'test_gem', path: 'vendor/gems/test_gem'

Desktop (please complete the following information):

OS: Ubuntu 20.04
NodeVersion 14.17.0
Ruby version 2.7.5
Bundler version 2.2.5

Add basic test case and configure circle ci

arm64 support

support for building with arm64 images would be good

https://gallery.ecr.aws/sam/build-ruby3.2

The `use_docker` example fails on nokogiri install

Describe the bug
I'm using the example listed under the example/use_docker directory. I'm using ruby version 2.5.8 for executing this code. However, every time I run sls package in the example directory, I get an error with nokogiri. Any help or guidance in this would be awesome.

Error Stack Trace
Running SLS_DEBUG=* sls package:

➜  use_docker git:(master) ✗ SLS_DEBUG=* sls package
Serverless: Load command interactiveCli
Serverless: Load command config
Serverless: Load command config:credentials
Serverless: Load command config:tabcompletion
Serverless: Load command config:tabcompletion:install
Serverless: Load command config:tabcompletion:uninstall
Serverless: Load command create
Serverless: Load command install
Serverless: Load command package
Serverless: Load command deploy
Serverless: Load command deploy:function
Serverless: Load command deploy:list
Serverless: Load command deploy:list:functions
Serverless: Load command invoke
Serverless: Load command invoke:local
Serverless: Load command info
Serverless: Load command logs
Serverless: Load command metrics
Serverless: Load command print
Serverless: Load command remove
Serverless: Load command rollback
Serverless: Load command rollback:function
Serverless: Load command slstats
Serverless: Load command plugin
Serverless: Load command plugin
Serverless: Load command plugin:install
Serverless: Load command plugin
Serverless: Load command plugin:uninstall
Serverless: Load command plugin
Serverless: Load command plugin:list
Serverless: Load command plugin
Serverless: Load command plugin:search
Serverless: Load command config
Serverless: Load command config:credentials
Serverless: Load command rollback
Serverless: Load command rollback:function
Serverless: Load command upgrade
Serverless: Load command uninstall
Serverless: Load command rubylayer
Serverless: Load command login
Serverless: Load command logout
Serverless: Load command generate-event
Serverless: Load command test
Serverless: Load command dashboard
Serverless: Load command output
Serverless: Load command output:get
Serverless: Load command output:list
Serverless: Load command param
Serverless: Load command param:get
Serverless: Load command param:list
Serverless: Load command studio
Serverless: Load command dev
Serverless: Invoke package
Serverless: Invoke aws:common:validate
Serverless: Invoke aws:common:cleanupTempDir
Serverless: Packaging service...
Serverless: Excluding development dependencies...
Serverless: Clearing previous build ruby layer build
Serverless: Installing gem using docker bundler
Serverless: [DEPRECATED] The `--without` flag is deprecated because it relies on being remembered across bundler invocations, which bundler will no longer do in future versions. Instead please use `bundle config set without 'test development'`, and stop using this flag
Gem::Ext::BuildError: ERROR: Failed to build gem native extension.

current directory:
/var/gem_build/build/ruby/2.5.0/gems/nokogiri-1.10.10/ext/nokogiri
/var/lang/bin/ruby -I /var/lang/lib/ruby/site_ruby/2.5.0 -r
./siteconf20201030-1-7135cl.rb extconf.rb
checking if the C compiler accepts ... yes
Building nokogiri using packaged libraries.
Using mini_portile version 2.4.0
checking for gzdopen() in -lz... yes
checking for iconv... yes
************************************************************************
IMPORTANT NOTICE:

Building Nokogiri with a packaged version of libxml2-2.9.10
with the following patches applied:
        - 0001-Revert-Do-not-URI-escape-in-server-side-includes.patch
        - 0002-Remove-script-macro-support.patch
        - 0003-Update-entities-to-remove-handling-of-ssi.patch
        - 0004-libxml2.la-is-in-top_builddir.patch
        - 0005-Fix-infinite-loop-in-xmlStringLenDecodeEntities.patch

Team Nokogiri will keep on doing their best to provide security
updates in a timely manner, but if this is a concern for you and want
to use the system library instead; abort this installation process and
reinstall nokogiri as follows:

    gem install nokogiri -- --use-system-libraries
        [--with-xml2-config=/path/to/xml2-config]
        [--with-xslt-config=/path/to/xslt-config]

If you are using Bundler, tell it to use the option:

    bundle config build.nokogiri --use-system-libraries
    bundle install

Note, however, that nokogiri is not fully compatible with arbitrary
versions of libxml2 provided by OS/package vendors.
************************************************************************
Extracting libxml2-2.9.10.tar.gz into
tmp/x86_64-pc-linux-gnu/ports/libxml2/2.9.10... ERROR, review
'/var/gem_build/build/ruby/2.5.0/gems/nokogiri-1.10.10/ext/nokogiri/tmp/x86_64-pc-linux-gnu/ports/libxml2/2.9.10/extract.log'
to see what happened. Last lines are:
========================================================================
gtar: libxml2-2.9.10/aclocal.m4: Cannot change ownership to uid 1000, gid 1000:
Operation not permitted
gtar: libxml2-2.9.10/testAutomata.c: Cannot change ownership to uid 1000, gid
1000: Operation not permitted
gtar: libxml2-2.9.10/check-relaxng-test-suite2.py: Cannot change ownership to
uid 1000, gid 1000: Operation not permitted
gtar: libxml2-2.9.10/INSTALL: Cannot change ownership to uid 1000, gid 1000:
Operation not permitted
gtar: libxml2-2.9.10/xmlcatalog.c: Cannot change ownership to uid 1000, gid
1000: Operation not permitted
gtar: libxml2-2.9.10/vms/config.vms: Cannot change ownership to uid 1000, gid
1000: Operation not permitted
gtar: libxml2-2.9.10/vms/readme.vms: Cannot change ownership to uid 1000, gid
1000: Operation not permitted
gtar: libxml2-2.9.10/vms/build_libxml.com: Cannot change ownership to uid 1000,
gid 1000: Operation not permitted
gtar: libxml2-2.9.10/vms/diffs.vms: Cannot change ownership to uid 1000, gid
1000: Operation not permitted
gtar: libxml2-2.9.10/vms: Cannot change ownership to uid 1000, gid 1000:
Operation not permitted
gtar: libxml2-2.9.10/triostr.h: Cannot change ownership to uid 1000, gid 1000:
Operation not permitted
gtar: libxml2-2.9.10/tree.c: Cannot change ownership to uid 1000, gid 1000:
Operation not permitted
gtar: libxml2-2.9.10/testModule.c: Cannot change ownership to uid 1000, gid
1000: Operation not permitted
gtar: libxml2-2.9.10/relaxng.c: Cannot change ownership to uid 1000, gid 1000:
Operation not permitted
gtar: libxml2-2.9.10/libxml2-config.cmake.in: Cannot change ownership to uid
1000, gid 1000: Operation not permitted
gtar: libxml2-2.9.10/Makefile.tests: Cannot change ownership to uid 1000, gid
1000: Operation not permitted
gtar: libxml2-2.9.10/dict.c: Cannot change ownership to uid 1000, gid 1000:
Operation not permitted
gtar: libxml2-2.9.10/libxml.spec.in: Cannot change ownership to uid 1000, gid
1000: Operation not permitted
gtar: libxml2-2.9.10: Cannot change ownership to uid 1000, gid 1000: Operation
not permitted
gtar: Exiting with failure status due to previous errors
========================================================================
*** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of necessary
libraries and/or headers.  Check the mkmf.log file for more details.  You may
need configuration options.

Provided configuration options:
        --with-opt-dir
        --without-opt-dir
        --with-opt-include
        --without-opt-include=${opt-dir}/include
        --with-opt-lib
        --without-opt-lib=${opt-dir}/lib
        --with-make-prog
        --without-make-prog
        --srcdir=.
        --curdir
        --ruby=/var/lang/bin/$(RUBY_BASE_NAME)
        --help
        --clean
        --use-system-libraries
        --enable-static
        --disable-static
        --with-zlib-dir
        --without-zlib-dir
        --with-zlib-include
        --without-zlib-include=${zlib-dir}/include
        --with-zlib-lib
        --without-zlib-lib=${zlib-dir}/lib
        --enable-cross-build
        --disable-cross-build
/var/gem_build/build/ruby/2.5.0/gems/mini_portile2-2.4.0/lib/mini_portile2/mini_portile.rb:402:in
`block in execute': Failed to complete extract task (RuntimeError)
from
/var/gem_build/build/ruby/2.5.0/gems/mini_portile2-2.4.0/lib/mini_portile2/mini_portile.rb:373:in
`chdir'
from
/var/gem_build/build/ruby/2.5.0/gems/mini_portile2-2.4.0/lib/mini_portile2/mini_portile.rb:373:in
`execute'
from
/var/gem_build/build/ruby/2.5.0/gems/mini_portile2-2.4.0/lib/mini_portile2/mini_portile.rb:367:in
`extract_file'
from
/var/gem_build/build/ruby/2.5.0/gems/mini_portile2-2.4.0/lib/mini_portile2/mini_portile.rb:62:in
`block in extract'
from
/var/gem_build/build/ruby/2.5.0/gems/mini_portile2-2.4.0/lib/mini_portile2/mini_portile.rb:60:in
`each'
from
/var/gem_build/build/ruby/2.5.0/gems/mini_portile2-2.4.0/lib/mini_portile2/mini_portile.rb:60:in
`extract'
from
/var/gem_build/build/ruby/2.5.0/gems/mini_portile2-2.4.0/lib/mini_portile2/mini_portile.rb:151:in
`cook'
        from extconf.rb:365:in `block (2 levels) in process_recipe'
        from extconf.rb:257:in `block in chdir_for_build'
        from extconf.rb:256:in `chdir'
        from extconf.rb:256:in `chdir_for_build'
        from extconf.rb:364:in `block in process_recipe'
        from extconf.rb:262:in `tap'
        from extconf.rb:262:in `process_recipe'
        from extconf.rb:557:in `<main>'

To see why this extension failed to compile, please check the mkmf.log which can
be found here:

/var/gem_build/build/ruby/2.5.0/extensions/x86_64-linux/2.5.0-static/nokogiri-1.10.10/mkmf.log

extconf failed, exit code 1

Gem files will remain installed in
/var/gem_build/build/ruby/2.5.0/gems/nokogiri-1.10.10 for inspection.
Results logged to
/var/gem_build/build/ruby/2.5.0/extensions/x86_64-linux/2.5.0-static/nokogiri-1.10.10/gem_make.out

An error occurred while installing nokogiri (1.10.10), and Bundler cannot
continue.
Make sure that `gem install nokogiri -v '1.10.10' --source
'https://rubygems.org/'` succeeds before bundling.

In Gemfile:
  nokogiri


  Error --------------------------------------------------

  Error: [DEPRECATED] The `--without` flag is deprecated because it relies on being remembered across bundler invocations, which bundler will no longer do in future versions. Instead please use `bundle config set without 'test development'`, and stop using this flag
  Gem::Ext::BuildError: ERROR: Failed to build gem native extension.

  current directory:
  /var/gem_build/build/ruby/2.5.0/gems/nokogiri-1.10.10/ext/nokogiri
  /var/lang/bin/ruby -I /var/lang/lib/ruby/site_ruby/2.5.0 -r
  ./siteconf20201030-1-7135cl.rb extconf.rb
  checking if the C compiler accepts ... yes
  Building nokogiri using packaged libraries.
  Using mini_portile version 2.4.0
  checking for gzdopen() in -lz... yes
  checking for iconv... yes
  ************************************************************************
  IMPORTANT NOTICE:

  Building Nokogiri with a packaged version of libxml2-2.9.10
  with the following patches applied:
        - 0001-Revert-Do-not-URI-escape-in-server-side-includes.patch
        - 0002-Remove-script-macro-support.patch
        - 0003-Update-entities-to-remove-handling-of-ssi.patch
        - 0004-libxml2.la-is-in-top_builddir.patch
        - 0005-Fix-infinite-loop-in-xmlStringLenDecodeEntities.patch

  Team Nokogiri will keep on doing their best to provide security
  updates in a timely manner, but if this is a concern for you and want
  to use the system library instead; abort this installation process and
  reinstall nokogiri as follows:

      gem install nokogiri -- --use-system-libraries
          [--with-xml2-config=/path/to/xml2-config]
          [--with-xslt-config=/path/to/xslt-config]

  If you are using Bundler, tell it to use the option:

      bundle config build.nokogiri --use-system-libraries
      bundle install

  Note, however, that nokogiri is not fully compatible with arbitrary
  versions of libxml2 provided by OS/package vendors.
  ************************************************************************
  Extracting libxml2-2.9.10.tar.gz into
  tmp/x86_64-pc-linux-gnu/ports/libxml2/2.9.10... ERROR, review
  '/var/gem_build/build/ruby/2.5.0/gems/nokogiri-1.10.10/ext/nokogiri/tmp/x86_64-pc-linux-gnu/ports/libxml2/2.9.10/extract.log'
  to see what happened. Last lines are:
  ========================================================================
  gtar: libxml2-2.9.10/aclocal.m4: Cannot change ownership to uid 1000, gid 1000:
  Operation not permitted
  gtar: libxml2-2.9.10/testAutomata.c: Cannot change ownership to uid 1000, gid
  1000: Operation not permitted
  gtar: libxml2-2.9.10/check-relaxng-test-suite2.py: Cannot change ownership to
  uid 1000, gid 1000: Operation not permitted
  gtar: libxml2-2.9.10/INSTALL: Cannot change ownership to uid 1000, gid 1000:
  Operation not permitted
  gtar: libxml2-2.9.10/xmlcatalog.c: Cannot change ownership to uid 1000, gid
  1000: Operation not permitted
  gtar: libxml2-2.9.10/vms/config.vms: Cannot change ownership to uid 1000, gid
  1000: Operation not permitted
  gtar: libxml2-2.9.10/vms/readme.vms: Cannot change ownership to uid 1000, gid
  1000: Operation not permitted
  gtar: libxml2-2.9.10/vms/build_libxml.com: Cannot change ownership to uid 1000,
  gid 1000: Operation not permitted
  gtar: libxml2-2.9.10/vms/diffs.vms: Cannot change ownership to uid 1000, gid
  1000: Operation not permitted
  gtar: libxml2-2.9.10/vms: Cannot change ownership to uid 1000, gid 1000:
  Operation not permitted
  gtar: libxml2-2.9.10/triostr.h: Cannot change ownership to uid 1000, gid 1000:
  Operation not permitted
  gtar: libxml2-2.9.10/tree.c: Cannot change ownership to uid 1000, gid 1000:
  Operation not permitted
  gtar: libxml2-2.9.10/testModule.c: Cannot change ownership to uid 1000, gid
  1000: Operation not permitted
  gtar: libxml2-2.9.10/relaxng.c: Cannot change ownership to uid 1000, gid 1000:
  Operation not permitted
  gtar: libxml2-2.9.10/libxml2-config.cmake.in: Cannot change ownership to uid
  1000, gid 1000: Operation not permitted
  gtar: libxml2-2.9.10/Makefile.tests: Cannot change ownership to uid 1000, gid
  1000: Operation not permitted
  gtar: libxml2-2.9.10/dict.c: Cannot change ownership to uid 1000, gid 1000:
  Operation not permitted
  gtar: libxml2-2.9.10/libxml.spec.in: Cannot change ownership to uid 1000, gid
  1000: Operation not permitted
  gtar: libxml2-2.9.10: Cannot change ownership to uid 1000, gid 1000: Operation
  not permitted
  gtar: Exiting with failure status due to previous errors
  ========================================================================
  *** extconf.rb failed ***
  Could not create Makefile due to some reason, probably lack of necessary
  libraries and/or headers.  Check the mkmf.log file for more details.  You may
  need configuration options.

  Provided configuration options:
        --with-opt-dir
        --without-opt-dir
        --with-opt-include
        --without-opt-include=${opt-dir}/include
        --with-opt-lib
        --without-opt-lib=${opt-dir}/lib
        --with-make-prog
        --without-make-prog
        --srcdir=.
        --curdir
        --ruby=/var/lang/bin/$(RUBY_BASE_NAME)
        --help
        --clean
        --use-system-libraries
        --enable-static
        --disable-static
        --with-zlib-dir
        --without-zlib-dir
        --with-zlib-include
        --without-zlib-include=${zlib-dir}/include
        --with-zlib-lib
        --without-zlib-lib=${zlib-dir}/lib
        --enable-cross-build
        --disable-cross-build
  /var/gem_build/build/ruby/2.5.0/gems/mini_portile2-2.4.0/lib/mini_portile2/mini_portile.rb:402:in
  `block in execute': Failed to complete extract task (RuntimeError)
  from
  /var/gem_build/build/ruby/2.5.0/gems/mini_portile2-2.4.0/lib/mini_portile2/mini_portile.rb:373:in
  `chdir'
  from
  /var/gem_build/build/ruby/2.5.0/gems/mini_portile2-2.4.0/lib/mini_portile2/mini_portile.rb:373:in
  `execute'
  from
  /var/gem_build/build/ruby/2.5.0/gems/mini_portile2-2.4.0/lib/mini_portile2/mini_portile.rb:367:in
  `extract_file'
  from
  /var/gem_build/build/ruby/2.5.0/gems/mini_portile2-2.4.0/lib/mini_portile2/mini_portile.rb:62:in
  `block in extract'
  from
  /var/gem_build/build/ruby/2.5.0/gems/mini_portile2-2.4.0/lib/mini_portile2/mini_portile.rb:60:in
  `each'
  from
  /var/gem_build/build/ruby/2.5.0/gems/mini_portile2-2.4.0/lib/mini_portile2/mini_portile.rb:60:in
  `extract'
  from
  /var/gem_build/build/ruby/2.5.0/gems/mini_portile2-2.4.0/lib/mini_portile2/mini_portile.rb:151:in
  `cook'
        from extconf.rb:365:in `block (2 levels) in process_recipe'
        from extconf.rb:257:in `block in chdir_for_build'
        from extconf.rb:256:in `chdir'
        from extconf.rb:256:in `chdir_for_build'
        from extconf.rb:364:in `block in process_recipe'
        from extconf.rb:262:in `tap'
        from extconf.rb:262:in `process_recipe'
        from extconf.rb:557:in `<main>'

  To see why this extension failed to compile, please check the mkmf.log which can
  be found here:

  /var/gem_build/build/ruby/2.5.0/extensions/x86_64-linux/2.5.0-static/nokogiri-1.10.10/mkmf.log

  extconf failed, exit code 1

  Gem files will remain installed in
  /var/gem_build/build/ruby/2.5.0/gems/nokogiri-1.10.10 for inspection.
  Results logged to
  /var/gem_build/build/ruby/2.5.0/extensions/x86_64-linux/2.5.0-static/nokogiri-1.10.10/gem_make.out

  An error occurred while installing nokogiri (1.10.10), and Bundler cannot
  continue.
  Make sure that `gem install nokogiri -v '1.10.10' --source
  'https://rubygems.org/'` succeeds before bundling.

  In Gemfile:
    nokogiri

      at runCommand (/Users/arjun/open_source/serverless-ruby-layer/examples/use_docker/node_modules/serverless-ruby-layer/lib/bundle.js:23:11)
      at docker (/Users/arjun/open_source/serverless-ruby-layer/examples/use_docker/node_modules/serverless-ruby-layer/lib/bundle.js:29:12)
      at ServerlessRubyBundler.bundleInstall (/Users/arjun/open_source/serverless-ruby-layer/examples/use_docker/node_modules/serverless-ruby-layer/lib/bundle.js:100:5)
  From previous event:
      at PluginManager.invoke (/usr/local/lib/node_modules/serverless/lib/classes/PluginManager.js:483:22)
      at /usr/local/lib/node_modules/serverless/lib/classes/PluginManager.js:518:24
  From previous event:
      at PluginManager.run (/usr/local/lib/node_modules/serverless/lib/classes/PluginManager.js:518:8)
      at /usr/local/lib/node_modules/serverless/lib/Serverless.js:136:33
  From previous event:
      at Serverless.run (/usr/local/lib/node_modules/serverless/lib/Serverless.js:123:74)
      at /usr/local/lib/node_modules/serverless/scripts/serverless.js:54:26
      at processImmediate (internal/timers.js:444:21)
      at process.topLevelDomainCallback (domain.js:134:15)
  From previous event:
      at Object.<anonymous> (/usr/local/lib/node_modules/serverless/scripts/serverless.js:54:4)
      at Module._compile (internal/modules/cjs/loader.js:1128:30)
      at Object.Module._extensions..js (internal/modules/cjs/loader.js:1167:10)
      at Module.load (internal/modules/cjs/loader.js:983:32)
      at Function.Module._load (internal/modules/cjs/loader.js:891:14)
      at Module.require (internal/modules/cjs/loader.js:1023:19)
      at require (internal/modules/cjs/helpers.js:72:18)
      at Object.<anonymous> (/usr/local/lib/node_modules/serverless/bin/serverless.js:41:1)
      at Module._compile (internal/modules/cjs/loader.js:1128:30)
      at Object.Module._extensions..js (internal/modules/cjs/loader.js:1167:10)
      at Module.load (internal/modules/cjs/loader.js:983:32)
      at Function.Module._load (internal/modules/cjs/loader.js:891:14)
      at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:71:12)
      at internal/main/run_main_module.js:17:47

  Get Support --------------------------------------------
     Docs:          docs.serverless.com
     Bugs:          github.com/serverless/serverless/issues
     Issues:        forum.serverless.com

  Your Environment Information ---------------------------
     Operating System:          darwin
     Node Version:              13.3.0
     Framework Version:         1.78.1
     Plugin Version:            3.7.0
     SDK Version:               2.3.1
     Components Version:        2.34.3

Example Config files
I am only running the example folder.

Desktop (please complete the following information):

OS: [e.g. Mac OS]
NodeVersion v13.3.0
Ruby version 2.5.8
Bundler version 1.17.2

Passing environment to dockerfile in serverless v3 + not working

Describe the bug
In serverless v3+ was changed CLI environment. Now to pass ENV need to use --param. More info: https://www.serverless.com/framework/docs/guides/parameters

After this change it's not possible to pass env to serverless-ruby-layer Dockerfile

Error Stack Trace

Environment: linux, node 14.21.3, framework 3.35.2, plugin 7.0.5, SDK 4.4.0
Credentials: Local, "default" profile
Docs:        docs.serverless.com
Support:     forum.serverless.com
Bugs:        github.com/serverless/serverless/issues

Error:
TypeError: this.options.environment.forEach is not a function
    at ServerlessRubyBundler.bundleInstall (/home/runner/work/project/project/node_modules/serverless-ruby-layer/lib/bundle.js:97:34)

Example Config files
Dockerfile:

ARG GH_PACKAGE_TOKEN
ENV GH_PACKAGE_TOKEN=${GH_PACKAGE_TOKEN}

RUN bundle config set https://rubygems.pkg.github.com username:${GH_PACKAGE_TOKEN}

serverless.yml:

  rubyLayer:
    use_docker: true
    docker_file: Dockerfile
    environment:
      GH_PACKAGE_TOKEN: ${param:gh_package_token}

Deploy run:

sls deploy --verbose --param="gh_package_token=${{ secrets.GH_PACKAGES_SECRET }}"

Desktop (please complete the following information):

OS: Ubuntu 22.04
NodeVersion: 14.21.13
Ruby version: 2.7.1
Bundler version: 2.2.17

Additional context

In the serverless v2+ env was passing with this config, which was works fine

custom:
  rubyLayer:
    use_docker: true
    docker_file: Dockerfile
    environment:
      - GH_PACKAGE_TOKEN=${opt:gh_package_token, 'default'}

sls deploy --stage --gh_package_token ${{ secrets.GH_PACKAGES_SECRET }}

In version v3+ this config was deprecated, and we catch this error:

Error:
Detected unrecognized CLI options: "--gh_package_token".

Add github issue template and Contributions.md

Add ignore_gemfile_lock option to ignore Gemfile.lock while building gem layer

Currently, by default, the Gemfile.lock is used while building the gem layer.
There should be an option to ignore if we don't want to consider it.

custom:
  rubyLayer:
    ignore_gemfile_lock: true

Could not find gem in any of the sources w/t require "bundler/setup"

Describe the bug

To auto require gems, we specific the following.

  require "rubygems"
  require "bundler/setup"

I can remove these requires, and it works, but, wondering why it doesn't

Error Stack Trace

Init error when loading handler handler.Handler.list
2021-02-22T14:55:03.403+00:00	{
2021-02-22T14:55:03.403+00:00	"errorMessage": "Could not find public_suffix-3.1.1 in any of the sources",
2021-02-22T14:55:03.403+00:00	"errorType": "Init<Bundler::GemNotFound>",
2021-02-22T14:55:03.403+00:00	"stackTrace": [
2021-02-22T14:55:03.403+00:00	"/var/runtime/gems/bundler-2.2.5/lib/bundler/spec_set.rb:87:in `block in materialize'",
2021-02-22T14:55:03.403+00:00	"/var/runtime/gems/bundler-2.2.5/lib/bundler/spec_set.rb:81:in `map!'",
2021-02-22T14:55:03.403+00:00	"/var/runtime/gems/bundler-2.2.5/lib/bundler/spec_set.rb:81:in `materialize'",
2021-02-22T14:55:03.403+00:00	"/var/runtime/gems/bundler-2.2.5/lib/bundler/definition.rb:175:in `specs'",
2021-02-22T14:55:03.403+00:00	"/var/runtime/gems/bundler-2.2.5/lib/bundler/definition.rb:245:in `specs_for'",
2021-02-22T14:55:03.403+00:00	"/var/runtime/gems/bundler-2.2.5/lib/bundler/definition.rb:227:in `requested_specs'",
2021-02-22T14:55:03.403+00:00	"/var/runtime/gems/bundler-2.2.5/lib/bundler/runtime.rb:91:in `block in definition_method'",
2021-02-22T14:55:03.403+00:00	"/var/runtime/gems/bundler-2.2.5/lib/bundler/runtime.rb:20:in `setup'",
2021-02-22T14:55:03.403+00:00	"/var/runtime/gems/bundler-2.2.5/lib/bundler.rb:148:in `setup'",
2021-02-22T14:55:03.403+00:00	"/var/runtime/gems/bundler-2.2.5/lib/bundler/setup.rb:20:in `block in <top (required)>'",
2021-02-22T14:55:03.403+00:00	"/var/runtime/gems/bundler-2.2.5/lib/bundler/ui/shell.rb:136:in `with_level'",
2021-02-22T14:55:03.403+00:00	"/var/runtime/gems/bundler-2.2.5/lib/bundler/ui/shell.rb:88:in `silence'",
2021-02-22T14:55:03.403+00:00	"/var/runtime/gems/bundler-2.2.5/lib/bundler/setup.rb:20:in `<top (required)>'",
2021-02-22T14:55:03.403+00:00	"/var/lang/lib/ruby/site_ruby/2.5.0/rubygems/core_ext/kernel_require.rb:85:in `require'",
2021-02-22T14:55:03.403+00:00	"/var/lang/lib/ruby/site_ruby/2.5.0/rubygems/core_ext/kernel_require.rb:85:in `require'",
2021-02-22T14:55:03.403+00:00	"/var/task/config/initializers/autoloader.rb:11:in `<top (required)>'",
2021-02-22T14:55:03.403+00:00	"/var/lang/lib/ruby/site_ruby/2.5.0/rubygems/core_ext/kernel_require.rb:85:in `require'",
2021-02-22T14:55:03.403+00:00	"/var/lang/lib/ruby/site_ruby/2.5.0/rubygems/core_ext/kernel_require.rb:85:in `require'",
2021-02-22T14:55:03.403+00:00	"/var/task/handler.rb:1:in `<top (required)>'",

Add validations in all workflows and throw user informative errors

Add validations in all workflows and throw user informative errors.
Add enough log to make the process status informative
debug logs for all necessary areas

Specifically need to include or exclude functions that require lambda layer instead of adding it to all functions

ActiveSupport cache strategies are excluded from gemLayer.zip

Describe the bug

The cache sub directory of the active_support gem is excluded from gemLayer.zip. I can see that the files are in the build area, but are missing from only the zip.

$ ls -al .serverless/ruby_layer/build/ruby/2.7.0/gems/act
ivesupport-6.1.4/lib/active_support/cache/
total 104
drwxr-xr-x   8 jgreen  staff   256B Jul 21 22:03 ./
drwxr-xr-x  85 jgreen  staff   2.7K Jul 21 22:03 ../
-rw-rw-r--   1 jgreen  staff   6.5K Jul 21 22:03 file_store.rb
-rw-rw-r--   1 jgreen  staff   8.1K Jul 21 22:03 mem_cache_store.rb
-rw-rw-r--   1 jgreen  staff   6.0K Jul 21 22:03 memory_store.rb
-rw-rw-r--   1 jgreen  staff   1.1K Jul 21 22:03 null_store.rb
-rw-rw-r--   1 jgreen  staff    16K Jul 21 22:03 redis_cache_store.rb
drwxr-xr-x   4 jgreen  staff   128B Jul 21 22:03 strategy/

$ unzip -l .serverless/ruby_layer/gemLayer.zip | grep "active_support/cache"
    33224  07-22-2021 03:03   ruby/2.7.0/gems/activesupport-6.1.4/lib/active_support/cache.rb

I'm pretty sure this block is what's causing the trouble. https://github.com/navarasu/serverless-ruby-layer/blob/master/lib/bundle.js#L175-L177

              if (new RegExp('ruby/.*/cache$').test(filePath)){
                return undefined;
              }

I'm not sure what the intent of that bit is, but I think it's being overly greedy on what it matches.

Trying to deploy example but gives out TypeError [ERR_INVALID_ARG_TYPE]

I just copied out the example to try things out but when doing sls deploy but gives an error.

TypeError [ERR_INVALID_ARG_TYPE]: The "path" argument must be of type string. Received an instance of Buffer
      at validateString (internal/validators.js:117:11)
      at Object.dirname (path.js:1128:5)
      at isRootPath (pkg/prelude/bootstrap.js:168:26)
      at fs.readdirSync (pkg/prelude/bootstrap.js:829:18)
      at _rmdirSync (internal/fs/rimraf.js:242:7)
      at rimrafSync (internal/fs/rimraf.js:191:7)
      at Object.rmdirSync (fs.js:764:12)
      layer/node_modules/bluebird/js/release/async.js:15:14)
      at processImmediate (internal/timers.js:456:21)
      at process.topLevelDomainCallback (domain.js:137:15)

     For debugging logs, run again after setting the "SLS_DEBUG=*" environment variable.

Here's the environment information:

     Operating System:          darwin
     Node Version:              12.16.1
     Framework Version:         1.67.3 (standalone)
     Plugin Version:            3.6.6
     SDK Version:               2.3.0
     Components Version:        2.29.0

use_docker: false doesn't work as expected

Describe the bug
use_docker: false One would expected this option to skip docker all together and do its thing. However it doesn't do that and instead doesn't include the native libs.

Use case: use_docker: false is useful for cases where serverless is already running in a codebuild container, there's no need to launch docker because codebuild can be used to compile layers.

In my case I cannot strip the serverless process out of codebuild so instead I had to configure codebuild to able able to run docker, run docker, then do the serverless deploy. painful.

Deprecated warnings are appearing.

Describe the warning
Deprecated warnings are appearing.
I need to set the config as shown in the message, and then run a normal bundle install.
bundle config set without 'test development'

Error Stack Trace

Serverless: [DEPRECATED] The `--without` flag is deprecated because it relies on being remembered across bundler invocations, which bundler will no longer do in future versions. Instead please use `bundle config set without 'test development'`, and stop using this flag

Example Config files

plugins:
  - serverless-ruby-layer

Desktop (please complete the following information):

OS: Mac OS
NodeVersion 14.14.0
Ruby version 2.7.0
Bundler version 2.1.4

Additional context

Add option use docker for deploying gem with native extention

Add option deploy gem with the native extensions using docker

https://blog.francium.tech/deploying-ruby-to-aws-lambda-function-and-layers-a6c00b53a5d0

Support for Ruby 3.2 Runtime

Is your feature request related to a problem? Please describe.
AWS Lamba has added support for the Ruby 3.2 runtime.

The serverless-ruby-layer use_docker option, when set to true, uses the lambci/lambda docker images, which project seems to be have been archived and wouldn't have a Docker image for Ruby 3.2.

Describe the solution you'd like
Support for Ruby 3.2 runtime is available. The use_docker option, when set to true, will use an appropriate Docker image for Ruby 3.2.

Describe alternatives you've considered

Additional context

Proposal to speed up repeated Docker based deploys

The Problem

When deploying with Docker we end up installing all the gems from scratch each time, even if the Gemfile hasn't changed. On my smallish demo project the bundle install step inside the running container takes about 90 seconds.

Proposed solution

If we moved the bundle install step (and the bundle config set ... steps) into the Dockerfile then we could take advantage of Docker layer caching and we'd only have to install gems when the Gemfile changes.

I've tested a very crude version of this change and in my demo project a second full deploy (without changes to Gemfile) takes about 2 min 30 seconds compared to it taking about 4 minutes with the currently implemented approach.

Complicating factors

Since we currently allow people to choose between using their own Dockerfile or an auto-generated one we'd need to handle both cases, and we'd have to require people to handle the bundle config and bundle install steps in their own Dockerfile. That would be non-backwards-compatible breaking change, so we'd probably want to consider bumping the MAJOR portion of the version number (bump to 2.0.0).

Example Dockerfile

Here's an example Dockerfile that is currently working with my demo project (and my hacked up modification to serverless-ruby-layer). The bits about creating and moving into /var/gem_build are there to accommodate other parts of the existing implementation, but since the resulting image won't be distributed for production it may not be necessary to do things in that directory, and we might be able to use the default working directory of /var/task just to keep things simple.

FROM lambci/lambda:build-ruby2.7 AS base

RUN yum install -y postgresql-devel

RUN gem update bundler

RUN mkdir /var/gem_build

WORKDIR /var/gem_build

RUN bundle config set --local path build
RUN bundle config set --local without test development

COPY Gemfile* .

RUN bundle install

CMD "/bin/bash"

And here's the output of a first build taking almost 2 minutes:

$ time docker build .
[+] Building 113.2s (15/15) FINISHED
 => [internal] load build definition from Dockerfile                                                            0.0s
 => => transferring dockerfile: 363B                                                                            0.0s
 => [internal] load .dockerignore                                                                               0.0s
 => => transferring context: 2B                                                                                 0.0s
 => [internal] load metadata for docker.io/lambci/lambda:build-ruby2.7                                          0.0s
 => CACHED [ 1/10] FROM docker.io/lambci/lambda:build-ruby2.7                                                   0.0s
 => [internal] load build context                                                                               0.0s
 => => transferring context: 61B                                                                                0.0s
 => [ 2/10] RUN echo "yo!"                                                                                      0.4s
 => [ 3/10] RUN yum install -y postgresql-devel                                                                13.5s
 => [ 4/10] RUN gem update bundler                                                                             12.2s
 => [ 5/10] RUN mkdir /var/gem_build                                                                            0.4s
 => [ 6/10] WORKDIR /var/gem_build                                                                              0.0s
 => [ 7/10] RUN bundle config set --local path build                                                            0.5s
 => [ 8/10] RUN bundle config set --local without test development                                              0.4s
 => [ 9/10] COPY Gemfile* .                                                                                     0.0s
 => [10/10] RUN bundle install                                                                                 84.0s
 => exporting to image                                                                                          1.5s
 => => exporting layers                                                                                         1.5s
 => => writing image sha256:b3e4ef459cc93a9a4d754447d6225b581a01a349f6507cc620fac2994aa62fe0                    0.0s

Use 'docker scan' to run Snyk tests against images to find vulnerabilities and learn how to fix them

real    1m54.092s
user    0m0.449s
sys     0m0.394s

And the output of a second build taking less than 1 second:

$ time docker build .
[+] Building 0.1s (15/15) FINISHED
 => [internal] load build definition from Dockerfile                                                            0.0s
 => => transferring dockerfile: 37B                                                                             0.0s
 => [internal] load .dockerignore                                                                               0.0s
 => => transferring context: 2B                                                                                 0.0s
 => [internal] load metadata for docker.io/lambci/lambda:build-ruby2.7                                          0.0s
 => [ 1/10] FROM docker.io/lambci/lambda:build-ruby2.7                                                          0.0s
 => [internal] load build context                                                                               0.0s
 => => transferring context: 61B                                                                                0.0s
 => CACHED [ 2/10] RUN echo "yo!"                                                                               0.0s
 => CACHED [ 3/10] RUN yum install -y postgresql-devel                                                          0.0s
 => CACHED [ 4/10] RUN gem update bundler                                                                       0.0s
 => CACHED [ 5/10] RUN mkdir /var/gem_build                                                                     0.0s
 => CACHED [ 6/10] WORKDIR /var/gem_build                                                                       0.0s
 => CACHED [ 7/10] RUN bundle config set --local path build                                                     0.0s
 => CACHED [ 8/10] RUN bundle config set --local without test development                                       0.0s
 => CACHED [ 9/10] COPY Gemfile* .                                                                              0.0s
 => CACHED [10/10] RUN bundle install                                                                           0.0s
 => exporting to image                                                                                          0.0s
 => => exporting layers                                                                                         0.0s
 => => writing image sha256:b3e4ef459cc93a9a4d754447d6225b581a01a349f6507cc620fac2994aa62fe0                    0.0s

Use 'docker scan' to run Snyk tests against images to find vulnerabilities and learn how to fix them

real    0m0.661s
user    0m0.193s
sys     0m0.139s

Lingering questions

Maybe there's some reason that I'm not aware of that doing bundle install as a Dockerfile layer won't work, or is problematic? Are there advantages to doing it after the container is built instead of as part of the container build process?

@navarasu If this sounds like an approach that you're open to investigating I'd be happy to submit a PR.

Add deployment workflow in circle ci to publish node package

Option to use Dockerfile to bundle gem with native os library dependencies

Option to use Dockerfile to bundle gems like pg and Mysql which native os library dependencies. These libraries must be present in the docker before installing gem

Provide option to create layer for specific gem groups

I am using RSpec testing, and I couldn't ignore having Rspec in my Lambda Layer. If there is an option to bundle ignoring gem groups like bundle install --without= test, it would be nice to have.

Handle docker inside docker case for using plugin in CI environment like Circle CI, etc.

Currently, the volume mount option doesn't work in circle ci. we need to handle it using with docker cp for getting the installed gems

Docker build doesn't respect `Gemfile.lock`

Describe the bug

My project has a Gemfile and a Gemfile.lock but when I sls deploy I end up with a different set of gems being installed than what is specified in the lock file. If I remove Gemfile.lock the build process also does not generate one, which it seems like it should.

Error Stack Trace

None

Example Config files

My Gemfile.lock contains:

aws-sdk-core (3.113.0)

But the .serverless/ruby_layer/gemLayer.zip package contains version 3.113.1.

unzip -l .serverless/ruby_layer/gemLayer.zip | grep aws-sdk-core
   0  04-02-2021 18:17   ruby/2.7.0/gems/aws-sdk-core-3.113.1/
...

Desktop (please complete the following information):

OS: [e.g. Mac OS]
NodeVersion [e.g. 12.16.1]
Ruby version [e.g. 2.5.8]
Bundler version [e.g. 2.1.2]

Additional context
Add any other context about the problem here.

Required option to preinstall os specific libraries (yum libs) before building gems

Mostly in custom dockerfile, we install necessary yum libraries. For e.g, for pg gem, we install postgresql-devel yum package before doing bundle install.
So need to option to specify a list of yums to be installed rather than specifying docker file

custom:
  rubyLayer:
    use_docker: true
    docker_yums:
      - postgresql-devel

navarasu / serverless-ruby-layer Goto Github PK

serverless-ruby-layer's Introduction

Install

Documentation

Getting Started

Simple Usage

Customization

Usage

Contributing

License

serverless-ruby-layer's People

Contributors

Stargazers

Watchers

Forkers

serverless-ruby-layer's Issues

Describe alternatives you've considered

Additional context

The Problem

Proposed solution

Complicating factors

Example Dockerfile

Lingering questions

Recommend Projects

Recommend Topics

Recommend Org