Code Monkey home page Code Monkey logo

chef-ebs's Introduction

chef-ebs

This is a cookbook that makes it easy to create/attach EBS volumes, and create filesystems and RAID arrays on them.

Usage

RAID Array Creation

Add recipe[ebs] to your run list, and configure these attributes:

Create a RAID 10 across four 10GB volumes each with 2000 provisioned iops, make it an lvm logical volume, format it with XFS, and mount it on /data.

{
  :ebs => {
    :raids => {
      '/dev/md0' => {
        :num_disks => 4,
        :disk_size => 10,
        :piops => 2000,
        :raid_level => 10,
        :fstype => 'xfs',
        :mount_point => '/data',
        :uselvm => true,
      }
    }
  }
}

Use Existing Volumes for RAID Array

Add recipe[persistent] to your run list, and configure these attributes:

Create a RAID 10 across the volumes specified in the persistent_volumes array, do not use LVM, format it with XFS, and mount it on /data.

{
  :ebs => {
    :raids => {
      '/dev/md0' => {
        :raid_level => 10,
        :fstype => 'xfs',
        :mount_point => '/data',
        :uselvm => false,
        :persistent_volumes => [
          "vol-xxxxxxxx",
          "vol-xxxxxxxx",
          "vol-xxxxxxxx",
          "vol-xxxxxxxx"
        ]
      }
    }
  }
}

EBS Volume Creation

Create a 10GB volume with 1000 provisioned iops, format it with XFS, and mount it on /data with noatime as an option.

{
  :ebs => {
    :volumes => {
      '/data' => {
        :size => 10,
        :piops => 1000,
        :fstype => 'xfs',
        :mount_options => 'noatime'
      }
    }
  }
}

mount_options are optional and will default to noatime,nobootwait on all platforms except Amazon linux, where they will default to noatime.

Credentials

Expects a credentials databag with an aws item that contains aws_access_key_id and aws_secret_access_key.

You can override the databag and item names with node[:ebs][:creds][:databag], and node[:ebs][:creds][:item], but the key names are static.

Requirements

Acknowledgments

This code was originally forked from the Scalarium ebs cookbook which has since been taken over by AWS Opsworks.

chef-ebs's People

Contributors

albertsj1 avatar aliang avatar baldowl avatar blakehilscher avatar dwradcliffe avatar ikanusim avatar jaysignorello avatar jbyck avatar jweiss avatar kensodev avatar kesor avatar kvs avatar logikal avatar mbbernstein avatar msaffitz avatar pallan avatar skilgore avatar titanous avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

chef-ebs's Issues

Keeps waiting for a volume...

Hi, I'm attempting to add a new raid ebs volume, and /dev/xvdf never becomes ready. I also don't see new ebs volumes being created in the in the aws console. I'm using the latest master branch of chef-ebs and latest chefdk:

default[:ebs][:raids]['/dev/md0'][:num_disks] = 4
default[:ebs][:raids]['/dev/md0'][:disk_size] = 10

default[:ebs][:raids]['/dev/md0'][:piops] = 30

default[:ebs][:raids]['/dev/md0'][:raid_level] = 0
default[:ebs][:raids]['/dev/md0'][:fstype] = 'xfs'
default[:ebs][:raids]['/dev/md0'][:mount_point] = '/mnt/mysql'
default[:ebs][:raids]['/dev/md0'][:uselvm] = true

ec2-54-82-103-248.compute-1.amazonaws.com * ruby_block[wait for devices] action run[2016-04-04T15:53:23+00:00] INFO: Processing ruby_block[wait for devices] action run (ebs::raids line 58)
DEBUG: received packet nr 1044 type 94 len 156
INFO: channel_data: 0 131b
ec2-54-82-103-248.compute-1.amazonaws.com [2016-04-04T15:53:23+00:00] DEBUG: Providers for generic ruby_block resource enabled on node include: [Chef::Provider::RubyBlock]
DEBUG: received packet nr 1045 type 94 len 156
INFO: channel_data: 0 130b
ec2-54-82-103-248.compute-1.amazonaws.com [2016-04-04T15:53:23+00:00] DEBUG: Provider for action run on resource ruby_block[wait for devices] is Chef::Provider::RubyBlock
DEBUG: received packet nr 1046 type 94 len 108
INFO: channel_data: 0 83b
ec2-54-82-103-248.compute-1.amazonaws.com [2016-04-04T15:53:23+00:00] INFO: Waiting for individual disks of RAID /mnt/mysql
DEBUG: received packet nr 1047 type 94 len 92
INFO: channel_data: 0 72b
ec2-54-82-103-248.compute-1.amazonaws.com [2016-04-04T15:53:23+00:00] INFO: device /dev/xvdf not ready - waiting
DEBUG: read 116 bytes
DEBUG: received packet nr 1048 type 94 len 92
INFO: channel_data: 0 72b
ec2-54-82-103-248.compute-1.amazonaws.com [2016-04-04T15:53:33+00:00] INFO: device /dev/xvdf not ready - waiting
DEBUG: read 116 bytes
DEBUG: received packet nr 1049 type 94 len 92

Issue with EBS Raid hanging on EBS volume creation

I've been using this cookbook for over a year now and have yet to run into any issues. This is my first attempt at a RAID array created with this cookbook.

It seems that it's just hanging waiting for /dev/xvdc to be mounted but it's never creating the ebs volumes in AWS.

I've tested that I can create single ebs volumes without issue on the same system with the same cookbook. I've let this run for about 2 hours and it's never gotten any further.

Below is the output from an strace on the chef worker

root@box1:/home/ubuntu# strace -p 10177
Process 10177 attached - interrupt to quit
restart_syscall(<... resuming interrupted call ...>) = -1 ETIMEDOUT (Connection timed out)
futex(0x1a57618, FUTEX_WAKE_PRIVATE, 1) = 0
clock_gettime(CLOCK_MONOTONIC, {976, 573213306}) = 0
stat("/dev/xvdc", 0x7fff2333b4b0) = -1 ENOENT (No such file or directory)
clock_gettime(CLOCK_MONOTONIC, {976, 573604612}) = 0
clock_gettime(CLOCK_MONOTONIC, {976, 573730133}) = 0
futex(0x1a575b4, FUTEX_WAIT_BITSET_PRIVATE, 25, {986, 573730133}, ffffffff) = -1 ETIMEDOUT (Connection timed out)
futex(0x1a57618, FUTEX_WAKE_PRIVATE, 1) = 0
clock_gettime(CLOCK_MONOTONIC, {986, 574298508}) = 0
stat("/dev/xvdc", 0x7fff2333b4b0) = -1 ENOENT (No such file or directory)
clock_gettime(CLOCK_MONOTONIC, {986, 574664859}) = 0
clock_gettime(CLOCK_MONOTONIC, {986, 574789903}) = 0
futex(0x1a575b4, FUTEX_WAIT_BITSET_PRIVATE, 27, {996, 574789903}, ffffffff) = -1 ETIMEDOUT (Connection timed out)
futex(0x1a57618, FUTEX_WAKE_PRIVATE, 1) = 0
clock_gettime(CLOCK_MONOTONIC, {996, 575247390}) = 0
stat("/dev/xvdc", 0x7fff2333b4b0) = -1 ENOENT (No such file or directory)

Any info/advice would be greatly appreciated.

Thanks.

Volumes recipe assumes there's at least one pre-existing /dev/xvd?

On a fresh instance-store EC2 instance with ami-74199744 in us-west-2, I configured one EBS volume to be mounted. The recipe blew up at line 4's Dir.glob('/dev/xvd?').sort.last[-1,1].succ because last returned nil (because no existing device matched the glob). I patched it with this:

diff --git a/cookbooks/ebs/recipes/volumes.rb b/cookbooks/ebs/recipes/volumes.rb
index e6a7503..edbfbb2 100644
--- a/cookbooks/ebs/recipes/volumes.rb
+++ b/cookbooks/ebs/recipes/volumes.rb
@@ -1,7 +1,8 @@
 node[:ebs][:volumes].each do |mount_point, options|
   if !options[:device] && options[:size]
     credentials = Chef::EncryptedDataBagItem.load('credentials', 'aws')
-    devid = Dir.glob('/dev/xvd?').sort.last[-1,1].succ
+    existing_xvdevices = Dir.glob('/dev/xvd?')
+    devid = existing_xvdevices.empty? ? 'f' : existing_xvdevices.sort.last[-1,1].succ
     device = "/dev/sd#{devid}"

     vol = aws_ebs_volume device do

If you like I can put that into a pull request. Let me know. If you think there's a better way, please let me know.

Support for gp2 volumes

This cookbook supports piops and standard volumes, but EC2 now supports general purpose SSD volumes with a type of gp2.

Thoughts: add volume_type to node[:ebs][:volumes] options. If not set, use the current logic. If set, accept the volume_type.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.