Comments (9)
Looks like you need to also specify --cluster-ami
to an AMI that is in eu-west-1
from eggo.
ok - cool - that brings me a step ahead :
using now:
eggo-cluster provision --region eu-west-1 --stack-name eggo --launcher-instance-type m3.medium --availability-zone eu-west-1a --launcher-ami ami-8cfc51fb --cluster-ami ami-78d29c0f -n 4
[52.xx.xx..94] Executing task 'install_private_key'
[52.xx.xx..94] put: /home/ec2-user/Nunhem-Hadoop-keypair.pem -> id.pem
[52.xx.xx..94] run: chmod 600 id.pem
[52.xx.xx..94] Executing task 'run_director_bootstrap'
[52.xx.xx..94] put: -> director.conf
[52.xx.xx..94] run: cloudera-director bootstrap director.conf
[52.xx.xx..94] out: Process logs can be found at /home/ec2-user/.cloudera-director/logs/application.log
[52.xx.xx..94] out: Plugins will be loaded from /var/lib/cloudera-director-client/plugins
[52.xx.xx..94] out: Cloudera Director 1.5.1 initializing ...
[52.xx.xx..94] out: Installing Cloudera Manager ...
[52.xx.xx..94] out: * Starting ...... done
[52.xx.xx..94] out: * Requesting an instance for Cloudera Manager ................................. done
[52.xx.xx..94] out: * Inspecting capabilities of 10.1.1.122 ........... done
[52.xx.xx..94] out: * Installing screen package (1/1) ......... done
[52.xx.xx.94] out: * Running custom bootstrap script on 10.1.1.122 ........ done
[52.xx.xx..94] out: * Waiting for SSH access to 10.1.1.122 on port 22 ...... done
[52.xx.xx..94] out: * Inspecting capabilities of 10.1.1.122 ....... done
[52.xx.xx.94] out: * DNS is not configured correctly on at least one instance. Both forward and reverse DNS resolution must work on all instances in order for Cloudera Manager and cluster services to work properly. Please check the DNS configuration of your environment and instance images. Misconfigured instance: PluggableComputeInstance{ipAddress=Optional.of(10.1.1.122), delegate=null} Instance{virtualInstance=VirtualInstance{id='3c4179e7-cc1d-4eeb-97c9-8fa4033b88b4', template=InstanceTemplate{name='manager', type='d2.xlarge', image='ami-78d29c0f', bootstrapScriptIsPresent=true, config={instanceNamePrefix=bdg-eggo-node, rootVolumeSizeGB=100, keyName=Nunhem-Hadoop-keypair, subnetId=subnet-935505e4, securityGroupsIds=sg-9351e3f7}, tags={application=Cloudera Manager 5, eggo_node_type=manager, owner=ec2-user, ec2_key_pair=xxxxxx-Hadoop-keypair, eggo_stack_name=eggo}, normalizeInstance=true, sshUsername=Optional.absent()}}, capabilities=Optional.of(Capabilities{operatingSystemType=REDHAT_COMPATIBLE, operatingSystemVersion=REDHAT_COMPATIBLE_6, virtualizationType=HARDWARE_ASSISTED, packageManager=Optional.of(YUM), javaVendor=Optional.absent(), javaVersion=Optional.absent(), pythonVersion=Optional.of(2.6.6), passwordlessSudoEnabled=true, selinuxEnabled=true, iptablesEnabled=false, dnsConfigured=false, clouderaManagerAgentInstalled=false})} ...
[52.xx.xx.94] out:
Fatal error: run() received nonzero return code 22 while executing!
Requested: cloudera-director bootstrap director.conf
Executed: /bin/bash -l -c "cloudera-director bootstrap director.conf"
Aborting.
[ec2-user@ip-172-xx-xx-26 ~]$
from eggo.
Could you put all the CLI output into a github gist and post the link? Also, could you try to get the Cloudera Director log file? There is an eggo command that might work even at that failure point:
eggo-cluster get_director_log --region eu-west-1 --stack-name eggo
If that fails, you can try ssh-ing into the launcher node manually and dnloading this file:
/home/ec2-user/.cloudera-director/logs/application.log
If you get that file, could you put its contents into a github gist as well?
Either delete any sensitive info, or also feel free to email me directly if you don't want to broadcast it publicly. (Though I will forward it on to Director engineers)
from eggo.
Actually, before you do that, could you modify the cloudformation.template
file?
from eggo.
You'll notice that provision
also takes a --cf-template-path
option, which defaults to the template in the repo. (You can see the path by passing -h
to provision
.) It appears that there is a us-east-...
hardcoded into it, which is possibly causing the DNS problem.
from eggo.
Looks like the provided aws.conf
file is ok though and properly interpolates the region that you give.
from eggo.
I updated the template. There was one Default for AZ initializing it to us-east-1b. Switched that to "eu-west-1a" - same issue. D/L the application log. Mail sent to your cloudera account
from eggo.
found the answer here: http://www.cloudera.com/content/www/en-us/documentation/director/latest/topics/director_troubleshoot.html
If you're using AmazonProvidedDNS in us-east-1, specify ec2.internal. If you're using AmazonProvidedDNS in another region, specify region.compute.internal (for example, ap-northeast-1.compute.internal).
Modified the cloud Formation script ...
from eggo.
k, closing this. To be followed up in #134
from eggo.
Related Issues (20)
- 1000 Genomes Phase 3 VCF data set to be hosted in S3 in parquet format HOT 4
- ENCODE data set to be hosted in S3 in parquet format
- Reduce number of CM/CDH restarts during cluster config
- Platinum Genomes data set
- dnload_raw operation requires the ability to `sudo -u hdfs` HOT 2
- `dnload_raw` shouldn't hardcode `ec2-user` when it `chown`s data
- Extract Hive queries into separate reusable files
- asking for password HOT 4
- AMI in eu-west-1 ? HOT 6
- Default parallelism set to 2 on cluster with 4 executor nodes? HOT 2
- Verify that requested AMIs are consistent with the requested region/AZ HOT 1
- us-east-1b is hardcoded into cloudformation.template HOT 2
- eggo-cluster provision - fails with return code 12 while executing HOT 1
- Remove minCount line from aws.conf
- how to set the instance type for master, manager and workers separately ? HOT 1
- No handlers could be found for logger "cm_api.resource" HOT 2
- results of "genotype count" are different - depending on used programs HOT 2
- sql query on Adam's flattened file -> Non-local session path expected to be non-null; HOT 4
- Upgrade Spark from 1.3 to 1.5 to make use of project Tungsten performance improvements. HOT 6
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from eggo.