Comments (3)
Hello,
Thanks for using cloudsuite.
One reason why the benchmark is not doing any progress is that the master is not receiving any packets from the slaves.
To check, we can use yarn's resource manager webUI.
To do that, first publish the master's webUI port:
"docker run -d --net hadoop-net -p 8088:8088 --name master --hostname master cloudsuite/data-analytics master"
Then run the slaves with the same command.
After launching the benchmark script ("docker exec master benchmark"), open, in your browser "http://localhost:8088/". You can see the status of the application here.
For more information about the webUI you check: http://hadooptutorial.info/yarn-web-ui/
You might need to publish other ports from the master node. A full list of yarn list can be found here: https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.2/bk_reference/content/yarn-ports.html
Please let us know if this helps.
Regards,
Hussein
from cloudsuite.
Hi,
I can run the below cmd -
$ docker run -d --net hadoop-net -p 8088:8088 --name master --hostname master cloudsuite/data-analytics master
But after running the master when I try to run slave -
$ docker run -d --net hadoop-net -p 8088:8088 --name slave01 --hostname slave01
cloudsuite/hadoop slave
I received the below error -
docker: Error response from daemon: driver failed programming external connectivity on endpoint slave01 (44e3108976295660bc8efbcca546707eca865861926a43eb761778a28c2bb3bc): Bind for 0.0.0.0:8088 failed: port is already allocated.
Can you please help me.
from cloudsuite.
Hi~!
I want to test my cloud.
So, I try to CloudSuite.
But Data_Analytics isn't working.....
Data_Analytics benchmark show this message.
Mahout: seqwiki
MAHOUT_LOCAL is not set; adding HADOOP_CONF_DIR to classpath.
Running on hadoop, using /opt/hadoop-2.7.3/bin/hadoop and HADOOP_CONF_DIR=/opt/hadoop-2.7.3/etc/hadoop
MAHOUT-JOB: /opt/mahout-0.12.2/mahout-examples-0.12.2-job.jar
17/08/07 12:45:51 INFO WikipediaToSequenceFile: Input: /user/root/wiki Out: /user/root/wiki-seq Categories: /root/categories All Files: false
17/08/07 12:45:52 INFO RMProxy: Connecting to ResourceManager at master/10.0.9.2:8032
17/08/07 12:45:53 WARN JobResourceUploader: Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this.
17/08/07 12:45:54 INFO FileInputFormat: Total input paths to process : 1
17/08/07 12:45:54 INFO JobSubmitter: number of splits:5
17/08/07 12:45:55 INFO JobSubmitter: Submitting tokens for job: job_1502109896912_0001
17/08/07 12:45:55 INFO YarnClientImpl: Submitted application application_1502109896912_0001
17/08/07 12:45:55 INFO Job: The url to track the job: http://master:8088/proxy/application_1502109896912_0001/
17/08/07 12:45:55 INFO Job: Running job: job_1502109896912_0001And i can't see next step....
I already complete this step in other computer.
How can i solve this problem?
I want to see next step.... :(
Help me~!~!
Thank you!!
Hi,
Did you find any solution?
I got "Opportunistic container queue is full, l Container is killed before being launched" a few lines after that
thanks
from cloudsuite.
Related Issues (20)
- HHVM and Django left overs
- Better documentation how to run the workloads
- Add to the configuration parameters of the docker, the number of vcpus that will be allocated to Spark and Hadoop
- Media Streaming Should Have High-Resolution Video (1080p, 4K)
- In-Memory doc
- Data Analytics implementation questions HOT 10
- Media Streaming Multi-node HOT 4
- Ramp up workload intensity for Data-Caching and Data-Serving Benchmarks HOT 1
- media streaming dataset: Issue with running ffmpeg - timestamp out of range HOT 1
- Can I run cloudsuite benchmark on arm64 architecture directly by offical code? HOT 13
- Sysbench Meets Error When Running on ARM64 Platform
- In-memory analytics benchmark run question HOT 8
- media-streaming dataset error HOT 7
- Investigate CloudSuite web serving workload in QEMU
- Instructions for compiling for RISC-V ISA (Data-caching) HOT 3
- Starting the web-server for the the web-serving benchmark fails as rootless
- data-analytics: Native library missing for arm64 HOT 1
- data-analytics: failure to run benchmark on different hosts without proper DNS resolution of nodes
- data-caching: cache not warmed up?
- Data caching DNS Error HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from cloudsuite.