(LDA)Example/LDADriver/ Job aborted due to stage failure: java.lang.ArrayIndexOutOfBoundsException: -6 about zen HOT 13 OPEN

My dataset is libsvm style:
0 0:3 1:2 2:10 ...
1 0:1 1:1 2:3 ...
.....

Is it a bug or something wrong with my dataset?

Thanks!

from zen.

Following the instructions from google, I enlarge the memory, however still failed. @witgo

from zen.

ping @bhoppi

from zen.

@bhoppi @hucheng Help!

from zen.

I can't get useful info from your log. Can you dig your spark log for more detail? And please share your cmd parameters.

from zen.

spark-submit --master yarn-client --class com.github.cloudml.zen.examples.ml.LDADriver
--jars ml/target/zen-ml_2.10-0.3-SNAPSHOT.jar
--executor-memory 6G --driver-memory 6G --num-executors 200 --executor-cores 1
--conf spark.driver.maxResultSize=6G
--conf spark.driver.extraJavaOptions="-XX:MaxPermSize=256m -XX:+CMSClassUnloadingEnabled -XX:+PrintGC -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintGCDateStamps -XX:+PrintHeapAtGC -XX:+PrintGCApplicationStoppedTime -XX:+PrintGCApplicationConcurrentTime -Xloggc:gc.log -XX:+HeapDumpOnOutOfMemoryError"
--conf spark.yarn.am.memory=2g
--conf spark.yarn.am.extraJavaOptions="-XX:MaxPermSize=256m -XX:+CMSClassUnloadingEnabled"
--conf spark.storage.memoryFraction=0.1
--conf spark.yarn.executor.memoryOverhead=6666
--conf spark.sql.shuffle.partitions=2000
--conf spark.executor.extraJavaOptions="-XX:MaxPermSize=256m -XX:+CMSClassUnloadingEnabled -XX:MaxDirectMemorySize=2048m -Xmn100m -XX:MaxTenuringThreshold=1 -XX:+UseConcMarkSweepGC -XX:+CMSParallelRemarkEnabled -XX:+UseCMSCompactAtFullCollection -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=10 -XX:+UseCompressedOops -XX:+PrintGC -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintGCDateStamps -XX:+PrintGCApplicationStoppedTime -XX:+PrintHeapAtGC -XX:+PrintGCApplicationConcurrentTime -Xloggc:gc.log"
examples/target/zen-examples_2.10-0.3-SNAPSHOT.jar
-numTopics=1000
-alpha=0.1
-beta=0.01
-alphaAS=0.01
-totalIter=50
-numPartitions=20
-useKryo=true
-ignoredocid=true
/user/distml/ldatest/input4
/user/distml/ldatest/output2

from zen.

Uploading log.txt…

from zen.

The log.txt is a little big, so a attched it in previous post. Tell me if you cannot see the file.

from zen.

Sorry I can't read the log file.

from zen.

gclog.txt
log.txt

from zen.

Upload done... @bhoppi

from zen.

@ylqfp Can you upload the container log? I can't still get the point from the master log.

from zen.

Dear Bhoppi,
Sorry for the late response!
I use yarn logs -applicationID to get container log, however got nothing.
Could you please tell me where to find the container log?
Thanks!

from zen.