crakjie / sbt-spark-plugin Goto Github PK

sbt clean package submit
.....
java.lang.ClassNotFoundException: StreamingApp
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:278)
at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:633)
at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:169)
at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:192)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:111)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)

The problem comes from SparkPlugin.scala:
val cmd = sparkHome.value+"/bin/spark-submit --class StreamingApp .....

The plugin looks great, but it doesn't work on my windows machine, the error is:

java.io.IOException: Cannot run program "C:\spark-1.5.0-bin-hadoop2.6/bin/spark-submit": 
CreateProcess error=193, %1 no es una aplicación Win32 válida

My HADOOP_HOME variable is set to C:\spark-1.5.0-bin-hadoop2.6

I think it may be related to the different path delimiters, I tried to use '/', but didn't work.

Mode could be by exemple : Test, Dev, PreProd , Prod

Here an example of the use of :run.mode=development

This option could be set in the build.sbt or set via submit command.

If I add these lines to the build.sbt file, it doesn't show the command line being launched

import sbt._

submitLogLevel := Level.Debug