Code Monkey home page Code Monkey logo

Comments (4)

brkyvz avatar brkyvz commented on August 15, 2024

That is a known issue. The compilation error occurs, because 1.0's Decision Tree doesn't match 1.1's Decision Tree. As a work around, could you either run the onepointone tests or onepointoh and comment out the other one in "Build.scala"? Basically from the line below:
lazy val root = project.in(file(".")).aggregate(onepointoh,onepointone). dependsOn(onepointoh,onepointone)

from spark-perf.

tsailiming avatar tsailiming commented on August 15, 2024

Thanks. Managed to build it with the following changes:

--- a/mllib-tests/project/Build.scala
+++ b/mllib-tests/project/Build.scala
@@ -14,18 +14,18 @@ object MyBuild extends Build {
   // features from the root build. For example, in order to build with the Spark 1.0.0 release, 
   // comment out .dependsOn(onepointone) from root.

-    lazy val root = project.in(file(".")).aggregate(onepointoh,onepointone).
-        dependsOn(onepointoh,onepointone)
+    lazy val root = project.in(file(".")).aggregate(onepointone).
+        dependsOn(onepointone)

     lazy val onepointone = project
         .settings(
           libraryDependencies ++= deps,
-          libraryDependencies += "org.apache.spark" %% "spark-mllib" % "1.1.1-SNAPSHOT" % "provided"
+          libraryDependencies += "org.apache.spark" %% "spark-mllib" % "1.1.0" % "provided"
         )

     lazy val onepointoh = project
         .settings(        //should be set to 1.0.0 or higher
-          libraryDependencies += "org.apache.spark" %% "spark-mllib" % "1.0.0" % "provided",
+          libraryDependencies += "org.apache.spark" %% "spark-mllib" % "1.1.0" % "provided",
           libraryDependencies ++= deps
         )
 }

But looking at config/config.py, we need to change the test runner for mllib to onepointone: mllib.perf.onepointone.TestRunner ?

But by doing so, it seems that many tests such as Linear Regression, Lasso, etc do not exist?

from spark-perf.

jkbradley avatar jkbradley commented on August 15, 2024

I'm working on a major refactoring of the code, currently in [https://github.com/jkbradley/spark-perf/tree/qa-1-2] I'll push a PR once it's definitely stable.

from spark-perf.

mengxr avatar mengxr commented on August 15, 2024

I think the issue is resolved now with separated source folders.

from spark-perf.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.