Code Monkey home page Code Monkey logo

Comments (5)

mindcrusher11 avatar mindcrusher11 commented on August 10, 2024

I am also looking for solution to save model using scala spark

from spark-knn.

Sambor123 avatar Sambor123 commented on August 10, 2024

I also had this problem,does it any solution for it?

from spark-knn.

rachmaninovquartet avatar rachmaninovquartet commented on August 10, 2024

I've tried like this:
sc.parallelize(Seq(knnModel), 1).saveAsObjectFile("/user/you/knnTest/" + "KNN")
val model = sc.objectFile[KNNClassificationModel]("/user/you/knnTest/" + "KNN").first()

but the model pulled back in no longer seems to work, which is strange since this has worked for all my other models.

from spark-knn.

wzjmail avatar wzjmail commented on August 10, 2024

I also encountered this problem. I attempted to serialize this model and load again. but rdd[tree] cannot be deserialized correctly. it looks like that metricTree have some problem. if you have sollution,comment please

from spark-knn.

alexnb avatar alexnb commented on August 10, 2024

Also saving using PipelineModel.save() does not work:

Caused by: java.lang.UnsupportedOperationException: Pipeline write will fail on this Pipeline because it contains a stage which does not implement Writable. Non-Writable stage: knnc_95d9ce15f990 of type class org.apache.spark.ml.classification.KNNClassificationModel
at org.apache.spark.ml.Pipeline$SharedReadWrite$$anonfun$validateStages$1.apply(Pipeline.scala:231)
at org.apache.spark.ml.Pipeline$SharedReadWrite$$anonfun$validateStages$1.apply(Pipeline.scala:228)
at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:186)
at org.apache.spark.ml.Pipeline$SharedReadWrite$.validateStages(Pipeline.scala:228)
at org.apache.spark.ml.PipelineModel$PipelineModelWriter.<init>(Pipeline.scala:336)
at org.apache.spark.ml.PipelineModel.write(Pipeline.scala:320)
at org.apache.spark.ml.util.MLWritable$class.save(ReadWrite.scala:306)
at org.apache.spark.ml.PipelineModel.save(Pipeline.scala:293)
... 16 more

from spark-knn.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.