Code Monkey home page Code Monkey logo

hermes's People

Contributors

aganeshlab41 avatar agude avatar bethke avatar lgensinger avatar ymt123 avatar

Stargazers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Forkers

agude bethke tiffapedia pcallier anukat2015 codeaudit lgensinger fototo rwzhao lomascolo veterun sandy4321 techscientist containerz shadrack4292 vyraun aganeshlab41 scpei chirayukong strategist922 kylemvz solertis littlewizardli allensmile fesp21 tony32769 epopova angelalali phillette matt783 medhassno1 jbdatascience pradeepkr12 mydlprojects swapnilmaddula upto-400mg-caffeine afcarl zxshinxz sugardove rawsharkyu zn16 haritaparikh llhwjj databill86 salusphilip worry1613 abhishek9sharma oiclid maciejmacko cacan rrmudano henrypoter ls039 hemza-zarzouni classicvalues wasral liaopsh

hermes's Issues

`randomSplit()` produces incorrect results for the Git Dataset on Spark 1.4

The user vectors for the Git dataset are being "mangled" by this line:

hermes/src/hermes_run_script.py

Line 68 in 0fd69ed

train_ratings, test_ratings = user_info.randomSplit([0.9,0.1], 41)

If you recombine the test and train vectors, they new combined RDD has the right number of rows, but the number of distinct users and items changes. It looks like some duplicate rows are introduced and some rows are dropped.

This problem does not seem to effect any other datasets (we tested them all), and is not present when run locally on my laptop using Spark 1.6. It only shows up when running on the cluster with Spark 1.4.

We have temporarily worked around this problem by saving the test and training vectors on my laptop and moving them to the cluster.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.