giladkatz / explorekit Goto Github PK

Was able to run ExploreKit against the ionosphere data to completion but can't get it to run to completion against my own data: 327 Categorical Attributes, 1 Categorical Class (Binary), 300000 instances.

In order to test hunches I've converted 327 Categorical attributes into 735 numerical attributes, 1 Categorical Class (Binary) and 10K (10000) instances.

Receiving the following sort of Exception for 300K and 150K instances.
Ran the JVM with access to all of my 64Gb of Ram so it's not system memory related:

num of attributes: 735
num of instances: 10000
Starting to run classifier Thu Dec 07 17:01:31 GMT 2017
Starting to process classification results Thu Dec 07 17:01:41 GMT 2017
Starting TPR/FPR calculations : Thu Dec 07 17:01:41 GMT 2017
Done : Thu Dec 07 17:01:41 GMT 2017
Done Thu Dec 07 17:01:41 GMT 2017
Starting to run classifier Thu Dec 07 17:01:41 GMT 2017
Starting to process classification results Thu Dec 07 17:01:50 GMT 2017
Starting TPR/FPR calculations : Thu Dec 07 17:01:50 GMT 2017
Done : Thu Dec 07 17:01:50 GMT 2017
Done Thu Dec 07 17:01:50 GMT 2017
Starting to run classifier Thu Dec 07 17:01:50 GMT 2017
Starting to process classification results Thu Dec 07 17:01:58 GMT 2017
Starting TPR/FPR calculations : Thu Dec 07 17:01:58 GMT 2017
Done : Thu Dec 07 17:01:58 GMT 2017
Done Thu Dec 07 17:01:58 GMT 2017
Starting to run classifier Thu Dec 07 17:01:58 GMT 2017
Starting to process classification results Thu Dec 07 17:02:05 GMT 2017
Starting TPR/FPR calculations : Thu Dec 07 17:02:05 GMT 2017
Done : Thu Dec 07 17:02:05 GMT 2017
Done Thu Dec 07 17:02:05 GMT 2017
Experiment Start Date/Time: Thu Dec 07 17:02:14 GMT 2017
Starting TPR/FPR calculations : Thu Dec 07 17:02:21 GMT 2017
Done : Thu Dec 07 17:02:21 GMT 2017
Initial score: 0.5996622694925404 : Thu Dec 07 17:02:22 GMT 2017
Producing initial classification results : Thu Dec 07 17:02:22 GMT 2017
Starting to run classifier Thu Dec 07 17:02:56 GMT 2017
Starting to process classification results Thu Dec 07 17:03:05 GMT 2017
Starting TPR/FPR calculations : Thu Dec 07 17:03:05 GMT 2017
Done : Thu Dec 07 17:03:05 GMT 2017
Done Thu Dec 07 17:03:05 GMT 2017
Starting to run classifier Thu Dec 07 17:03:05 GMT 2017
Starting to process classification results Thu Dec 07 17:03:14 GMT 2017
Starting TPR/FPR calculations : Thu Dec 07 17:03:14 GMT 2017
Done : Thu Dec 07 17:03:14 GMT 2017
Done Thu Dec 07 17:03:14 GMT 2017
Starting to run classifier Thu Dec 07 17:03:14 GMT 2017
Starting to process classification results Thu Dec 07 17:03:23 GMT 2017
Starting TPR/FPR calculations : Thu Dec 07 17:03:23 GMT 2017
Done : Thu Dec 07 17:03:23 GMT 2017
Done Thu Dec 07 17:03:23 GMT 2017
Starting to run classifier Thu Dec 07 17:03:23 GMT 2017
Starting to process classification results Thu Dec 07 17:03:30 GMT 2017
Starting TPR/FPR calculations : Thu Dec 07 17:03:30 GMT 2017
Done : Thu Dec 07 17:03:30 GMT 2017
Done Thu Dec 07 17:03:30 GMT 2017
.....done Thu Dec 07 17:03:30 GMT 2017
Starting to apply unary operators: : Thu Dec 07 17:03:30 GMT 2017
num of attributes to evaluate: 1468
Error in ML score generation : 530
Error in ML score generation : 530
java.lang.ArrayIndexOutOfBoundsException: 530
at weka.core.DenseInstance.value(DenseInstance.java:347)
at weka.core.AbstractInstance.isMissing(AbstractInstance.java:319)
at weka.classifiers.trees.RandomTree$Tree.distributionForInstance(RandomTree.java:1048)
at weka.classifiers.trees.RandomTree.distributionForInstance(RandomTree.java:823)
at weka.classifiers.meta.Bagging.distributionForInstance(Bagging.java:819)
at weka.classifiers.evaluation.Evaluation.evaluationForSingleInstance(Evaluation.java:1963)
at weka.classifiers.evaluation.Evaluation.evaluateModelOnceAndRecordPrediction(Evaluation.java:2001)
at weka.classifiers.evaluation.Evaluation.evaluateModel(Evaluation.java:1877)
at weka.classifiers.Evaluation.evaluateModel(Evaluation.java:671)
at explorekit.Evaluation.FilterEvaluators.MLFilterEvaluator.produceScore(MLFilterEvaluator.java:114)
at explorekit.operators.OperatorsAssignmentsManager.EvaluateAttributeUsingTrainingSubFolds(OperatorsAssignmentsManager.java:307)
at explorekit.operators.OperatorsAssignmentsManager.lambda$generateAttributeAndCalculateFilterEvaluatorScore$0(OperatorsAssignmentsManager.java:218)
at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1380)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.ForEachOps$ForEachTask.compute(ForEachOps.java:291)
at java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:731)
at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)

@giladkatz : I am trying to use the algorithm on one of my models but facing technical issues.

What platform was used to run the java code of ExploreKit ?

It's 2020, so I'm sure the Java language has changed quite a bit. Any chance you can provide some instructions on how to get this code working again?

Hi,
I'm trying to compile ExploreKit on windows 7 using eclipse and I get the following error:

'The import com.sun.applet2 cannot be resolved WrapperEvaluator.java /ExploreKit-master (1)_ExploreKit-master/src/main/java/explorekit/Evaluation/WrapperEvaluation line 3'

I've done some googling, installed an older version of java sdk (that seem to have applet2), etc., but can't find a solution. I'm a newbie with java compilation so I may be missing something obvious.

Any help would be greatly appreciated,
Gavin.