astralway / phrasecount Goto Github PK

View Code? Open in Web Editor NEW

4.0 4.0 5.0 157 KB

Example Apache Fluo application that counts phrases in documents

License: Apache License 2.0

Java 93.71% Shell 6.29%

phrasecount's People

Contributors

Stargazers

Watchers

Forkers

mikewalch keith-turner joshelser ericnewton wjsl

phrasecount's Issues

Counts in print command are zero

I just ran through the example and noticed that the counts from the Print command are zero

Create dev branch and use latest MiniFluo code

Master should still use fluo alpha but dev branch could starting using changes from apache/fluo#318

Create 1.0.0-alpha-1 tag

This will allow this example to be able to be used with alpha release of Fluo over time.

url and group id in pom are wrong

Add release information to root pom so that release artifacts can be staged in Sonatype OSS

NPE in PhraseExporter

java.lang.NullPointerException: null
        at phrasecount.PhraseExporter.process(PhraseExporter.java:87) ~[phrasecount-0.0.1-SNAPSHOT.jar:na]
        at accismus.impl.Worker.processUpdates(Worker.java:174) ~[accismus-core-1.0.0-alpha-1-SNAPSHOT.jar:na]
        at accismus.impl.WorkerTask.run(WorkerTask.java:67) [accismus-core-1.0.0-alpha-1-SNAPSHOT.jar:na]
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [na:1.7.0_55]
        at java.util.concurrent.FutureTask.run(FutureTask.java:262) [na:1.7.0_55]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_55]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_55]
        at java.lang.Thread.run(Thread.java:745) [na:1.7.0_55]

Use fluo exec command

Make scripts use new command introduced in apache/fluo#581

Error Message Parameter Possibly Incorrect

phrasecount/src/main/java/phrasecount/cmd/Load.java

Line 32 in 7e301ab

System.out.println("Text file dir does not exist: " + args[0]);

I'm just eyeballing here, but it looks like the line above should reference args[1] instead of the args[0].

Create scripts for phrasecount to be run by fluo-dev & fluo-deploy

Follow same scripts used by fluo-stress. Scripts should accept directory of text documents to index.

Failed when running on EC2

I tried to run phrasecount on EC2 with multiple zookeepers. However, it failed because phrasecount is using the old method (comma seperated key/value pairs) to configure observers.

Have TravisCI start building this project

fix typos in README.md

Reviewing the example and found a few minor typos.

Incorrect README elements for standalone Accumulo

A few errors in the README for running the example against a real instance of Accumulo.

Update to Fluo alpha1 and Accumulo 1.6.1

Possibly merge run.sh and run-cluster.sh

While reviewing #20 I noticed the run.sh and run-cluster.sh scripts have become very similar. Need to look into merging them. I can look into this while converting phrase count to use fluo-recipes.

Exception while runing mini.sh

I tried to run mini.sh using Fluo 1.2.0 and received the following exception:

java.lang.InterruptedException
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2048)
	at java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:403)
	at org.apache.fluo.core.impl.SharedBatchWriter$FlushTask.run(SharedBatchWriter.java:90)
	at java.lang.Thread.run(Thread.java:748)
[WARNING] 
java.lang.RuntimeException: java.lang.InterruptedException
	at org.apache.fluo.recipes.accumulo.export.function.AccumuloWriter$ExportTask.run(AccumuloWriter.java:106)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.InterruptedException
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2048)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at org.apache.fluo.recipes.accumulo.export.function.AccumuloWriter$ExportTask.run(AccumuloWriter.java:91)
	... 1 more
2018-02-28 14:04:11,492 [util.Halt] ERROR: Oracle has lost leadership unexpectedly and is now halting.