kylemarkwilliams / maui-indexer Goto Github PK

Hi, 
I'm trying to do keyphrse extraction for my documents against LCSH skos  
dictionary (about 400 MB). The problem I'm experiencing is an out of memory...I 
had a look at the code and as far I see it seems the cause is the (huge) 
in-memory vocabulary object.
Is there a workaround for that? I tried to use a RDBMS-based persistent Jena 
model but I still have the mentioned problem when the module tries to build the 
vocabulary loading all the model statements...

Any thought?

Best regards,
Andrea

What steps will reproduce the problem?
1. Use maui as a sub-component of a much larger maven project
2. Discover maui and its parts (weka, etc) aren't in maven repos
3. Nor are version numbers provided
4. Frustration!
5. Limp along for now with eclipse export runnable jar

What is the expected output? What do you see instead?
Expect working maven pom.xml to build with "maven install"

What version of the product are you using? On what operating system?
Latest; 1.1 I believe

Please provide any additional information below.
Maven poms aren't optional; they're essential.

Upload sample models for different tasks and test them

What steps will reproduce the problem?
1. Install Wikipedia Miner english
2. Comment lines 272-280 (topic extraction)
3. Uncomment lines 262-270 (model build)
3. Execute maui.main.Examples indexing_with_wikipedia

Expect to build a model.
Throw exceptions. Cf. below.
It doesn't build the model. If I launch an extraction after, it doesn't work : 
-- Reading instance
-- Converting instance for document 0018
Warning! This documents does not contain valid keyphrases
---- Extracting candidates... 
---- Disambiguating candidates...
54 candidates 
0 positive; 54 negative instances
-- Processing document: 0018
-- Keyphrases and feature values:

What version of the product are you using? On what operating system?
Maui 1.0 on Windows XP JDK 1.6

Here's the exception:

java.sql.SQLException: Table 'wiki_db_en.anchor_occurance_casefolder'
doesn't exist
    at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:2975)
    at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1600)
    at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:1695)
    at com.mysql.jdbc.Connection.execSQL(Connection.java:2998)
    at com.mysql.jdbc.Connection.execSQL(Connection.java:2927)
    at com.mysql.jdbc.Statement.executeQuery(Statement.java:956)
    at org.wikipedia.miner.model.Anchor.initializeFromDatabase(Anchor.java:105)
    at org.wikipedia.miner.model.Anchor.<init>(Anchor.java:69)
    at maui.filters.MauiFilter.getCandidates(MauiFilter.java:1556)
    at maui.filters.MauiFilter.selectCandidates(MauiFilter.java:660)
    at maui.filters.MauiFilter.batchFinished(MauiFilter.java:626)
    at maui.main.MauiModelBuilder.buildModel(MauiModelBuilder.java:785)
    at maui.main.Examples.testIndexingWithWikipedia(Examples.java:269)
    at maui.main.Examples.main(Examples.java:319)
Error adding ngram approach

The complete trace is attached.

Thanks for publishing this application! I've been using KEA 5.0 in a pilot
project and am very impressed with the keywords it extracts from complex
biodiversity documents using AGROVOC.
We'd like to trial maui but we're not Java developers. Would it be possible
for you to provide a maui jar file - as you did in the KEA download ?

TIA muchly!

It might be useful to refactor extractKeyphrases so that it can accept an input 
stream rather than 
relying on files.  In this way it can be used to process text in-memory.  
Currently the only way to do 
that would be to pass contents through a fifo (not supported on a lot of 
platforms).

Thoughts?

Hi,

I have started to use Maui1.2 for the last two days. I read at some site that 
it is possible to use more than one vocabulary files at the same time. Can 
someone please tell me how can I use it?

I am using Maui1.2 with Eclipse.

Thanks

Fahmida 

Hi,

not sure this is the best place, please bear with me.

I just want to ask - how do I create training keys for indexing with wikipedia. 
Do I do the same thing as with automatic-tagging? but how do i ensure the 
keyword is consistent with wikipedia? 

Lastly, are the numbers beside the keyphrases sentence number? are they needed?

Hi,

I have just downloaded the file(maui1.2_all-incl.tar.gz) into eclipse.
I wanted to run Examples.java but it appeared this error code.

Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: 0
    at maui.main.Examples.main(Examples.java:293)

Any solutions?

Thanks

Regards,
Darryl