fusepoolp3 / p3-dictionary-matcher-transformer Goto Github PK
View Code? Open in Web Editor NEWDictionary Matcher is P3 transformer for SKOS based entity extraction.
License: Apache License 2.0
Dictionary Matcher is P3 transformer for SKOS based entity extraction.
License: Apache License 2.0
For example
With the query
$ curl -X POST -d "Frauds and Swindlings cause significant concerns with regards to Ethics." "http://sandbox.fusepool.info:8301/?taxonomy=http://data.nytimes.com/descriptors.rdf"
I get (extract)
<http://sandbox.fusepool.info:8301/ae85b2ab-4347-42a5-9758-edcb547dd635#annotation-body1>
a <http://vocab.fusepool.info/fam#LinkedEntity> ;
<http://vocab.fusepool.info/fam#extracted-from>
<http://sandbox.fusepool.info:8301/ae85b2ab-4347-42a5-9758-edcb547dd635> ;
<http://vocab.fusepool.info/fam#selector>
<http://sandbox.fusepool.info:8301/ae85b2ab-4347-42a5-9758-edcb547dd635#char=65,71> .
Dereferencing http://sandbox.fusepool.info:8301/ae85b2ab-4347-42a5-9758-edcb547dd635 does not return the text from which the annotations are extracted. Clearly it is not the business of the transformer to store the posted text, but the transformer shouldn't just make up HTTP URI that dereference to something quite unrelated. As no dereferenceable URI can be provided the transformer should use Blank Nodes or alternatively non dereferenceable URNs. The blank node approach is better as it means that annotating the same text with the same taxonomy generates the same result (i.e. an isomorphic graph)
If I have a taxonomy in Turtle I can't get the transformer to work:
[qtp933699219-15] ERROR com.hp.hpl.jena.rdf.model.impl.RDFDefaultErrorHandler - http://relative-uri.fake/(line 1 column 1): Content is not allowed in prolog.
[qtp933699219-15] WARN org.eclipse.jetty.servlet.ServletHandler - /
java.lang.RuntimeException: com.hp.hpl.jena.shared.JenaException: org.xml.sax.SAXParseException; systemId: http://relative-uri.fake/; lineNumber: 1; columnNumber: 1; Content is not allowed in prolog.
at eu.fusepool.p3.dictionarymatcher.Reader.readDictionary(Reader.java:92)
at eu.fusepool.p3.transformer.dictionarymatcher.DictionaryMatcherTransformer.<init>(DictionaryMatcherTransformer.java:124)
at eu.fusepool.p3.transformer.dictionarymatcher.Main$1.getTransformer(Main.java:46)
at eu.fusepool.p3.transformer.server.handler.TransformerFactoryServlet.getTransformer(TransformerFactoryServlet.java:126)
at eu.fusepool.p3.transformer.server.handler.TransformerFactoryServlet.handlePost(TransformerFactoryServlet.java:71)
at eu.fusepool.p3.transformer.server.handler.TransformerFactoryServlet.service(TransformerFactoryServlet.java:60)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:769)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:517)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:98)
at org.eclipse.jetty.server.Server.handle(Server.java:461)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:284)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244)
at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:534)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:607)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:536)
at java.lang.Thread.run(Thread.java:745)
Caused by: com.hp.hpl.jena.shared.JenaException: org.xml.sax.SAXParseException; systemId: http://relative-uri.fake/; lineNumber: 1; columnNumber: 1; Content is not allowed in prolog.
at com.hp.hpl.jena.rdf.model.impl.RDFDefaultErrorHandler.fatalError(RDFDefaultErrorHandler.java:58)
at com.hp.hpl.jena.rdf.arp.impl.ARPSaxErrorHandler.fatalError(ARPSaxErrorHandler.java:48)
at com.hp.hpl.jena.rdf.arp.impl.XMLHandler.warning(XMLHandler.java:209)
at com.hp.hpl.jena.rdf.arp.impl.XMLHandler.fatalError(XMLHandler.java:239)
at org.apache.xerces.util.ErrorHandlerWrapper.fatalError(Unknown Source)
at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source)
at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source)
at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source)
at org.apache.xerces.impl.XMLScanner.reportFatalError(Unknown Source)
at org.apache.xerces.impl.XMLDocumentScannerImpl$PrologDispatcher.dispatch(Unknown Source)
at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source)
at org.apache.xerces.parsers.DTDConfiguration.parse(Unknown Source)
at org.apache.xerces.parsers.DTDConfiguration.parse(Unknown Source)
at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source)
at com.hp.hpl.jena.rdf.arp.impl.RDFXMLParser.parse(RDFXMLParser.java:151)
at com.hp.hpl.jena.rdf.arp.JenaReader.read(JenaReader.java:168)
at com.hp.hpl.jena.rdf.arp.JenaReader.read(JenaReader.java:155)
at com.hp.hpl.jena.rdf.arp.JenaReader.read(JenaReader.java:226)
at com.hp.hpl.jena.rdf.model.impl.ModelCom.read(ModelCom.java:274)
at org.apache.clerezza.rdf.jena.parser.JenaParserProvider.parse(JenaParserProvider.java:68)
at org.apache.clerezza.rdf.core.serializedform.Parser.parse(Parser.java:240)
at org.apache.clerezza.rdf.core.serializedform.Parser.parse(Parser.java:193)
at eu.fusepool.p3.dictionarymatcher.Reader.readDictionary(Reader.java:36)
... 18 more
Caused by: org.xml.sax.SAXParseException; systemId: http://relative-uri.fake/; lineNumber: 1; columnNumber: 1; Content is not allowed in prolog.
at org.apache.xerces.util.ErrorHandlerWrapper.createSAXParseException(Unknown Source)
... 38 more
If the word in the taxonomy is used at the end of a sentence followed by a dot and no white space it does not seem to match it.
Taxonomy containing the term Brand (with capital B).
When using dictionary matcher, with parameter casesensitive false it only finds the term when written with Capital B. Dictionary matcher does not find it when the word is written with lowercase b.
Using this curl request:
curl -X POST -d "brand" "http://10.0.0.6:8301/?taxonomy=http://10.0.0.6/firebrary.rdf&stemming=dutch&casesensitive=false"
I`am playing around with the application. It seems to work well. Earlier (a few months) I tried the example of New York times and it worked. However the uris seems to be death at the moment. I trying to use one of my own SKOS ontologies. But I cannot figuring out how the dictionary matcher it can use.
In the example I see the there is a redirection towards a descriptors.rdf. The SKOS ontology I want to use does not have such descriptors file. Is this the missing link, should I add this file. And if so, according what specifications?
UPDATE
Found that if I give a uri to a concept, that the matcher finds this specific concept in the text. But I want to give more than only one concept. It seems I cannot refer to a concept-scheme or so. Where should I refer to if I want to use the whole ontology?
The matcher does not recognize alternative skos labels. If somebody uses a synonym for a specific word which we added as alternative label. The matcher is unable to find the word.
The URI https://web.archive.org/web/20151025232812/http://data.nytimes.com/descriptors.rdf is accepted by browsers, however the dictionary matcher gives an error
$ curl -X POST -d "Frauds and Swindlings cause significant concerns with regards to Ethics." "http://sandbox.fusepool.info:8301/?taxonomy=https://web.archive.org/web/20151025232812/http://data.nytimes.com/descriptors.rdf"
ERROR: Taxonomy URI is invalid! ("https://web.archive.org/web/20151025232812/http://data.nytimes.com/descriptors.rdf")
Hi,
I played around with the dictionary matcher, and I see that it loads the dictionary prior to each request/annotation task, right? Is there a way to persistently store the dictionary? I have a very huge file (see also: fusepool/fusepool-sma#6 ) and it would not be feasible to load it prior to each request.
BR, Johannes
Please update the default port on which the transformer starts according to https://github.com/fusepoolP3/overall-architecture/blob/master/default-ports.md
Having a look at the releases, I noticed that the name of the jar-file including all the dependencies does not start with p3-*... Simply close this issue if that's not important...
the try it out sections uses some example.org-URIs which of course do not dereference.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.