esipfed / eskg Goto Github PK

Earth Science Knowledge Graph - An Automatic Approach to Building Earth Science Knowledge Graph to Improve Data Discovery

License: Apache License 2.0

Java 100.00%

earth-science esip knowledge-graph knowledge-discovery extraction semantic-web semantic-data esip-lab

eskg's Introduction

eskg

Introduction

Earth Science Knowledge Graph - An Automatic Approach to Building Earth Science Knowledge Graph to Improve Data Discovery.

Big Earth observation data have been produced, archived and made available online, but discovering the right data in a manner that precisely and efficiently satisfies user need presents a significant challenge to the Earth Science (ES) community. An emerging trend in information retrieval community is to utilize knowledge graph to assist user fast finding desired information. This is particularly prevalent within the fields of social media and complex multimodal information processing to name but a few.

However, building a domain-specific knowledge graph is labour-intensive and hard to keep up-to-date. We propose an automatic approach to building a dynamic knowledge graph for ES to improve data discovery by leveraging implicit, latent existing knowledge present within the Web Pages of NASA DAACs websites. This project will strengthen ties between observations and user communities by:

developing a knowledge graph derived from Web Pages via natural language processing and knowledge extraction techniques, and
allowing users to traverse, explore, query, reason and navigate ES data via knowledge graph interaction.

Installation

The prerequisites are

Java SE Development Kit 8
Apache Maven 3.X Once installed, you should be able to confirm using the following command

$ mvn -version

...

Apache Maven 3.3.9 (bb52d8502b132ec0a5a3f4c09453c07478323dc5; 2015-11-10T08:41:47-08:00)
Maven home: /usr/local/Cellar/maven/3.3.9/libexec
Java version: 1.8.0_131, vendor: Oracle Corporation
Java home: /Library/Java/JavaVirtualMachines/jdk1.8.0_131.jdk/Contents/Home/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "mac os x", version: "10.11.6", arch: "x86_64", family: "mac"

As you can see from above the output displays JDK 1.8.0_131 and Maven 3.3.9. You should also download and install this code

$ git clone https://github.com/ESIPFed/apache-semtech.git && cd apache-semtech
$ mvn clean install

The above task will install the code locally so you can try out the examples below.

Running ESKG

Creation of an ESKG dataset can be achieved by executing the main method of PODAAC Web Services Client. This will build a local Ontology model which, by default, can be found at target/classes/podaacDatasets.ttl.

Conveniently, this can be executed from the command line as follows

$ mvn exec:java -Dexec.mainClass="org.esipfed.eskg.aquisition.PODAACWebServiceClient"

ESKG Dataset

The canonical, current ESKG dataset is hosted at the ESIP Community Ontology Repository.

The dataset URI is http://cor.esipfed.org/ont/eskg/PODAACDataset

Acknowledgements

ESKG was initially conceived and funded through the ESIP Testbed initiative. ESIP funding acknowledged.

Community

License

ESKG is licensed permissively under the Apache License v2.0 a copy of which ships with this source code.

eskg's People

Contributors

Stargazers

Watchers

Forkers

ecoblockchain lewismc yongyao wangcug shubhampachori12110095 hkcaesar

eskg's Issues

README link dead?

Should a data source be presented at this website, as is mentioned in the readme?

The dataset URI is http://cor.esipfed.org/ont/eskg/

Fix incorrect values for individual 'hasSummary' element

The following value for each Individual hasSummary entry needs to be fixed.

    <j.0:hasSummary rdf:datatype="java:javax.xml.bind.JAXBElement"
    >javax.xml.bind.JAXBElement@383cd5d8</j.0:hasSummary>

Migrate OntoUploader.java code to ESKG

@Yongyao can you please import the code from https://github.com/mudrod/mudrod/blob/MUDROD-33/core/src/main/java/esiptestbed/mudrod/ontology/eskg/OntoUploader.java to here under a new package dataflow. We an build on it from there. Thanks.

Refactor OntoUploader into an Interface

The OntoUploader.java code should be refactored into an Interface with individual implementations for both ESIP COR AND ESIP Semantic Portal. The implementation should be driven by a configuration file which is read from src/main/resources via an InputStream. All credentials for interfacing with the various ontology repositories should also then be added there as well.

Forbidden API Usage in OntoUploader

When I attempt to build master branch I get the following

[INFO] --- forbiddenapis:1.8:check (default) @ eskg ---
[INFO] Scanning for classes to check...
[INFO] Reading bundled API signatures: jdk-unsafe
[INFO] Reading bundled API signatures: jdk-deprecated
[INFO] Reading bundled API signatures: jdk-system-out
[INFO] Loading classes to check...
[INFO] Scanning for API signatures and dependencies...
[ERROR] Forbidden method invocation: java.io.InputStreamReader#<init>(java.io.InputStream) [Uses default charset]
[ERROR]   in org.esipfed.eskg.portalservice.OntoUploader (OntoUploader.java:27)
[ERROR] Forbidden method invocation: java.lang.Throwable#printStackTrace() [Eclipse auto-generated stubs; exceptions should be correctly bubbled up and handled accordingly]
[ERROR]   in org.esipfed.eskg.portalservice.OntoUploader (OntoUploader.java:33)
[ERROR] Forbidden method invocation: java.lang.String#getBytes() [Uses default charset]
[ERROR]   in org.esipfed.eskg.portalservice.OntoUploader (OntoUploader.java:59)
[ERROR] Forbidden method invocation: java.io.InputStreamReader#<init>(java.io.InputStream) [Uses default charset]
[ERROR]   in org.esipfed.eskg.portalservice.OntoUploader (OntoUploader.java:73)
[ERROR] Forbidden method invocation: java.lang.Throwable#printStackTrace() [Eclipse auto-generated stubs; exceptions should be correctly bubbled up and handled accordingly]
[ERROR]   in org.esipfed.eskg.portalservice.OntoUploader (OntoUploader.java:81)
[ERROR] Forbidden method invocation: java.io.InputStreamReader#<init>(java.io.InputStream) [Uses default charset]
[ERROR]   in org.esipfed.eskg.portalservice.OntoUploader (OntoUploader.java:129)
[ERROR] Forbidden method invocation: java.lang.Throwable#printStackTrace() [Eclipse auto-generated stubs; exceptions should be correctly bubbled up and handled accordingly]
[ERROR]   in org.esipfed.eskg.portalservice.OntoUploader (OntoUploader.java:137)
[ERROR] Forbidden field access: java.lang.System#out [prints to System.out; should only be used for debugging, not in production code]
[ERROR]   in org.esipfed.eskg.portalservice.OntoUploader (OntoUploader.java:145)
[ERROR] Forbidden field access: java.lang.System#out [prints to System.out; should only be used for debugging, not in production code]
[ERROR]   in org.esipfed.eskg.portalservice.OntoUploader (OntoUploader.java:150)
[ERROR] Scanned 5 (and 292 related) class file(s) for forbidden API invocations (in 0.10s), 9 error(s).
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1.623 s
[INFO] Finished at: 2017-03-29T18:59:04-07:00
[INFO] Final Memory: 25M/255M
[INFO] ------------------------------------------------------------------------

PR coming soon.

Implement GeoSPARQL namespace prefixes and Classes in PODAACOntologyMapper

The OGC's GeoSPARQL standard provides us with the ability to embed semantics in Individual instances of PODAACDataset. We should utilize these for representing spatial objects and geographic features which occur within PO.DAAC dataset metadata.

Would this be useful?

A colleague came across this, not sure if it would be useful. Haven't tried it.
http://sualeh.github.io/SchemaCrawler/

Implement PODAACOntologyMapper.java

In order to pass an Ontology/OWL File to OntoUploader we need to implement the PODAACOntologyMapper. PR coming up.

Feed Crawler triples into ESIP SemanticPortal

Since #1 we have a basic crawler implementation however right now no triples are extracted from the crawl data. We need to run Any23's SingleDocumentExtraction over each page in order to extract structure before streaming the content into one of the following two Semantic Repositories.

Community Ontology Repository - http://cor.esipfed.org/

Semantic Portal - http://semanticportal.esipfed.org/ontologies

Get OpenIE running as an Any23 extractor

We could greatly improve the Any23 library by having OpenIE run as an extractor and generating triples. One of us should look into hooking this up such that we can use the core Any23 API's conveniently within ESKG.

Maven install issue

After I downloaded the project and ran it with "mvn clean install", it gave me the error below (I am using mac).

[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 02:25 min
[INFO] Finished at: 2017-02-08T12:09:21-05:00
[INFO] Final Memory: 12M/320M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project eskg: Could not resolve dependencies for project org.esipfed:eskg:jar:1.0-SNAPSHOT: Could not find artifact org.apache.any23.plugins:apache-any23-basic-crawler:jar:2.0-SNAPSHOT -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]

Also, please see my commit here.
Yongyao@681aadb

mvn install issue

After I pulled the code to my local machine, and run "mvn clean install". The system gives me the following error.

[ERROR] Failed to execute goal on project eskg: Could not resolve dependencies for project org.esipfed:eskg:jar:1.0-SNAPSHOT: The following artifacts could not be resolved: org.allenai.openie:openie_2.11:jar:4.2.6, org.allenai.openie:openie_2.11:pom:4.2.6: Could not find artifact org.allenai.openie:openie_2.11:jar:4.2.6 in central (https://repo.maven.apache.org/maven2) -> [Help 1]

Implement initial basic crawler for extracting structred relations from PO.DAAC

As discussed at our first meeting, we should use the Any23 basic-crawler to implement a lightweight crawler which will run Any23 extractions over Webpages.
I will link a WIP implementation which we can build on.