Code Monkey home page Code Monkey logo

web-karma's Introduction

Karma: A Data Integration Tool

travis ci


Feedback Request

We value your feedback!

We would greatly appreciate it if you could take a few minutes to provide us with your feedback on our application. Your input will help us improve and enhance the overall user experience.

Please click here to access the feedback form, and share your thoughts, suggestions, or report any issues you encountered.

Thank you for your support!


The Karma tutorial at https://github.com/szeke/karma-tcdl-tutorial, also check out our DIG web site, where we use Karma extensively to process > 90M web pages.

See our release stats

What is Karma?

Karma is an information integration tool that enables users to quickly and easily integrate data from a variety of data sources including databases, spreadsheets, delimited text files, XML, JSON, KML and Web APIs. Users integrate information by modeling it according to an ontology of their choice using a graphical user interface that automates much of the process. Karma learns to recognize the mapping of data to ontology classes and then uses the ontology to propose a model that ties together these classes. Users then interact with the system to adjust the automatically generated model. During this process, users can transform the data as needed to normalize data expressed in different formats and to restructure it. Once the model is complete, users can published the integrated data as RDF or store it in a database.

You can find useful tutorials on the project Website: http://www.isi.edu/integration/karma/

Installation and Setup

Look in the Wiki Installation

Frequently Asked Questions

How to perform offline RDF generation for a data source using a published model?

  1. Model your source and publish it's model (the published models are located at src/main/webapp/publish/R2RML/ inside the Karma directory).
  2. To generate RDF of a CSV/JSON/XML file, go to the top level Karma directory and run the following command from terminal:
cd karma-offline
mvn exec:java -Dexec.mainClass="edu.isi.karma.rdf.OfflineRdfGenerator" -Dexec.args="--sourcetype
<sourcetype> --filepath <filepath> --modelfilepath <modelfilepath> --sourcename <sourcename> --outputfile <outputfile> --JSONOutputFile<outputJSON-LD>" -Dexec.classpathScope=compile
Valid argument values for sourcetype are: CSV, JSON, XML. Also, you need to escape the double quotes that go inside argument values. Example invocation for a JSON file:
mvn exec:java -Dexec.mainClass="edu.isi.karma.rdf.OfflineRdfGenerator" -Dexec.args="
--sourcetype JSON
--filepath \"/Users/shubhamgupta/Documents/wikipedia.json\"
--modelfilepath \"/Users/shubhamgupta/Documents/model-wikipedia.n3\"
--sourcename wikipedia
--outputfile wikipedia-rdf.n3
--JSONOutputFile wikipedia-rdf.json" -Dexec.classpathScope=compile
  1. To generate RDF of a database table, go to the top level Karma directory and run the following command from terminal:
cd karma-offline
mvn exec:java -Dexec.mainClass="edu.isi.karma.rdf.OfflineRdfGenerator" -Dexec.args="--sourcetype DB
--modelfilepath <modelfilepath> --outputfile <outputfile> --dbtype <dbtype> --hostname <hostname>
--username <username> --password <password> --portnumber <portnumber> --dbname <dbname> --tablename <tablename> --JSONOutputFile<outputJSON-LD>" -Dexec.classpathScope=compile
Valid argument values for `dbtype` are Oracle, MySQL, SQLServer, PostGIS, Sybase. Example invocation:
mvn exec:java -Dexec.mainClass="edu.isi.karma.rdf.OfflineRdfGenerator" -Dexec.args="
--sourcetype DB --dbtype SQLServer
--hostname example.com --username root --password secret
--portnumber 1433 --dbname Employees --tablename Person
--modelfilepath \"/Users/shubhamgupta/Documents/db-r2rml-model.ttl\"
--outputfile db-rdf.n3
--JSONOutputFile db-rdf.json" -Dexec.classpathScope=compile

You can do mvn exec:java -Dexec.mainClass="edu.isi.karma.rdf.OfflineRdfGenerator" -Dexec.args="--help" to get information about required arguments.

How to set up password protection for accessing Karma?

  • in /src/main/config/jettyrealm.properties change user/password (if you wish)
  • in /src/main/webapp/WEB-INF/web.xml uncomment security section at the end of the file
  • in pom.xml uncomment security section (search for loginServices)

Are there additional steps required to import data from Oracle database?

Yes. Due to Oracles binary license issues, we can't distribute the JAR file that is required for importing data from an Oracle database. Following are the steps to resolve the runtime error that you will get if you try to do it with the current source code:

  1. Download the appropriate JDBC drive JAR file (for JDK 1.5 and above) that matches your Oracle DB version. Link: http://www.oracle.com/technetwork/database/features/jdbc/index-091264.html
  2. Put the downloaded JAR file inside lib folder of the Karma source code.
  3. Add the following snippet in the pom.xml file (present inside the top level folder of Karma source code) inside the dependencies XML element:
<dependency>
    <groupId>com.oracle</groupId>
    <artifactId>ojdbc</artifactId>
    <version>14</version>
    <scope>system</scope>
    <systemPath>/Users/karma/Web-Karma/lib/ojdbc14.jar</systemPath>
</dependency>

Make sure that the filename mentioned in the systemPath element matches with your downloaded JAR file; it is likely that your installation folder is different from /Users/karma so make sure you use the correct one.

Are there additional steps required to import data from MySQL database?

Yes. Due to MySQL binary license issues, we can't distribute the JAR file that is required for importing data from an MySQL database. Following are the steps to resolve the runtime error that you will get if you try to do it with the current source code:

  1. Download the appropriate MySQL driver JAR file (for JDK 1.5 and above) that matches your MySQL version. Link: http://dev.mysql.com/downloads/connector/j/
  2. Put the downloaded JAR file inside lib folder of the Karma source code.
  3. Add the following snippet in the pom.xml file of the karma-jdbc project inside the dependencies XML element:
<dependency>
    <groupId>mysql</groupId>
    <artifactId>mysql-connector-java</artifactId>
    <version>5.1.32</version>
    <scope>system</scope>
    <systemPath>/Users/karma/Web-Karma/lib/mysql-connector-java-5.1.32-bin.jar</systemPath>
</dependency>

Make sure that the filename mentioned in the systemPath element matches with your downloaded JAR file; it is likely that your installation folder is different from /Users/karma so make sure you use the correct one. The version will be the version of the JAR that you downloaded.

web-karma's People

Contributors

adamczerniejewski1 avatar akshayrd avatar alseambusher avatar areshand avatar aymandf avatar bidisha010496 avatar dependabot[bot] avatar dkapoor avatar frueyang avatar greatyyx avatar jasonslepicka avatar jimflip avatar jlbbj111 avatar kaushalv274 avatar namrata1012 avatar philpot avatar punith300i avatar rutujarane avatar saggu avatar sanmeetshikh avatar shrikanthn avatar shubhamg avatar siddharthaeron avatar szeke avatar taheriyan avatar tknandu avatar ukby1234 avatar vishalhemnani avatar yaoyichi avatar yingzhang avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

web-karma's Issues

Bug in JSON import

Does not show all the information present in the JSON (its a tweet JSON snippet).

JSON:

{
"id_str": "174866918208057344",
"retweeted": false,
"retweet_count": 2,
"created_at": "Wed Feb 29 14:41:36 +0000 2012",
"geo": {
"type": "Point",
"coordinates": [
38.56689474,
-121.47886353
]
},
"in_reply_to_status_id_str": null,
"in_reply_to_user_id": null,
"in_reply_to_user_id_str": null,
"user": {
"is_translator": false,
"id": 14411304,
"favourites_count": 1,
"profile_background_color": "024A82",
"lang": "en",
"geo_enabled": true,
"location": "Sacramento, CA",
"profile_background_image_url": "http://a3.twimg.com/profile_background_images/3214313/header-top-bg.gif",
"utc_offset": -28800,
"profile_link_color": "024A82",
"profile_background_image_url_https": "https://si0.twimg.com/profile_background_images/3214313/header-top-bg.gif",
"name": "The Sacramento Bee",
"profile_image_url": "http://a0.twimg.com/profile_images/62987579/scoopy_normal.png",
"time_zone": "Pacific Time (US & Canada)",
"followers_count": 19586,
"default_profile": false,
"protected": false,
"profile_use_background_image": true,
"following": true,
"id_str": "14411304",
"profile_text_color": "000",
"listed_count": 1258,
"description": "Official account of The Sacramento Bee in California. News updates for sports, entertainment, politics, and more. Managed by @journalistnate and @lgonzales.",
"contributors_enabled": false,
"profile_sidebar_border_color": "ccc",
"follow_request_sent": false,
"friends_count": 11053,
"url": "http://www.sacbee.com",
"statuses_count": 45495,
"screen_name": "sacbee_news",
"created_at": "Wed Apr 16 18:44:06 +0000 2008",
"profile_background_tile": false,
"show_all_inline_media": false,
"profile_image_url_https": "https://si0.twimg.com/profile_images/62987579/scoopy_normal.png",
"profile_sidebar_fill_color": "eee",
"notifications": false,
"default_profile_image": false,
"verified": false
},
"in_reply_to_status_id": null,
"favorited": false,
"in_reply_to_screen_name": null,
"truncated": false,
"possibly_sensitive": false,
"contributors": null,
"place": {
"name": "Sacramento",
"attributes": {},
"full_name": "Sacramento, CA",
"place_type": "city",
"country": "United States",
"url": "http://api.twitter.com/1/geo/id/b71fac2ee9792cbe.json",
"country_code": "US",
"bounding_box": {
"type": "Polygon",
"coordinates": [
[
[
-121.56012,
38.43782
],
[
-121.36274,
38.43782
],
[
-121.36274,
38.685506
],
[
-121.56012,
38.685506
]
]
]
},
"id": "b71fac2ee9792cbe"
},
"source": "<a href="http://dlvr.it\" rel="nofollow">dlvr.it",
"id": 174866918208057340,
"coordinates": {
"type": "Point",
"coordinates": [
-121.47886353,
38.56689474
]
},
"text": "Economy grew at a faster pace at end of 2011 http://t.co/fdK9WYaz"
}

Exception on set semantic type

I've seen the following exception a few times.

Pedro

ERROR qtp150326234-16 - Alignment returned null root!
ERROR qtp150326234-16 - Bad JSON received from server!
org.json.JSONException: JSONObject["FullType"] not found.
at org.json.JSONObject.get(JSONObject.java:498)
at org.json.JSONObject.getString(JSONObject.java:669)
at edu.isi.karma.controller.command.alignment.SetSemanticTypeCommandFactory.createCommand(SetSemanticTypeCommandFactory.java:64)
at edu.isi.karma.webserver.ExecutionController.getCommand(ExecutionController.java:168)
at edu.isi.karma.webserver.RequestController.doPost(RequestController.java:56)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)

Table name in published model

Hi, I'm currently creating an R2RML model from karma web UI, publishing it and using it to periodically generate RDF in batch mode. My problem is that in the R2RML contains some reference about the name of the MySQL table that I used to generate it, but I would like to use it with tables that have different names but still the same content.
For example, I published the model using a table named "sensors1", I would like to use the R2RML to publish RDF of a table named "sensors2" that has the same columns as "sensors1". Is this possible to to this? Maybe using some regular expressions in the R2RML? Thanks

Publish RDF for databases should prompt for what to do

RIght now it generates RDF for the data that's loaded in Karma. We should prompt to ask the user whether to generate RDF for the whole table or only the part that is loaded in Karma.

We will need this to generate RDF for the smithsonian tables.

UI suggestions

The Command history takes too much place;
it could be shrinkable like panels in eclipse, or in another tab.

The checkbox
" Include input attributes in the output worksheet"
in import from API
is not clear to me.
Maybe a tooltip can help.

Maven compilation error

I have just downloaded and tried to run the Karma
I get the following error :
( I didn't try to fix anything, except changing the ports for jetty)

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile) on project webkarma: Fatal error compiling: invalid target release: 1.7 -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile) on project webkarma: Fatal error compiling
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:216)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
at org.apache.maven.lifecycle.internal.MojoExecutor.executeForkedExecutions(MojoExecutor.java:364)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:198)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:84)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:59)
at org.apache.maven.lifecycle.internal.LifecycleStarter.singleThreadedBuild(LifecycleStarter.java:183)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:161)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:317)
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:152)
at org.apache.maven.cli.MavenCli.execute(MavenCli.java:555)
at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:214)
at org.apache.maven.cli.MavenCli.main(MavenCli.java:158)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
Caused by: org.apache.maven.plugin.MojoExecutionException: Fatal error compiling
at org.apache.maven.plugin.compiler.AbstractCompilerMojo.execute(AbstractCompilerMojo.java:796)
at org.apache.maven.plugin.compiler.CompilerMojo.execute(CompilerMojo.java:129)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:106)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208)
... 23 more
Caused by: org.codehaus.plexus.compiler.CompilerException: invalid target release: 1.7
at org.codehaus.plexus.compiler.javac.JavaxToolsCompiler.compileInProcess(JavaxToolsCompiler.java:191)
at org.codehaus.plexus.compiler.javac.JavacCompiler.performCompile(JavacCompiler.java:169)
at org.apache.maven.plugin.compiler.AbstractCompilerMojo.execute(AbstractCompilerMojo.java:785)
... 26 more
Caused by: java.lang.IllegalArgumentException: invalid target release: 1.7
at com.sun.tools.javac.main.RecognizedOptions$GrumpyHelper.error(RecognizedOptions.java:75)
at com.sun.tools.javac.main.RecognizedOptions$14.process(RecognizedOptions.java:380)
at com.sun.tools.javac.api.JavacTool.processOptions(JavacTool.java:247)
at com.sun.tools.javac.api.JavacTool.getTask(JavacTool.java:207)
at com.sun.tools.javac.api.JavacTool.getTask(JavacTool.java:53)
at org.codehaus.plexus.compiler.javac.JavaxToolsCompiler.compileInProcess(JavaxToolsCompiler.java:115)
... 28 more
[ERROR]

Literal types in RDF

Hi, I'm generating RDF triples using karma and I'm wondering if literal types should appear in them. I mean, what is the utility of specifyng the literal type for a certain field? Is it ok that in the RDF file that I publish there is no reference to the literal type that I set in the model?

JSON import not working

I'm using Karma for the first time,
and I'm assuming that it's OK to import a JSON with no model pre-existing.

I tried from the web UI (gihub clone) this JSON URL:
http://data.iledefrance.fr/api/datasets/1.0/search?rows=100
I raised the memory to 3Gb ; but it's real slow, and it takes 100% of both CPU's,
and it ends up in "OutOfMemoryError: GC overhead limit exceeded" anyway.

But the data, after download with wget, is not huge:
% wc data.iledefrance.fr_search_rows_100.json
0 19320 206512 data.iledefrance.fr_search_rows_100.json

Then I retarted, and tried this smaller JSON URL:
http://data.iledefrance.fr/api/datasets/1.0/search?rows=20

Now no CPU used, no special message on the console, and running forever in the browser.

Imports deactivated

The last commit, d60b905 ,
is unusable because items in Imports pulldown are deactivated except SQL .
Also, the pulldown menu is mostly behind the history panel.

I tried on fresh git clone, Chrome browser on Ubuntu 13.10 , Java 1.7 51 .

Build Failure

Here is the relevant output after attempting a build:

[INFO] Using the builder org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder with a thread count of 1
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building ISI Karma Maven Webapp 0.0.1-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ webkarma ---
[INFO]
[INFO] --- maven-install-plugin:2.4:install (default-install) @ webkarma ---
[INFO] Installing /usr/local/Web-Karma/pom.xml to /home/dhenry/.m2/repository/edu/isi/webkarma/0.0.1-SNAPSHOT/webkarma-0.0.1-SNAPSHOT.pom
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building karma-util 0.0.1-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ karma-util ---
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ karma-util ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /usr/local/Web-Karma/karma-util/src/main/config
[INFO] skip non existing resourceDirectory /usr/local/Web-Karma/karma-util/src/main/resources
[INFO]
[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ karma-util ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 40 source files to /usr/local/Web-Karma/karma-util/target/classes
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] ISI Karma Maven Webapp ............................ SUCCESS [ 0.387 s]
[INFO] karma-util ........................................ FAILURE [ 0.673 s]
[INFO] karma-crf ......................................... SKIPPED
[INFO] karma-cleaning .................................... SKIPPED
[INFO] karma-common ...................................... SKIPPED
[INFO] karma-commands .................................... SKIPPED
[INFO] commands-common ................................... SKIPPED
[INFO] karma-jdbc ........................................ SKIPPED
[INFO] karma-not-common .................................. SKIPPED
[INFO] commands-update ................................... SKIPPED
[INFO] commands-alignment ................................ SKIPPED
[INFO] commands-worksheet ................................ SKIPPED
[INFO] commands-python ................................... SKIPPED
[INFO] commands-alignment-openrdf ........................ SKIPPED
[INFO] commands-cleaning ................................. SKIPPED
[INFO] commands-publish .................................. SKIPPED
[INFO] commands-publish-jdbc ............................. SKIPPED
[INFO] commands-publish-alignment-openrdf ................ SKIPPED
[INFO] commands-update-jdbc .............................. SKIPPED
[INFO] commands-import ................................... SKIPPED
[INFO] import-common ..................................... SKIPPED
[INFO] import-csv ........................................ SKIPPED
[INFO] import-database ................................... SKIPPED
[INFO] import-excel ...................................... SKIPPED
[INFO] import-file ....................................... SKIPPED
[INFO] import-ontology ................................... SKIPPED
[INFO] import-json ....................................... SKIPPED
[INFO] import-service .................................... SKIPPED
[INFO] import-spatial .................................... SKIPPED
[INFO] import-xml ........................................ SKIPPED
[INFO] commands-include .................................. SKIPPED
[INFO] karma-web ......................................... SKIPPED
[INFO] karma-offline ..................................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1.520 s
[INFO] Finished at: 2014-05-07T15:12:58-06:00
[INFO] Final Memory: 9M/23M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile) on project karma-util: Fatal error compiling: directory not found: /usr/local/Web-Karma/karma-util/target/classes -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn -rf :karma-util

Ontology manager per workspace

The ontology manager should be defined per workspace and should not be global. We need this capability to enable different users to load different ontologies. Mohsen and Shubham are both involved.

Problem during installation with Java 1.7 and Maven 3.0

Hello,
I just downloaded Karma to test but I have a problem when I run mvn jetty: run in the htdocs directory karma placed on local server

The problem is as follows:

Failed to execute goal org.apache.maven.plugins: maven-compiler-plugin:
 3.1: compile (default-compile) 
on project webkarma: Fatal error compiling: 
invalid target release: 1.7 -> [Help 1] 

However, my version of java is correct

imac-de-antoine: ~ $ java-version antoinecourtin 
java version "1.7.0_51" 
Java (TM) SE Runtime Environment (build 1.7.0_51-b13) 
Java HotSpot (TM) 64-bit Server VM (build 24.51-b03, mixed mode) 
imac-de-antoine: ~ $ antoinecourtin 

Thank you in advance,
Ps: sorry for the question probably stupid, I'm not a developer, just curious;)

Error popup when showing model: For input string: "0,33"

For the same data in issue #70 :
http://data.iledefrance.fr/api/datasets/1.0/search?rows=100

when clicking on "show model" a red popup appers on top right:
Error: For input string: "0,33"

It looks like a number in french format, but I found no 0,33 in the data.

I hope that despite the message, the "show model" view is usable...

Talking about error message, trying to import an HTML URL gives no message at all in the browser, just in the console.

Master branch does not build: Test failure

The master branch does not pass two unit tests when build from a fresh clone. Best practice is to keep the master branch always stable and deployable.

Tests run: 2, Failures: 2, Errors: 0, Skipped: 0, Time elapsed: 0.431 sec <<< FAILURE!
testDoGetWithCookies(edu.isi.karma.webserver.KarmaServletTest) Time elapsed: 0.416 sec <<< FAILURE!
Argument(s) are different! Wanted:
printWriter.println(
"{
"workspaceId" : "WSP1" ,
"elements" : [
{
"updateType" : "WorksheetListUpdate" ,
"worksheets" : [
]
}
]
}
"
);
-> at edu.isi.karma.webserver.KarmaServletTest.testDoGetWithCookies(KarmaServletTest.java:109)
Actual invocation has different arguments:
printWriter.println(
"{
"workspaceId" : "WSP6" ,
"elements" : [
{
"updateType" : "WorksheetListUpdate" ,
"worksheets" : [
]
}
]
}
"
);
-> at edu.isi.karma.webserver.KarmaServlet.doGet(KarmaServlet.java:151)

at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
at edu.isi.karma.webserver.KarmaServletTest.testDoGetWithCookies(KarmaServletTest.java:109)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252)
at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141)
at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)

testDoGetWithoutCookies(edu.isi.karma.webserver.KarmaServletTest) Time elapsed: 0.014 sec <<< FAILURE!
Argument(s) are different! Wanted:
printWriter.println(
"{
"workspaceId" : "WSP2" ,
"elements" : [
{
"updateType" : "WorksheetListUpdate" ,
"worksheets" : [
]
}
]
}
"
);
-> at edu.isi.karma.webserver.KarmaServletTest.testDoGetWithoutCookies(KarmaServletTest.java:97)
Actual invocation has different arguments:
printWriter.println(
"{
"workspaceId" : "WSP7" ,
"elements" : [
{
"updateType" : "WorksheetListUpdate" ,
"worksheets" : [
]
}
]
}
"
);
-> at edu.isi.karma.webserver.KarmaServlet.doGet(KarmaServlet.java:151)

at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
at edu.isi.karma.webserver.KarmaServletTest.testDoGetWithoutCookies(KarmaServletTest.java:97)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252)
at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141)
at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)

Results :

Failed tests: testDoGetWithCookies(edu.isi.karma.webserver.KarmaServletTest): (..)
testDoGetWithoutCookies(edu.isi.karma.webserver.KarmaServletTest): (..)

Tests run: 7, Failures: 2, Errors: 0, Skipped: 0

Need to generate rdf files in batch mode

Hi ,

I am running cmd in batch mode. I dont see any error in log rather the output file also.
Please help.
"mvn exec:java -Dexec.mainClass="edu.isi.karma.rdf.OfflineRdfGenerator" -Dexec.args="
--sourcetype CSV --sourcename org --filepath "/Users/bigdata/Downloads/demo-files/org2.csv"
--modelfilepath "/Users/bigdata/d/Web-Karma-master/karma-web/src/main/webapp/publish/RDF/WSP1WS961.ttl"
--outputfile /Users/bigdata/Downloads/resultOrgrdf.ttl" -Dexec.classpathScope=compile"

Output:

INFO] Scanning for projects...
[INFO]
[INFO] Using the builder org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder with a thread count of 1
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building karma-offline 0.0.1-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[WARNING] The POM for de.micromata.jak:JavaAPIforKml:jar:2.2.0-SNAPSHOT is missing, no dependency information available
[INFO]
[INFO] --- exec-maven-plugin:1.3:java (default-cli) @ karma-offline ---
[WARNING] Warning: killAfter is now deprecated. Do you need it ? Please comment on MEXEC-6.
Unexpected
--sourcetype while processing options
Usage:
[--sourcetype --filepath --modelfilepath
--modelurl --sourcename --outputfile
--dbtype --hostname --username
--password --portnumber --dbname --tablename
--help]
options
--sourcetype sourcetype type of source. Valid values: DB, CSV, JSON,
XML
--filepath filepath location of the input file
--modelfilepath modelfilepath location of the model file
--modelurl modelurl location of the model
--sourcename sourcename name of the source in the model to use
--outputfile outputfile location of the output file
--dbtype dbtype database type. Valid values: Oracle, MySQL,
SQLServer, PostGIS
--hostname hostname hostname for database connection
--username username username for database connection
--password password password for database connection
--portnumber portnumber portnumber for database connection
--dbname dbname database or SID name for database connection
--tablename tablename hostname for database connection
--help print this message
Usage:
[--sourcetype --filepath --modelfilepath
--modelurl --sourcename --outputfile
--dbtype --hostname --username
--password --portnumber --dbname --tablename
--help]
options
--sourcetype sourcetype type of source. Valid values: DB, CSV, JSON,
XML
--filepath filepath location of the input file
--modelfilepath modelfilepath location of the model file
--modelurl modelurl location of the model
--sourcename sourcename name of the source in the model to use
--outputfile outputfile location of the output file
--dbtype dbtype database type. Valid values: Oracle, MySQL,
SQLServer, PostGIS
--hostname hostname hostname for database connection
--username username username for database connection
--password password password for database connection
--portnumber portnumber portnumber for database connection
--dbname dbname database or SID name for database connection
--tablename tablename hostname for database connection
--help print this message
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 3.146 s
[INFO] Finished at: 2014-05-01T15:20:32+05:30
[INFO] Final Memory: 18M/439M
[INFO] ------------------------------------------------------------------------

Pipe-Delimited CSV File Import (flat file)

Good afternoon, and thank you for creating Web-Karma!

I downloaded and installed the software today, but I did not see an option to import pipe-delimited files in the 'Import CSV' form.

I tried to figure out how to change the code so that I could do this, and I made the following additions to 2 of the source files:

// File - ..\Web-Karma-master\src\main\webapp\index.jsp
// line 450:

    <option>pipe</option>

// File - ..\Web-Karma-master\src\main\java\edu\isi\karma\controller\command\importdata\ImportCSVFileCommand.java
// line 148:

    } else if (request.getParameter("delimiter").equals("pipe")) {
        setDelimiter('|');

It seems to work after a jetty restart, and I was then able to import my pipe-delimited files.

Thank you again for your hard work!

Michael

Various issues

I'm taking this JSON tabular data:
http://data.iledefrance.fr/api/records/1.0/download?dataset=carte-des-pharmacies-dile-de-france&format=json
that has 4000 lines of an average of 7 fields per line.

I have these preloaded ontologies:

ls -l --dereference preloaded-ontologies
total 1244
-rw-rw-r-- 1 jmv jmv 1150363 oct. 14 21:16 dbpedia_3.9.owl
-rw-r--r-- 1 jmv jmv 17568 mars 17 13:17 dc.rdf
-rw-rw-r-- 1 jmv jmv 43191 nov. 9 2011 foaf.rdf
-rw-r--r-- 1 jmv jmv 48977 mars 17 13:19 sioc.rdf
-rw-r--r-- 1 jmv jmv 7866 mars 17 13:20 wgs84_pos.rdf

  • the memory used is quite large : 2.1Gb
  • Once I choose http://dbpedia.org/ontology/postalCode for the property URI, I expect the "Literal type" to be at once populated to xsd:int, because this is the range of dbo:postalCode.
  • choosing a property is slow ( several seconds )
  • when preloading foaf.rdf, the abbreviated URI's like foaf:name are not recognized
  • doc.: speaking of "semantic type" is misleading for users knowing basics of semantic web, and probably not helpful to others; I suggest "semantic property"
  • starting in another browser tab http://localhost:8080/ blocks the first one (is grey)
  • was obliged several times to do
    rm -r ~/.aduna
    because importing had become ineffective
  • it does not seem possible to save one's work on semantic mapping to continue it later
  • it not clear how smart is Karma in proposing properties: I've seen it propose long for "lng"

Compiling Issue Installing Karma

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile) on project webkarma: Fatal error compiling: invalid target release: 1.7 -> [Help 1]

Offline RDF generation

First of all I would like to report that in the README file you should fix the
"-Dexec.mainClass" parameter: it should be "edu.isi.karma.rdf.OfflineRdfGenerator" (like in the last example) and not "edu.isi.karma.rdf.OfflineDbRdfGenerator" (like in the first examples).

Then, as you told me, I saw you are fixing the "not able to create user preferences" issue ( https://webkarma.atlassian.net/browse/WK-2 ), but I was wondering if this error is affecting the produced RDF files or it is an unrelated issue. Are the produced RDF files the same as those that I get when I use Karma from the browser? Thanks.

Set up password protection for accessing Karma

I followed the following two points to set password but nothing is reflected.

  1. in /src/main/webapp/WEB-INF/web.xml uncomment security section at the end of the file
  2. in pom.xml uncomment security section (search for loginServices)
    How can I set the password protection for accessing Karma?

BUILD FAILURE

Hi
I am not able to start server, getting below error. Please suggest how to resolve it.

[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 28:01 min
[INFO] Finished at: 2014-04-19T18:34:07+05:00
[INFO] Final Memory: 8M/76M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project webkarma: Could not resolve dependenci
es for project edu.isi:webkarma:war:0.0.1-SNAPSHOT: Failed to collect dependenci
es at com.hp.hpl.jena:arq:jar:2.8.8 -> com.hp.hpl.jena:jena:jar:2.6.4 -> com.hp.
hpl.jena:iri:jar:0.8 -> com.ibm.icu:icu4j:jar:3.4.4: Failed to read artifact des
criptor for com.ibm.icu:icu4j:jar:3.4.4: Could not transfer artifact com.ibm.icu
:icu4j:pom:3.4.4 from/to maven2.maven.org (http://repo1.maven.org/maven2/): repo
1.maven.org: Unknown host repo1.maven.org -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e swit
ch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please rea
d the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyReso
lutionException

Many Thanks

Concurrent modification exception when clearing semantic types

We need to ensure that all our commands are serialized in the server.

INFO qtp859944758-22 - Source description written to file: ./publish/Source Description/4_PharmGKBPathways.csv.txt
INFO qtp859944758-22 - RDF written to file: ./src/main/webapp/RDF/VW1.n3
2012-03-09 19:26:43.503:WARN::/RequestController
java.util.ConcurrentModificationException
at java.util.HashMap$HashIterator.nextEntry(HashMap.java:793)
at java.util.HashMap$KeyIterator.next(HashMap.java:828)
at edu.isi.karma.modeling.alignment.AlignmentManager.removeWorkspaceAlignments(AlignmentManager.java:28)
at edu.isi.karma.controller.command.CloseWorkspaceCommand.doIt(CloseWorkspaceCommand.java:46)
at edu.isi.karma.controller.history.CommandHistory.doCommand(CommandHistory.java:114)
at edu.isi.karma.webserver.ExecutionController.invokeCommand(ExecutionController.java:184)
at edu.isi.karma.webserver.RequestController.doPost(RequestController.java:56)

Bug in displaying null values

2012-03-14 14:39:12.809:WARN::/RequestController
java.lang.NullPointerException
at edu.isi.karma.view.tabledata.VDTableCells.generateJsonContentCell(VDT
ableCells.java:914)
at edu.isi.karma.view.tabledata.VDTableCells.generateJsonContentRow(VDTa
bleCells.java:873)
at edu.isi.karma.view.tabledata.VDTableCells.generateJsonRows(VDTableCel
ls.java:437)
at edu.isi.karma.view.tabledata.VDTableCells.generateAllJsonRows(VDTable
Cells.java:415)
at edu.isi.karma.view.tabledata.VDTableCells.generateJson(VDTableCells.j
ava:394)
at edu.isi.karma.view.tabledata.VDTableData.generateJson(VDTableData.jav
a:116)
at edu.isi.karma.view.VWorksheet.generateWorksheetHierarchicalDataJson(V
Worksheet.java:311)

Show namespace prefixes

Show them in the main view on the worksheet, the semantic type dialogue box and the pencil dialogue box.

CSV files transformation in bach mode

Hi, i need to generate RDF triples from a CSV file. I have to do that in batch mode.
So i run the command:

"mvn exec:java -Dexec.mainClass="edu.isi.karma.rdf.OfflineRdfGenerator" -Dexec.args="--sourcetype CSV --filepath "/home/ubuntu/Dati-statici/Fermate_Toscana/Fermate_Pre_Triple/2_LOTTO_ARETINO.csv" --modelfilepath "/home/ubuntu/Dati-statici/Fermate_Toscana/Modello_Karma/BusStop_R2RML.ttl" --outputfile /home/ubuntu/Dati-statici/Fermate_Toscana/Triple/2_LOTTO_ARETINO.ttl" -Dexec.classpathScope=compile" in the shell.

The output is:

[INFO] Scanning for projects...
[WARNING]
[WARNING] Some problems were encountered while building the effective model for edu.isi:webkarma:war:0.0.1-SNAPSHOT
[WARNING] 'dependencies.dependency.systemPath' for net.sourceforge.jtds:jtds:jar should not point at files within the project directory, ${project.basedir}/lib/jtds-1.2.5.jar will be unresolvable by dependent projects @ line 110, column 16
[WARNING]
[WARNING] It is highly recommended to fix these problems because they threaten the stability of your build.
[WARNING]
[WARNING] For this reason, future Maven versions might no longer support building such malformed projects.
[WARNING]
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building ISI Karma Maven Webapp 0.0.1-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] >>> exec-maven-plugin:1.2.1:java (default-cli) @ webkarma >>>
[INFO]
[INFO] <<< exec-maven-plugin:1.2.1:java (default-cli) @ webkarma <<<
[INFO]
[INFO] --- exec-maven-plugin:1.2.1:java (default-cli) @ webkarma ---
INFO 2014-03-12 15:47:49,997: Detected encoding for file: 2_LOTTO_ARETINO.csv: null
ERROR 2014-03-12 15:47:50,000: You need to supply a value for '--sourcename'
INFO 2014-03-12 15:47:50,001: done
INFO 2014-03-12 15:47:50,001: RDF published at: /home/ubuntu/Dati-statici/Fermate_Toscana/Triple/2_LOTTO_ARETINO.ttl
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 6.298s
[INFO] Finished at: Wed Mar 12 15:47:50 CET 2014
[INFO] Final Memory: 12M/152M
[INFO] ------------------------------------------------------------------------

And the output file is actually empty.
Anybody know what's wrong ?

No plugin found for prefix 'jetty' in the current project

The compilation seems to have smoothly (I got BUILD SUCCESS), but when trying to start with mvn jetty:run I get many skipped items, BUILD FAILURE and the following error message:

[ERROR] No plugin found for prefix 'jetty' in the current project and in the plugin groups [org.apache.maven.plugins, org.codehaus.mojo] available from the repositories [local (/home/karma/.m2/repository), central (http://repo.maven.apache.org/maven2)] -> [Help 1]

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.