rdfjs / n3.js Goto Github PK

Hi @RubenVerborgh,
Is it possible for now to use prefix with a node-n3 store like this
store.find(null, 'dc11:date', null) ?

Also, will you continue to develop node-n3 ?

Many thanks,

Samuel

Hi Ruben.

I have has an issue with such a Turtle:
http://data.bnf.fr/13918021#frbr:Work http://www.w3.org/2000/01/rdf-schema#label "Elisabetta, regina d'Inghilterra" .
http://data.bnf.fr/10453033#foaf:Person http://purl.org/dc/terms/contributor http://data.bnf.fr/13918021#frbr:Work .
http://data.bnf.fr/10453033#foaf:Person http://purl.org/dc/terms/title "Giovanni Schmidt"@FR .
http://data.bnf.fr/13918021#frbr:Work http://www.w3.org/1999/02/22-rdf-syntax-ns#type http://rdvocab.info/uri/schema/FRBRentitiesRDA/Work .

Parsing it fails at line 3 because of a syntax error (the "@FR", it seems).

Is it something that you do reproduce?

The regular expressions in lib/n3lexer.js contain a number of U+FFFD REPLACEMENT CHARACTER symbols that aren’t supposed to be there (at least not as far as I can tell).

You can easily confirm the issue using a hex viewer or even on the GitHub website itself: Look for the “�” marks.

I’m guessing something went wrong when copying/pasting the regular expressions from inimino’s tool, or perhaps the file was at some point saved using a non-UTF-8 encoding, or perhaps Git/GitHub messed it up at some point…

Anyway, a solution could be to use escape sequences for those characters so that they don’t get replaced by U+FFFD, so that the original symbol isn’t lost.

N3.Util already provides utility functions to unfold the internal representation for RDF terms.

It would be very useful to have their counterparts, so that we could construct such elements e.g. makeLiteral("3", "http://www.w3.org/2001/XMLSchema#integer"), makeLiteral(3), makeUri("http://example.org/cartoons#Mickey"), etc.

I'm getting an error using @base Turtle syntax and other things. For example it seems like there needs a space after @prefix bibo:. Here's the errors I'm getting: http://jsfiddle.net/lmatteis/dm8G2/

A literal with datatype definition and no space between a closing bracket causes a "Syntax error: unexpected "^^xsd:time]" on line 6." error. This file [1] was used for testing.

[1] https://github.com/w3c/microdata-rdf/blob/gh-pages/tests/0047.ttl

The current writer can only write named blank nodes. Sometimes, the anonymous […] syntax is preferred.

Raised by @iherman.

Please add the version number inside the library files. Can be very help full for debugging purposes.
Either as simple comment on top of the file. Or also as an property which can be accessed inside the JavaScript context.

URLs with parent paths are not correctly resolved:

var parser = require('n3').Parser();
parser.parse('@base <http://example.org/foo/bar/../foo/bar/>.\n<#a> <b> <../c/d/e>.',
             function (error, triple) { triple && console.log(triple); });

Hi RubenVerborgh,

I found another problem maybe.
When I parse the following turtle

<http://xxx.com/res/font/weblysleekui/WeblySleek UI Normal.ttf> <http://xxx.com/Predicate#cacheControl> "private, max-age=604800" .

I get this error

[Error: Syntax error: unexpected "<http://collectit.invend.eu/res/font/weblysleekui/WeblySleek" on line 1.]

I think it's the reason that IRI of subject has a whitespace. However, I'm sure the whitespace is allowed in json-ld

Could you also test with that case?

Bg,
Kevin

To do some tests I converted Tim Berners-Lees profile to turtle using rapper:

rapper -i guess -o turtle HTTP://www.w3.org/People/Berners-Lee/card.rdf

The result can be found here: http://ktk.netlabs.org/misc/tbl.ttl

The current version of the parser implemented in rdfstore does not seem to be able to parse that though.

Hello Ruben,

I ran into an issue today where the browserified version of N3.js fails to parse a file, but the "normal" N3.js has no trouble with that file.

The browserified version of N3 fails to parse this in Firefox, Chrome and under node.js, so it doesn't seem like a browser specific issue.

I'm testing with a reasonably large file (64577 triples). Could this be a bug or am I running into a limitation of the browserify environment?

The stack trace I get is:

Syntax error: unexpected ""To" on line 9873.
    at Object.N3Lexer._syntaxError (/home/warp/www/n3/node_modules/n3/browser/n3-browser.js:415:12)
    at reportSyntaxError (/home/warp/www/n3/node_modules/n3/browser/n3-browser.js:382:54)
    at Object.N3Lexer._tokenizeToEnd (/home/warp/www/n3/node_modules/n3/browser/n3-browser.js:230:20)
    at Object._onImmediate (/home/warp/www/n3/node_modules/n3/browser/n3-browser.js:430:38)
    at processImmediate [as _immediateCallback] (timers.js:336:15)

I have a test case at https://gist.github.com/warpr/c4da9a48b9cb8f11a550 , "test.html" shows the bug in a browser, "test.js" shows it under node.js.

If I truncate the licensedb.2014-01-19.ttl file slightly before the syntax error there is no trouble and I can query the N3 Store as I'd expect.

Hi Ruben,
This is a follow up on the report started in #41. I've made a sample code detailing the issue in https://github.com/tibotiber/blank. Hope this helps you. I'm available for any help if needed.
Cheers and thanks.

Parsing example 10 from http://www.w3.org/2007/02/turtle/primer/ results in the error: Error: Undefined prefix ":"

rdf-ext based jsfiddle: https://jsfiddle.net/9wgLptcs/

908   _readPunctuation: function (token) {
909     var next, subject = this._subject, graph = this._graph;
910     switch (token.type) {
911     // A closing brace ends a graph
912     case '}':
913       if (this._graph === null)
914         return this._error('Unexpected graph closing', token);
915       this._graph = null;
916     // A dot just ends the statement, without sharing anything with the next.
917     case '.':

I'm still investigating, but I think there should be a 'break' after 915, and that the effect of falling through may be a null triple getting sent to parse()'s callback.

There are some other cases of unexpected fallthroughs too, like this:

    930     case 'IRI':
    931       if (this._supportsQuads && this._graph === null) {
...
    938         break;
    939       }
    941     case 'prefixed':

If 938/939 are really in the right order, I recommend a comment noting that you meant that logic.

Hello,

I recently found you made an update for removing angular brackets from types in this commit
70fff5d

I do not understand why it must be done like this. It's the new literal syntax from turtle or n3? I think this modification will be incompatible with lots of other modules.

Bg,
Kevin

Hi Ruben,

This is very much secondary but while putting my nose into the lib's code for the removal of triples, I was looking at the index mapping entities to numbers and I thought it could be nice to have another kind of size public method to get the number of individuals in the store. It's an info I often need. So far, I am using the following code to get it:

var uu = require('underscore');
countTypedIndividuals : function() {
    /* return the number of unique typed (with rdf:type) individuals in the store */
    return uu.uniq(store.find(null, 'rdf:type', null)).length;
}

But I feel this could be done much faster internally to N3.js by returning the length of the entities index. I am not sure about this though. What do you think?

Hi Ruben,

I'm finally working on integrating N3.js to our framework. Just wondering if there are ways to update and/or remove triples from the store? Can't seem to find that...

Thanks,
Thibaut.

I am trying to "wire" this library into LevelGraph, but I am facing a big issue regarding the 'streaming' interface.
In fact, most of LeveGraph is built on top node's streams and I finding extremely difficult to pipe the parse method into another stream. I am using: https://github.com/mcollina/node-levelgraph#putting-and-deleting-through-streams

In order to be compliant to node's stream, the parser should be exposed with Transform or with Through. The first requires the https://github.com/isaacs/readable-stream package to be run on node v0.8.x.

I am also linking the Stream Handbook for reference.

This is the behaviour we might want to achieve:

fs.createReadStream("a file").pipe(new n3.ParserStream()).pipe(someOtherStream)

At the N3Util section (https://github.com/RubenVerborgh/N3.js#utility) only the URIs' and literals' helpers are documented but not the blank nodes' helpers ( N3Util.isBlank("_:node") ) .

Could you provide some more insights about the blank nodes, please?
Thanks!

The writer supports prefixes in its output, but the addTriple method expects full IRIs. It would be nice if the writer also accepted prefixed names (like the store currently does).

Raised by @iherman.

Hi,

It is not really an issue, but I didn't find better place to ask. I'm going to ask very stupid question, but maybe you have any suggestions :)

Is it possible to parse turtle with N3.js synchronously? I'm trying to use ES6 taged template strings for turtle parsing. E.g something like this:

const res = turtle`<http://example.com/A> a <http://example.com/B> .`

so turtle function needs to parse content in synchronous way.

I have recently released LevelGraph, which is a graph database built on Hexastore.

The interesting thing about it is that in can be persisted on disk using LevelDB, and in a near future on top of IndexedDB.

Do you think it might be hard to integrate it with node-n3?

Instead of

var hydra = 'http://www.w3.org/ns/hydra/core#';

var triple = { subject: 'x', object: hydra + 'property', predicate: hydra + 'freetextQuery' }

we could have

var hydra = N3Util.prefix('http://www.w3.org/ns/hydra/core#');

var triple = { subject: 'x', object: hydra('property'), predicate: hydra('freetextQuery') };

in which each of the generated URLs would be cached.

Furthermore, we could have

var prefixes = N3Util.prefixes();
prefixes('hydra', 'http://www.w3.org/ns/hydra/core#');

var hydra = prefixes('hydra');
var property1 = hydra('property');
var property2 = prefixes('hydra')('property');

where

N3Util.prefix = function (uri) { return N3Util.prefixes({ '': uri })(''); }

From @sandhawke:

When I browserify my code, which require's './n3', the result doesn't include N3Parser because of your 'lazy' inclusion technique in N3.js. As a workaround for now I changed that file to...

module.exports.Parser = require('./lib/N3Parser');
module.exports.Lexer = require('./lib/N3Lexer');
module.exports.Util = require('./lib/N3Util');

...and now it works for me.

This is actually a bug of the C implementation, rather than the N3.Writer; but maybe we can make a small change in the N3.Writer and get rid of the problem.
Basically, rdf2hdt fails whenever the final dot in each triple in N-triples format does not have a space before it.

e.g.: The following line errors out:

<http://www.w3.org/2001/sw/RDFCore/ntriples/> <http://purl.org/dc/terms/title> "N-Triples"@en-us.

While the following one doesn't:

<http://www.w3.org/2001/sw/RDFCore/ntriples/> <http://purl.org/dc/terms/title> "N-Triples"@en-us .

Note the whitespace between the object and the finishing ..
Would you want to add it? I can make a pull request with it, if you think it's fine.
Best

P.

I have an application where a great many ttl fragments are read from file and parsed one after the other. Each parse call runs through the setImmediate shim in the browser version:
var immediately = typeof setImmediate === 'function' ? setImmediate :
function setImmediate(func) { setTimeout(func, 0);};
However, even though the timeout is set to 0 this does not mean that func is evaluated immediately. In my case, with thousands of fragments thus handled, this adds up to many seconds of idleness. There are faster ways of doing this, see e.g. http://ajaxian.com/archives/settimeout-delay, which depends on window.postmessage. Using this reduced the load time in my application from about 25 to 5 seconds (load time is not just N3 parsing).
And to satisfy my curiosity: what is the reason for including this source of asynchronicity?
Thanks,
Joop

Hi Ruben and thanks for this work.

Until now i was able to use node-n3 directly in the browser with just a few modifications.
But as now you use node.js stream, ...
I think it is not possible anymore.

What do you think about using node-n3 in the browser ?

Many thanks,

Samuel

Via @sandhawke:

Parsing http://www.w3.org/ns/dcat.ttl (380 triples) using node-N3 is taking about 30ms in chromium and about 1000ms in firefox. (ubuntu 12.04, ff 25.0.1, chr 30.0.1599.114) It doesn't seem slow doing normal webapp stuff.

In the first release of LevelGraph-N3 I have written a Stream for generating N3. It is far from being perfect, but if you need it we can move it in this package.

When running "npm test" the vows test is failing.

Running vows with verbose shows:

vows bin argv []
vows bin options {
    reporter: 'dot-matrix',
    matcher: /.*/
}
vows bin discovering folder structure
vows bin discovered ./test
vows runner loading [ '/home/warp/code/node-n3/test/n3lexer-test', '/home/warp/code/node-n3/test/n3parser-test', '/home/warp/code/node-n3/test/n3store-test' ]
vows runner running N3Lexer ✗ Errored » 1 errored

I am parsing large N3 files in parallel. I've noticed that sometimes when two files are being parsed in parallel or close to at the same time, I will get the following unhandled error:

          firstChar = input[0], match = null, matchLength = 0, unescaped, inco
                           ^
TypeError: Cannot read property '0' of null
    at Object.N3Lexer._tokenizeToEnd (/devhomes/wboyd/atk-proxy/node_modules/n3/lib/N3Lexer.js:98:28)
    at Immediate._onImmediate (/devhomes/wboyd/atk-proxy/node_modules/n3/lib/N3Lexer.js:331:38)
    at processImmediate [as _immediateCallback] (timers.js:358:17)

I believe this happens because this._input inside the N3Lexer can change to null after parsing. I think this is only happening because I am using the N3Parser as a singleton instead of creating an N3Parser for each parsing job. I am not sure if this is a bug, since the workaround is just to create an instance of the N3Parser for each parsing job, which is fine for my case, but I figured I would report it.

Please add a license file and/or licensing information to readme. Thanks!

Apparently, N3 does not support the complete set of allowed characters in prefix handling.
For example if we have the URL http://example.org/graphs#_0, we expect it to shorten it to g:_0 if we have the prefix g: <http://example.org/graphs#>.
Unfortunately, it does not do this, while the Turtle spec says that it should, including support for a bunch of other special characters.

In my fork, I added full support for this with the purpose of sending a PR, but while executing the tests I ran into an interesting issue.

The Turtle spec shows that prefixed entities like g:123 are not allowed, they should start with a character or something similar, and there are unit tests in this repo to confirm this behaviour.
Then I discovered that the nameStartChar also allows a character from the range [#x10000-#xEFFFF], and apparently this range includes a lot of characters.

If you would open up a node console and execute

/[\u10000-\uEFFFF]/.test("1")

You would see that it accepts all numbers (including a lot of other special characters as well).
So I am not sure if this specific range is intended to include numbers, if so, this would require some changes in the unit tests.
I assume this is a mistake in the spec, because for example the term nameChar explicitly accepts the range [0-9], even though it was already accepted by the range [#x10000-#xEFFFF].

I tried updating levelgraph-n3 to latest version of n3.js, when running this test
https://github.com/mcollina/levelgraph-n3/blob/3d3b6d64ea0b028dc9195633d6553e2920252237/test/putStream_spec.js

I started getting

 1) n3.putStream should store some triples:
     Error: Expected punctuation to follow "http://example.org/cartoons#Cat.c:Jerry" at line 1.
      at Object.N3Parser._error (/cache/code/maintain/levelgraph-n3/node_modules/n3/lib/N3Parser.js:457:20)
      at Object.N3Parser._readPunctuation (/cache/code/maintain/levelgraph-n3/node_modules/n3/lib/N3Parser.js:382:19)
...

Adding in test fixture a white space before . (after c:Cat) fixed those symptoms but I don't think (couldn't find in spec) that Turtle requires this white space before .

Hi Ruben,

It seems that the store accepts literals only for objects, it will refuse them for subjects right? This made my application crash when loading the store with triples parsed from EYE inference results using the following rule.

{?P rdfs:range ?C. ?X ?P ?Y} => {?Y a ?C}.

I have solved it by modifying the rule to

{?P rdfs:range ?C. ?X ?P ?Y. ?Y log:rawType ?T. ?T owl:differentFrom rdfs:Literal} => {?Y a ?C}.

I was wondering whether it would be better to make the store more permissive on this aspect. What do you think?

Thibaut.

Could you please provide some example code. I often find it more easy to grasp code then written documentation.

Hi, all~

When doing writer.addTriple, it seems that only one tripple a time. And that would be deadly slow (say 300000 tripples would cost about 10 min) when dealing with large numbers of tripples.

As what i asked, i'm trying to do it like this:

writer.addTriple({
subject: fullNameUrl,
predicate: 'http://example.org/property/contact#FN',
object: '"' + 'FN' + i + '"'
}, {
subject: fullNameUrl,
predicate: 'http://example.org/property/contact#name',
object: '"' + 'Name' + i + '"'
}, {
subject: fullNameUrl,
predicate: 'http://example.org/property/contact#sex',
object: '"' + 'sex' + '"'
}, {
subject: fullNameUrl,
predicate: 'http://example.org/property/contact#Email',
object: '"' + 'EmailAddr' + '"'
}
)

But somehow, it only works with the first tripple there.
If anyone could give me some advice or clue about it and that would be realy helpful.

Thanks, all~

I get this error when parsing an N3:
"Syntax error: unexpected "ns3:Juliane_Köhler" on line 13."

The original data definitely include funny characters.

So who to blame?
The DB for returning badly formatted data?
Or N3.js for crashing on a (perfectly valid?) UTF-something character?

Hi Ruben

Problem

I'm working on a DCAT validator for oSoc15 and I seem to have a problem with the language tag after finding a triple his object in the store.

The problem is that if i try to find a certain triple with the find function and I console.log the triple his object i get:
De DataTank Demo"^^http://www.w3.org/1999/02/22-rdf-syntax-ns#langString

While the triple i inserted was:
http://demo.thedatatank.com/api/dcat dc:title "De DataTank Demo"@nl ;

There is an "nl" tag behind the object of the triple when i insert it in the store but I can't get it when I use the find function.

How I've tried to solve the problem

I've looked through your documentation and saw that there is a function:
N3Util.getLiteralLanguage('"Mickey Mouse"@en');

But the triple object I get back from the find function doesn't return the literal with the language tag so concrete the find function would return: "Mickey Mouse" without @en.

Is there a way to solve this issue, if there is i'll be happy to hear it. Thanks in advance!!

Btw: you can find my DCAT validator project here: https://github.com/oSoc15/dcat-validator.js

Problem

Currently, if you want to use N3 in the browser, you need to clone this repository, build a js file use browserify and use that in your project.

Solution

Let's make a bower package for N3: http://bower.io/ (a package manager for browser applications).

The documentation at https://github.com/RubenVerborgh/N3.js#from-rdf-chunks-to-triples is misleading: the interleaved console.log do not necessarily see any triple as everything is asynchronous. Also, the call to parser.end() is not synchronous either.

N3Util.getLiteralType() for all untyped strings returns http://www.w3.org/1999/02/22-rdf-syntax-ns#PlainLiteral

while i read in: http://www.w3.org/TR/2013/CR-turtle-20130219/#literals

[...] If there is no datatype IRI and no language tag, the datatype is xsd:string

also the way i understand: http://www.w3.org/TR/rdf11-concepts/#dfn-literal

A literal is a language-tagged string if and only if its datatype IRI is http://www.w3.org/1999/02/22-rdf-syntax-ns#langString [...]

N3Util.getLiteralType('"That Seventies Show"@en')

should return http://www.w3.org/1999/02/22-rdf-syntax-ns#langString

Hi Ruben,

Seems that N3Store.find() does not expand the type of literals before searching for a match. E.g. executing

store.find(null, null, '"true"^^xsd:boolean')

won't find anything whereas

store.find(null, null, '"true"^^<http://www.w3.org/2001/XMLSchema#boolean>')

will. Any idea? Am I doing anything wrong or is that a bug?

Thanks,
Thibaut.

Hi Ruben,

thank you for your great Parser! I'm parsing a big file and writing the output to a file. It is really fast and I would like to parse even faster by executing it in parallel. My idea was the following:

1. assuming the machine has N-cores, split the file to parse into n parts, say file1, file2, file3, ..., fileN ;

2. add to file2 all lines of file1 beginning with @ ,
   add to file3 all lines of file1 and file1 beginning with @,
   and so on .... (this way prefixes will not be lost)

3. launch N-times the parser using N child processing on files1, ..., fileN respectively.

Then I would be able to parse N-time faster. Now my problem. How can I split the files instep 1) such that data that belongs to one turtle entry is not split up. Do you have an idea? Do you see some other problems using this strategy?

Thank you
Dennis

Hi, I am using levelgraph-n3, which has this module as a dependency, and the fancy submodule loading in N3.js is causing problems with webpack (module bundler ala browserify).

Here is a failing test case:

https://gist.github.com/edrex/9303090

I'm not sure yet what is causing the trouble.

Lists are common structures; in RDF, their represented by rdf:first, rdf:rest, and rdf:nil. It would be handy if there as a helper method to create the relevant triples from a JavaScript array.
This could be implemented in N3.Util, but also in N3.Writer.

Raised by @iherman.

Hello,
Is it possible to store triples as they are being parsed, regardless of whether they originate from a document or a stream?
Thanks

When I try to parse the contents of this URI http://dbpedia.org/resource/Henry's_Crime I only get owl:sameAs triples as an output of the parser. Not the others.

Here's the code I'm using (I have CORS enabled by default):

$.ajax({
    url: "http://dbpedia.org/resource/Henry's_Crime",
    headers: { 'Accept': 'text/turtle' }
}).done(function(data) {
    console.log(data) // outputs the actual Turtle data
    var parser = N3.Parser();
    parser.parse(data, function(err, triple, prefixes) {
        console.log(triple) // many triples are not logged
    })
})

Hi,

Today we worked with Antonio Garrote author of RDFStore which uses under the hood your n3 parser.

It works fine but for the following foaf profile (which seems to be valid turtle according to an online validator) we got an error.
The foaf profile is here: http://bblfish.net/people/henry/card#me

The error was:
"Expected punctuation to follow "[object Object]" at line 44."

Do you have any idea? thanks?