Error on getNodes() about rneo4j HOT 13 CLOSED

I'll have to investigate the fromJSON() options. This looks related, so I'll wait for someone to answer: http://r.789695.n4.nabble.com/prblem-when-using-fromJSON-td4669306.html

In the meantime, you might have better results using the following if you're only interested in the tweet text:

tweets = cypher(graph, "MATCH (t:Tweet) WHERE HAS(t.text) RETURN t.text")

This will return a data.frame tweets where the text is in the column t.text.

from rneo4j.

That returns the same error - so I assume some nasty characters in the text in one of the tweets.

Would be good to have a fix, but not desperately urgent.

I will try to narrow this down to the offending tweet.

from rneo4j.

Unicode nastiness is also a possibility.

http://stackoverflow.com/questions/15768116/how-to-replace-unexpected-escaped-character-in-r

from rneo4j.

backslash! I removed '' from 51 tweets and no problem with the remaining 25k.

from rneo4j.

Glad you figured it out.

from rneo4j.

Nicole,

I still think there is a bug that needs fixing. For the time being I can remove backslashes from tweets, but it would be useful to fix the underlying problem with the json corruption.

Let me know if you can fix this.

thanks,
js

from rneo4j.

here is a bit more information. It was getting late last night so sorry if I was a bit misleading. The fix I have got is a workaround to avoid this bug. I think that the text in the nodes I am retrieving from Neo4j are perfectly valid and should be returned without the json error.

The code I am using is adapted from your tutorial code https://github.com/nicolewhite/graphs_r_cool

Here is the line:

## Word cloud.
tweets = getNodes(Graph, "MATCH (t:Tweet) WHERE HAS(t.text) RETURN t ")

here is an example of the text in the node that causes the problems:

"@xtabel_25: Nigerian Union of Teachers said teachers should not resume till ebola is taking care of. :-\"forget that, saturday is still on

here is what it looks like in Neo4j browser -

please ping me if you need any further information.

from rneo4j.

According to your code block, getNodes() works fine but the problems are in subsequent functions for cleaning up the text for the word cloud? Let me know if I am understanding this correctly.

from rneo4j.

No, the line:
tweets = getNodes(Graph, "MATCH (t:Tweet) WHERE HAS(t.text) RETURN t")
gives error
Error in fromJSON(content, handler, default.size, depth, allowComments, :
invalid JSON input

I have updated the comment above to remove the comment about ASCII conversion - that is a separate (but maybe related) issue. Sorry for the confusion.

from rneo4j.

I'm going to look into switching to rjson from RJSONIO. rjson's fromJSON() has an option for handling unexpected escape characters and also doesn't round numbers unexpectedly like RJSONIO does. I'll let you know when I've tested this with the tweet text you supplied. Thanks for letting me know.

from rneo4j.

Nicole, any update on this issue? I would like to start doing some text analysis on tweets. It would be useful to know if I need to start work on a workaround.

thanks,
js

from rneo4j.

I haven't been able to reproduce your problem. When I use RJSONIO's fromJSON() on the string you provided, I get no errors. This is sitting on my back burner right now, as I still want to switch from RJSONIO to rjson. I think you'll have to continue cleaning up unwanted escape characters from your strings. I will comment here again when I get around to switching everything over to rjson.

from rneo4j.

I have a workaround - I just create a clean property on the node with all non alphanumerics stripped out. I can access the nodes ok from Python which I use for running processes, so no worries for time being.

from rneo4j.