Problem parsing JSON in RTXQuery about rtx HOT 20 CLOSED

oops, I'm working on this in parallel here. I thought this was my task. I have it figured out here. so, maybe you can abandon your RTXQuery in favor of mine?
The issue is that your Q2Solution is not printing JSON, but rather printing Python serialization of an object, which is subtley different.
My example was using Python print object serialization, so that's fine. I was just going to continue with that. But I wonder if maybe it's better to have your code serialize in JSON? I don't know.

from rtx.

You can now try Q2 here:
http://rtx.ncats.io/devED/
and get a reasonable result.
It still needs some tidying around the edges.

from rtx.

@edeutsch Ah, that explains it! I will/have quite looking at RTXQuery and am now just focusing on getting Q1 to the standard format. I wasn't aware of that subtle different: should I be doing something else than having a print() command in main()?

from rtx.

One thing that I would like to ask you to fix is the Q2Solution when things go bad. Unless I'm misunderstanding, Q2Solution just dumps this to stdout when things aren't found:

Sorry, I could not find any paths connecting DOID:1686 to CHEMBL94 via protein, pathway, tissue, and phenotype. The drug and/or disease may not be one of the entities I know about, or they do not connect via a known pathway, tissue, and phenotype (understudied)

from rtx.

Instead, of dumping to just prose to stdout, that needs to be wrapped in a Response object. with a proper result_code and message. Q2Solution should never print anything but a Response object when invoked with -j

from rtx.

Also, the following error message must be wrapped in a Response object:
Sorry, the drug glaucoma is not yet in our knowledge graph.

from rtx.

@edeutsch Ok, I think I can do that. Is there a Response package somewhere that I should be using?

from rtx.

Regarding your question above, I suppose the cmd line option is -j for JSON, so maybe it is wise to switch from:
print(response)
to:
print(json.dumps(response,sort_keys=True,indent=2))

The difference is mainly just in single quote vs double quotes. But if you want to parse it with the json parser, it needs to be written in true JSON not Python object serialization.
I think I'm going to suggest that you switch to JSON. That will mean I need to change back over here on RTXQuery.py but that's probably all for the best.

from rtx.

yes, same Response class as you use for the full response, so:
from swagger_server.models.response import Response
response = Response()
response.result_code = "UnknownTerm"
response.message = "Sorry, I could not find any paths connecting DOID:1686 to CHEMBL94 via protein, pathway, tissue, and phenotype. The drug and/or disease may not be one of the entities I know about, or they do not connect via a known pathway, tissue, and phenotype (understudied)."

from rtx.

Got it!

from rtx.

Forgot to reference this in commit 1583836

from rtx.

@dkoslicki
We are almost there.
But the current Q1 Q2 Q3 solutions are not printing JSON, but rather Python object notation plus one line of "null".
Can we switch the output to full JSON notation as per thread above? Since the flag is -j and advertizes json?
Also, unlike the other two, Q2 doesn't have a -j flag. Can it? should it?

from rtx.

@edeutsch Hmmm... I thought I was using full JSON: the FormatOutput.FormatResponse class has a method to_print() that consists of print(json.dumps(print(self.response), sort_keys=True, indent=2)). Is this not sufficient to return full JSON? All the QXSolution.py's use this method to print the results.
Hopefully this can be accomplished withing the FormatOutput class since it's quite handy to not need to hand populate the response class each time...

from rtx.

@edeutsch Then again, I am doing the following to print(self.response):

def __str__(self):
	return repr(self.response)
def print(self):
	print(json.dumps(print(self.response), sort_keys=True, indent=2))

Maybe there's a built in __str__ that I should be using?

from rtx.

Well, I don't really know where the problem is. but the symptom is that it isn't printing JSON. In code that was fiddling with, it definitely worked. so I'm guessing that the code you think it being run may not be what's actually running?

The symptom is this:
$ python3 Q2Solution.py -r 'DOID:1686' -d 'physostigmine'
{'context': 'http://translator.ncats.io',
'datetime': '2018-03-29 02:36:53',
'id': 'http://rtx.ncats.io/api/v1/response/1234',
'message': 'Sorry, I could not find any paths connecting DOID:1686 to '
'CHEMBL94 via protein, pathway, tissue, and phenotype. The drug '
'and/or disease may not be one of the entities I know about, or '
'they do not connect via a known pathway, tissue, and phenotype '
'(understudied)',
'original_question_text': 'ERIC FILL THIS IN FROM QuestionTranslator.py',
'restated_question_text': 'ERIC FILL THIS IN FROM QuestionTranslator.py',
'result_code': 'NoPathsFound',
'result_list': None,
'schema_version': '0.5',
'tool_version': 'RTX 0.4',
'type': 'medical_translator_query_result'}
null

Notice that there are no double quotes, so this is Python object printing not json printing. Also notice that after the last close brace, there is a "null" (which I think is JSON vs. "None" which is Python). So maybe whatever is being executed is printing the object in python whereas the json.dumps is seeing a null object?

oh wait, i see. you need to replace:
print(json.dumps(print(self.response), sort_keys=True, indent=2))
with:
print(json.dumps(self.response, sort_keys=True, indent=2))

The inner print() prints out the object in Python notation to stdout and the print() function itself simply returns null. It has no return (like void print()) and that is what json.dumps is seeing.

from rtx.

oops, that isn't right either. working on it...

from rtx.

okay, I have pushed new code. please pull @dkoslicki
In the end, the somewhat icky workaround I had to do was this:
print(json.dumps(ast.literal_eval(repr(self.response)), sort_keys=True, indent=2))
json.dumps won't operate on an object, just a pure dict. So the repr() prints out the object as a dict. Then ast.literal_eval slurps that string into a real dict. And the dumps is happy. This was the best workaround I could find after considerable thrashing around.

So, the result is: At:
http://rtx.ncats.io/devED/
We are getting nice printed JSON for each Q0 Q1 Q2 Q3. yay!

Two quibbles for you:

1. I'm calling Q2Solution.py without -j because it has none. But I suggest for uniformity with the other QXSolutions, it should have the same -j option.

2. For the default Q3, you are returning 30 results. Wouldn't this make a lot more sense as one result with 31 nodes and 30 edges? After musing about this some more, this is a bit of a philosophical question again, either way is possibly correct, I suppose. I would argue this should be one result because the user asked for a list of proteins, but I can see it the other way, too. Another lively topic for Monday, perhaps?

from rtx.

@edeutsch I'll add a -j to Q2Solution. As for the second quibble, I don't really care if it's one graph with 31 nodes and 30 edges, or just 30 nodes. My initial reasoning is that the user wanted a list, so I returned a "list" (i.e. a bunch of different entries). I'm fine with returning a single graph if that's preferred. Let me know if I should do that.

Last question: for the UI, is @isbluis working to parse the JSON so it looks prettier than how it currently is at http://rtx.ncats.io/devED/? I assume after this is done, we can close this issue. Correct?

from rtx.

After musing about it some more, separate graphs may be better after all.

Yes, Luis is working on a pretty display based on the new JSON.

I suppose we can close this issue now, as that work isn't directly related to this issue, I think?

from rtx.

@edeutsch Sounds good, I'll close the issue. Should we merge NewStdAPI into master?

from rtx.