JSONDecodeError about pycap HOT 15 OPEN

The administrators agreed to upgrade to the last version of REDCap to v14.3.14 so I will test soon against and update status

from pycap.

Great to hear that you and your team have enjoyed using the package so far.

Which version of PyCap are you using? If it's not latest version (2.4.0), I suggest trying the latest version.

If it is the latest version, I suggest using a previous version and seeing if the error goes away

However -- you said you just started having issues. I haven't released to PyPI since March 28th 2023. So I suspect that this has to do with your site's REDCap version -- perhaps the newest/your version of REDCap breaks the assumptions PyCap makes about it's API.

What version of REDCap is your site using? And did they recently change it?

from pycap.

Hi @pwildenhain, I started to use PyCap to export records from our REDCap instance. I was running data = project.export_records() and it throw me the same JSONDecodeError described by @EdgarHSJD . My REDCap version is 13.5.1 and it's recently updated. Any suggestion is recommended.

raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

from pycap.

Thanks for reporting -- I'll look into this.

In the meantime you can use the format_type parameter in export_records to export to a different format to avoid the JSONDecodeError. Available options include 'df' (a pandas DataFrame), 'csv', or 'xml'.

from pycap.

Great to hear that you and your team have enjoyed using the package so far.

Which version of PyCap are you using? If it's not latest version (2.4.0), I suggest trying the latest version.

If it is the latest version, I suggest using a previous version and seeing if the error goes away

However -- you said you just started having issues. I haven't released to PyPI since March 28th 2023. So I suspect that this has to do with your site's REDCap version -- perhaps the newest/your version of REDCap breaks the assumptions PyCap makes about it's API.

What version of REDCap is your site using? And did they recently change it?

Hello @pwildenhain, I am using the last version of the API.
Our RedCap Manager has confirmed that it was recently updated, which also caused the native API to stop working properly.
The current RedCap version we are using is 13.4.11.

I haven't got any other update. I will follow your recommendation for exporting records, and will get back as soon as I get any update.

Thank you!!

from pycap.

@pwildenhain thanks for your instruction. I tried adding format_type parameter but it returns the same error. Here are the codes I ran:

import redcap
project = redcap.Project('url', 'token')
data = project.export_records(format_type='df')

The error message I got is

Traceback (most recent call last):
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/code.py", line 90, in runcode
    exec(code, self.locals)
  File "<input>", line 1, in <module>
  File "/Users/PycharmProjects/REDCapAPI/venv/lib/python3.8/site-packages/redcap/methods/records.py", line 181, in export_records
    fields = self._backfill_fields(fields, forms)
  File "/Users/PycharmProjects/REDCapAPI/venv/lib/python3.8/site-packages/redcap/methods/records.py", line 47, in _backfill_fields
    return self.field_names + [f"{ form }_complete" for form in self.forms]
  File "/Users/PycharmProjects/REDCapAPI/venv/lib/python3.8/site-packages/redcap/methods/base.py", line 100, in field_names
    self._field_names = self._filter_metadata(key="field_name")
  File "/Users/PycharmProjects/REDCapAPI/venv/lib/python3.8/site-packages/redcap/methods/base.py", line 254, in _filter_metadata
    res = [row[key] for row in self.metadata]
  File "/Users/PycharmProjects/REDCapAPI/venv/lib/python3.8/site-packages/redcap/methods/base.py", line 80, in metadata
    self._metadata = cast(Json, self._call_api(payload, return_type="json"))
  File "/Users/PycharmProjects/REDCapAPI/venv/lib/python3.8/site-packages/redcap/methods/base.py", line 436, in _call_api
    return rcr.execute(
  File "/Users/PycharmProjects/REDCapAPI/venv/lib/python3.8/site-packages/redcap/request.py", line 189, in execute
    content = self.get_content(
  File "/Users/PycharmProjects/REDCapAPI/venv/lib/python3.8/site-packages/redcap/request.py", line 153, in get_content
    return response.json()
  File "/Users/PycharmProjects/REDCapAPI/venv/lib/python3.8/site-packages/requests/models.py", line 975, in json
    raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

from pycap.

Ah right, that's because I hit the API to get the metadata in JSON format, so you'll still get this error further down the stack trace.

Unfortunately since this is a REDCap bug there's not much I can do about it. If the errors persist I recommend temporarily switch bing over to using the requests package to get data (you can find code examples in the API playground of your project)

from pycap.

Ah right, that's because I hit the API to get the metadata in JSON format, so you'll still get this error further down the stack trace.

Unfortunately since this is a REDCap bug there's not much I can do about it. If the errors persist I recommend temporarily switch bing over to using the requests package to get data (you can find code examples in the API playground of your project)

Dear pwildenhain,

I have the same issue reported by @EdgarHSJD and @xj2193 and I tried also what you said in your last comment (I executed one of the API examples from RedCap) and still does not work.

from pycap.

Yea if the examples in the API playground aren't working, that means something is fundamentally broken with the API in the version of REDCap that you're using. I suggest asking your REDCap admins if they can roll back the version or install a later version if there is a bug fix release out already

from pycap.

Hello all,

Our RedCap admin recently updated it to version 13.6.0, and we still have the same problem.
We cannot roll back to a previous version for security reasons.

Do you have any other suggestion on how to address this issue?

Thank you.

Best

from pycap.

Hello all,

Our RedCap admin recently updated it to version 13.6.0, and we still have the same problem. We cannot roll back to a previous version for security reasons.

Do you have any other suggestion on how to address this issue?

Thank you.

Best

I'm not sure what the issue could be. Right now I can test on 13.5.0 and on 13.5.4 without any issue.

@EdgarHSJD does the issue still persist if you try the python code example from the API Playground? If there is something that PyCap can do to help workaround this, then I'm happy to assist with that.

from pycap.

How is the status of this issue? I am getting the same error...
raceback (most recent call last):
File "/data/xnat-phrt/venv-xnat/lib64/python3.9/site-packages/requests/models.py", line 974, in json
return complexjson.loads(self.text, **kwargs)
File "/usr/lib64/python3.9/json/init.py", line 346, in loads
return _default_decoder.decode(s)
File "/usr/lib64/python3.9/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib64/python3.9/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/data/phrt/src/REDCap.py", line 17, in
records = project.export_records(format_type='df')
File "/venv/lib64/python3.9/site-packages/redcap/methods/records.py", line 189, in export_records
fields = self._backfill_fields(fields, forms)
File "/venv/lib64/python3.9/site-packages/redcap/methods/records.py", line 47, in _backfill_fields
return self.field_names + [f"{ form }_complete" for form in self.forms]
File "/venv/lib64/python3.9/site-packages/redcap/methods/base.py", line 100, in field_names
self._field_names = self._filter_metadata(key="field_name")
File "/venv/lib64/python3.9/site-packages/redcap/methods/base.py", line 254, in _filter_metadata
res = [row[key] for row in self.metadata]
File "/data/xnat-phrt/venv/lib64/python3.9/site-packages/redcap/methods/base.py", line 80, in metadata
self._metadata = cast(Json, self._call_api(payload, return_type="json"))
File "venv-/lib64/python3.9/site-packages/redcap/methods/base.py", line 437, in _call_api
return rcr.execute(
File "/venv/lib64/python3.9/site-packages/redcap/request.py", line 189, in execute
content = self.get_content(
File "/venv/lib64/python3.9/site-packages/redcap/request.py", line 153, in get_content
return response.json()
File "venv/lib64/python3.9/site-packages/requests/models.py", line 978, in json
raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

from pycap.

Hi Maria,

Can you try running the sample Python code in the API playground to export records and seeing if that works or not?

And can you report what version of REDCap you're using? Because if it's the latest maybe I can test it out again since it's been a while

from pycap.

I tried the code in the API playground and does not work. I also tried to reach the API via curl to aassure the endpoint is correct and that seems to work.

And the REDCap version is v14.1.0 (I need to stick to this, the management does not depend of me)

I also tried to traceback. It seems the header from the REDCap API get response changed the format (maybe schema, or space in the response) and It is not able to decode the response as JSON compatible string

Thanks in advance fr your support!

from pycap.

I also realized the test suite hadn't ran in like 8 months, so I just re-ran it manually and everything passed (including tests related to exporting records) so I'm pretty convinced this is a redcap versioning issue

from pycap.