Unable to open database file about purple_air_api HOT 22 CLOSED

Can you list the full trace and the code you are trying to run? This sounds like an issue with requests_cache which creates a sql database to cache API request results.

from purple_air_api.

I installed it with pip install from within the PyScripter IDE. Here's the trace. Thanks.

from purpleair.network import SensorList
Traceback (most recent call last):
File "", line 1, in
File "C:\Users\xxxxx\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone2\lib\site-packages\purpleair\network.py", line 15, in
from .sensor import Sensor
File "C:\Users\xxxxx\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone2\lib\site-packages\purpleair\sensor.py", line 20, in
requests_cache.core.remove_expired_responses()
File "C:\Users\xxxxx\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone2\lib\site-packages\requests_cache\core.py", line 287, in remove_expired_responses
return requests.Session().remove_expired_responses()
File "C:\Users\xxxxx\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone2\lib\site-packages\requests_cache\core.py", line 212, in init
**backend_options
File "C:\Users\xxxxx\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone2\lib\site-packages\requests_cache\core.py", line 73, in init
self.cache = backends.create_backend(backend, cache_name, backend_options)
File "C:\Users\xxxxx\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone2\lib\site-packages\requests_cache\backends_init_.py", line 63, in create_backend
return registry[backend_name](cache_name, **options)
File "C:\Users\xxxxx\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone2\lib\site-packages\requests_cache\backends\sqlite.py", line 28, in init
self.responses = DbPickleDict(location + extension, 'responses', fast_save=fast_save)
File "C:\Users\xxxxx\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone2\lib\site-packages\requests_cache\backends\storage\dbdict.py", line 62, in init
with self.connection() as con:
File "C:\Users\xxxxx\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone2\lib\contextlib.py", line 81, in enter
return next(self.gen)
File "C:\Users\xxxxx\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone2\lib\site-packages\requests_cache\backends\storage\dbdict.py", line 74, in connection
con = sqlite.connect(self.filename)
sqlite3.OperationalError: unable to open database file

from purple_air_api.

Okay, I think I know what the problem is. Are you able to run the following code in the same environment:

import requests
import requests_cache

requests_cache.install_cache('demo_cache')

from purple_air_api.

Thank you for your rapid responses. At first I did not have the request_cache library but installed it and get the following. Do you think I should try a different environment or install it differently? Thanks.

*** Python 3.6.10 |Anaconda, Inc.| (default, May 20 2020, 01:49:13) [MSC v.1916 64 bit (AMD64)] on win32. ***
*** Remote Python engine is active ***

import requests
import requests_cache
requests_cache.install_cache('demo_cache')
from purpleair.network import SensorList
Traceback (most recent call last):
File "", line 1, in
File "C:\Users\xxxxx\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone2\lib\site-packages\purpleair\network.py", line 15, in
from .sensor import Sensor
File "C:\Users\xxxxx\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone2\lib\site-packages\purpleair\sensor.py", line 20, in
requests_cache.core.remove_expired_responses()
File "C:\Users\xxxxx\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone2\lib\site-packages\requests_cache\core.py", line 287, in remove_expired_responses
return requests.Session().remove_expired_responses()
File "C:\Users\xxxxx\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone2\lib\site-packages\requests_cache\core.py", line 212, in init
**backend_options
File "C:\Users\xxxxx\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone2\lib\site-packages\requests_cache\core.py", line 73, in init
self.cache = backends.create_backend(backend, cache_name, backend_options)
File "C:\Users\xxxxx\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone2\lib\site-packages\requests_cache\backends_init_.py", line 63, in create_backend
return registry[backend_name](cache_name, **options)
File "C:\Users\xxxxx\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone2\lib\site-packages\requests_cache\backends\sqlite.py", line 28, in init
self.responses = DbPickleDict(location + extension, 'responses', fast_save=fast_save)
File "C:\Users\xxxxx\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone2\lib\site-packages\requests_cache\backends\storage\dbdict.py", line 62, in init
with self.connection() as con:
File "C:\Users\xxxxx\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone2\lib\contextlib.py", line 81, in enter
return next(self.gen)
File "C:\Users\xxxxx\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone2\lib\site-packages\requests_cache\backends\storage\dbdict.py", line 74, in connection
con = sqlite.connect(self.filename)
sqlite3.OperationalError: unable to open database file

from purple_air_api.

As a temporary solution, I can wrap the cache code in purpleair in a try/except block to catch this error, but it is probably an issue with your environment missing a SQLite dependency.

You can open a ticket for requests_cache here and see if they have any ideas.

from purple_air_api.

If you want to fix this locally yourself you can just remove lines 18 and 19 from this file.

Judging by your traceback, the file path should be C:\Users\xxxxx\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone2\lib\site-packages\purpleair\network.py.

from purple_air_api.

I tried a different non-Esri environment and commented out the two lines and get following with the same error. This may be a naive question, but would sqlite be on my computer from install python or something else?

*** Python 3.7.7 (default, May 6 2020, 11:45:54) [MSC v.1916 64 bit (AMD64)] on win32. ***

import purpleair
from purpleair.network import SensorList
Traceback (most recent call last):
File "", line 1, in
File "C:\ProgramData\Anaconda3\lib\site-packages\purpleair\network.py", line 15, in
from .sensor import Sensor
File "C:\ProgramData\Anaconda3\lib\site-packages\purpleair\sensor.py", line 20, in
requests_cache.core.remove_expired_responses()
File "C:\ProgramData\Anaconda3\lib\site-packages\requests_cache\core.py", line 287, in remove_expired_responses
return requests.Session().remove_expired_responses()
File "C:\ProgramData\Anaconda3\lib\site-packages\requests_cache\core.py", line 212, in init
**backend_options
File "C:\ProgramData\Anaconda3\lib\site-packages\requests_cache\core.py", line 73, in init
self.cache = backends.create_backend(backend, cache_name, backend_options)
File "C:\ProgramData\Anaconda3\lib\site-packages\requests_cache\backends_init_.py", line 63, in create_backend
return registry[backend_name](cache_name, **options)
File "C:\ProgramData\Anaconda3\lib\site-packages\requests_cache\backends\sqlite.py", line 28, in init
self.responses = DbPickleDict(location + extension, 'responses', fast_save=fast_save)
File "C:\ProgramData\Anaconda3\lib\site-packages\requests_cache\backends\storage\dbdict.py", line 62, in init
with self.connection() as con:
File "C:\ProgramData\Anaconda3\lib\contextlib.py", line 112, in enter
return next(self.gen)
File "C:\ProgramData\Anaconda3\lib\site-packages\requests_cache\backends\storage\dbdict.py", line 74, in connection
con = sqlite.connect(self.filename)
sqlite3.OperationalError: unable to open database file

from purple_air_api.

According to your traceback the lines of code are still there:

File "C:\ProgramData\Anaconda3\lib\site-packages\purpleair\network.py", line 15, in 
from .sensor import Sensor
File "C:\ProgramData\Anaconda3\lib\site-packages\purpleair\sensor.py", line 20, in 
requests_cache.core.remove_expired_responses()

These are the lines you need to remove temporarily:

requests_cache.install_cache(expire_after=timedelta(hours=1))
requests_cache.core.remove_expired_responses()

This is either a bug requests_cache or a problem with your environment missing a sqlite dependency. You should open an issue in the requests_cache repo.

from purple_air_api.

Just to reiterate, you are unable to run the following code:

import requests
import requests_cache

requests_cache.install_cache('demo_cache')

This means the bug is not with purpleair but rather with requests_cache.

from purple_air_api.

Sorry I need to be better about reading the tracebacks. I found that if I only comment out
requests_cache.core.remove_expired_responses() in sensor.py and network.py
it works!!! Yea.
I confirmed it entering the line manually (assuming I don't need to load any other dependencies)

from datetime import timedelta
import requests_cache
requests_cache.install_cache(expire_after=timedelta(hours=1))
requests_cache.core.remove_expired_responses()
Traceback (most recent call last):
File "", line 1, in
File "C:\ProgramData\Anaconda3\lib\site-packages\requests_cache\core.py", line 287, in remove_expired_responses
return requests.Session().remove_expired_responses()
File "C:\ProgramData\Anaconda3\lib\site-packages\requests_cache\core.py", line 212, in init
**backend_options
File "C:\ProgramData\Anaconda3\lib\site-packages\requests_cache\core.py", line 73, in init
self.cache = backends.create_backend(backend, cache_name, backend_options)
File "C:\ProgramData\Anaconda3\lib\site-packages\requests_cache\backends_init_.py", line 63, in create_backend
return registry[backend_name](cache_name, **options)
File "C:\ProgramData\Anaconda3\lib\site-packages\requests_cache\backends\sqlite.py", line 28, in init
self.responses = DbPickleDict(location + extension, 'responses', fast_save=fast_save)
File "C:\ProgramData\Anaconda3\lib\site-packages\requests_cache\backends\storage\dbdict.py", line 62, in init
with self.connection() as con:
File "C:\ProgramData\Anaconda3\lib\contextlib.py", line 112, in enter
return next(self.gen)
File "C:\ProgramData\Anaconda3\lib\site-packages\requests_cache\backends\storage\dbdict.py", line 74, in connection
con = sqlite.connect(self.filename)
sqlite3.OperationalError: unable to open database file

from purple_air_api.

I am able to run
requests_cache.install_cache('demo_cache')

from purple_air_api.

Thanks for you help. What effect does the loss of requests_cache.core.remove_expired_responses() have?

from purple_air_api.

I left a comment with request_cache. Thanks for you help. The purpleair data is helpful with wildfires and especially with the current wildfires on the west coast now.

from purple_air_api.

According to their docs, remove_expired_responses() is supposed to "Remove expired responses from storage."

I am not sure what you lose by disabling this, so I will just push a new build that will optionally run this statement.

from purple_air_api.

Thank you.

from purple_air_api.

Okay, the updated version should be on PyPI. You can update with pip install --upgrade purpleair. Thank you for the report!

from purple_air_api.

Thank you. I'm sorry to report now that when I invoke p=SensorList() I get the following. I think it is time to rest for the weekend.

from purpleair.network import SensorList
Unable to purge cache database, cache may contain old data!!!
p = SensorList()
Traceback (most recent call last):
File "", line 1, in
File "C:\ProgramData\Anaconda3\lib\site-packages\purpleair\network.py", line 23, in init
self.get_all_data()
File "C:\ProgramData\Anaconda3\lib\site-packages\purpleair\network.py", line 34, in get_all_data
response = requests.get(f'{API_ROOT}')
File "C:\ProgramData\Anaconda3\lib\site-packages\requests\api.py", line 76, in get
return request('get', url, params=params, **kwargs)
File "C:\ProgramData\Anaconda3\lib\site-packages\requests\api.py", line 60, in request
with sessions.Session() as session:
File "C:\ProgramData\Anaconda3\lib\site-packages\requests_cache\core.py", line 212, in init
**backend_options
File "C:\ProgramData\Anaconda3\lib\site-packages\requests_cache\core.py", line 73, in init
self.cache = backends.create_backend(backend, cache_name, backend_options)
File "C:\ProgramData\Anaconda3\lib\site-packages\requests_cache\backends_init_.py", line 63, in create_backend
return registry[backend_name](cache_name, **options)
File "C:\ProgramData\Anaconda3\lib\site-packages\requests_cache\backends\sqlite.py", line 28, in init
self.responses = DbPickleDict(location + extension, 'responses', fast_save=fast_save)
File "C:\ProgramData\Anaconda3\lib\site-packages\requests_cache\backends\storage\dbdict.py", line 62, in init
with self.connection() as con:
File "C:\ProgramData\Anaconda3\lib\contextlib.py", line 112, in enter
return next(self.gen)
File "C:\ProgramData\Anaconda3\lib\site-packages\requests_cache\backends\storage\dbdict.py", line 74, in connection
con = sqlite.connect(self.filename)
sqlite3.OperationalError: unable to open database file

from purple_air_api.

Interesting, so even though the cache is installed you are unable to access it. I guess if we hit the case where we cannot purge the cache we should just disable the cache altogether. Luckily this is a simple fix.

from purple_air_api.

In your local copy, can you replace the entire contents of C:\ProgramData\Anaconda3\lib\site-packages\purpleair\api_data.py with this code:

"""
Constants for PurpleAir API
"""
from datetime import timedelta
from sqlite3 import OperationalError

import requests_cache

# Setup cache for requests
requests_cache.install_cache(expire_after=timedelta(hours=1))
try:
    requests_cache.core.remove_expired_responses()
except OperationalError:
    requests_cache.core.uninstall_cache()
    print('Unable to open cache or purge cache database, requests will not be cached!!!')


API_ROOT = 'https://www.purpleair.com/json'

from purple_air_api.

I was able to replicate your issue by making the cache.sqlite file that requests_cache writes inaccessible (chmod 0 cache.sqlite) and I can confirm that the above code resolves the issue.

from purple_air_api.

Version 1.0.5 resolves this along with a couple other fixes. Please update with pip install --upgrade purpleair. Again, thank you for the report!

from purple_air_api.

Thank you very much. It is working for me. Thanks again for putting together a great tool. Have a great weekend.

from purple_air_api.