brightway-lca / brightway2-regional Goto Github PK

Hello,

I am trying to create intersections in bw2regional, and having trouble. For example for the (ecoinvent, world) intersection (gpkg downloaded with bwr.bw2regionalsetup())
For each calculation engine possible, there is a problem:

• the pandarus remote server is not available
• the pandarus library says the two geocollections are overlapping
• the geopandas library manage to create the intersection very fast compared to the others. The resulting df provides datas. However, when loading the npy file to create the geo_transform_matrix during an lca, the array is empty (all row index are equal to MAX_INT_32).
  So has anyone managed to get an intersection file working ? (particularly with geopandas ?)

thanks a lot !
(@cmutel )

Hello,

I am trying the brightway2-regional library in Brightway2

I have installed packages required by using conda install as instance:
conda create -y -n ab_dev python=3.8.10 activity-browser-dev brightway2 jupyter
conda activate ab_dev
conda install -y -q fiona rasterio geopandas bw2regional pandarus bw2-lcimpact folium

for the bw2-lcimpact, the 0.1.1 versions has been installed.

So I tested the regionalization tools by running in a jupyter notebok the following commands (after creating a project and installed biosphere3 and ecoinvent 3.7.1 cutoff databases) :

• 1. reg.bw2regionalsetup() :
     • I got the following errors : HTTPSConnectionPool(host='geograpy.ecoinvent.org', port=443): Max retries exceeded with
       url: /files/all-ecoinvent.pkg (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at
       0x000001E3B03A9A30>: Failed to establish a new connection [Errno 11001] getaddring failed

  So, I could solved this problem by catching this errors with try.. except statement in the utils.py files of the bw2data package
  and by downloading the files all-
  ecoinvent.gpk and countries.gpkg manually

• 2. import_regionalized_lcimpact():
     - I got the following error: Can't import data from pandarus remote

      So I tried to identify the origin from this error by removing the try.. except statement catching this  error in the 
      main.py file  of bw-lcimpact package and I got the following error:   
     
       HTTPSConnectionPool(host='pandarus.brightwaylca.org', 
     port=443): Max retries exceeded with url: / (Caused by SSLError(SSLCertVerificationError(1, '[SSL: 
     CERTIFICATE_VERIFY_FAILED]   certificate  verify failed: self signed certificate in certificate chain (_ssl.c:1131)')))
     
     I upgrade the bw2-lcimpact to the version 0.3.1 by using the command pip install (conda install didn't work), but I got 
     the same error. 
     

So I couldn't solve this second problem, because the url "pandarus.brightwaylca.org" seems not to be available.

So I hope my explanation was clear?

Thank you for your help.

Best regards,

Please run the utility function bw2regionalsetup() in each new project that will do regionalized LCA calculations.

Any version of ecoinvent will need to be processed with the utility function fix_ecoinvent_database("some ecoinvent database name"). This function will do the following:

    Convert any Rest-of-World locations into actual world regions and subsequently relabel them.
    Relabel any ecoinvent-specific locations names to place them inside the ecoinvent geocollection.

Unable to find which module contains the fix_ecoinvent_database function. Thanks.

Hello. I am trying to create a conda environment with bw2regional and bw25. First I tried to simply install bw2regional but alongside it installed older versions of brightway. So I tried adding the requirements of brightway25 repo to a conda environment (fixing the version of numpy to get around the np.bool issue). Something like this:

name: lca_reg
channels:
  - conda-forge
  - cmutel
  - konstantinstadler

dependencies:
- numpy==1.23.5
- bw2analyzer>=0.11.1
- bw2calc>=2.0.dev5
- bw2data>=4.0.dev11
- bw2io>=0.9.dev6
- bw2parameters>=0.6.6
- bw_processing>=0.7.1
- matrix_utils>=0.2.2
- mrio_common_metadata>=0.2
- bw_migrations>=0.2
- scipy
- stats_arrays>=0.6.5
- bw2regional
- jupyterlab

but the installation resulted in conflicted packages. Any advice on how to do this smootly ?

Original report by Estelle Louineau (Bitbucket: estelle_louineau, ).

Hi,
I need to generate geocollections covering countries, ecoinvent areas, provinces, territories and RoWs.

I did:
"import bw2regional as bwR ;
bwR.bw2regionalsetup()",
it answered:
"Downloading and creating world geocollections
Adding world topology
Adding ecoinvent-specific topology"

Then I checked what geocollections had been created by:
"bwR.geocollections",
and I got:
"Geocollections dictionary with 2 object(s):
ecoinvent
world"

Trying to understand why the RoW geocollection hadn't been generated, I came to the following question:

In base_data.py, why is all the code related to other geocollections than "world" and "ecoinvent" commented?

It starts at: https://bitbucket.org/cmutel/brightway2-regional/src/565000fd47052dece5d523d5a295910a6dedba7c/bw2regional/base_data.py#lines-30

If I un-comment it, will it work "correctly"?

Also, I have another question about the geocollections generated by bw2regionalsetup():
Why aren't the provinces and states (of Canada for example, but none is) present in "world" geocollection or "ecoinvent" geocollection?

I opened the geocollections in QGIS and then exported the attributes table to Excel to visualize what was included and in "world" there are only countries, and in "ecoinvent" only the specific regions such as "Canada without Quebec".

Original report by Chris Mutel (Bitbucket: cmutel, GitHub: cmutel).

Causes issues on Windows (255 character limit).

Original report by Chris Mutel (Bitbucket: cmutel, GitHub: cmutel).

Noticed in PSIE class

Original report by Chris Mutel (Bitbucket: cmutel, GitHub: cmutel).

Needed for activity browser.

Please, can someone help me with the following issue when running "create_regionalized_intersections"?:

Intersection for world and watersheds-aware already calculated
Calculation job submitted.

gaierror Traceback (most recent call last)
~\Miniconda3\envs\bw25reg\lib\site-packages\urllib3\connection.py in _new_conn(self)
173 try:
--> 174 conn = connection.create_connection(
175 (self._dns_host, self.port), self.timeout, **extra_kw

~\Miniconda3\envs\bw25reg\lib\site-packages\urllib3\util\connection.py in create_connection(address, timeout, source_address, socket_options)
72
---> 73 for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
74 af, socktype, proto, canonname, sa = res

~\Miniconda3\envs\bw25reg\lib\socket.py in getaddrinfo(host, port, family, type, proto, flags)
953 addrlist = []
--> 954 for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
955 af, socktype, proto, canonname, sa = res

gaierror: [Errno 11001] getaddrinfo failed

During handling of the above exception, another exception occurred:

NewConnectionError Traceback (most recent call last)
~\Miniconda3\envs\bw25reg\lib\site-packages\urllib3\connectionpool.py in urlopen(self, method, url, body, headers, retries, redirect, assert_same_host, timeout, pool_timeout, release_conn, chunked, body_pos, **response_kw)
698 # Make the request on the httplib connection object.
--> 699 httplib_response = self._make_request(
700 conn,

~\Miniconda3\envs\bw25reg\lib\site-packages\urllib3\connectionpool.py in _make_request(self, conn, method, url, timeout, chunked, **httplib_request_kw)
381 try:
--> 382 self._validate_conn(conn)
383 except (SocketTimeout, BaseSSLError) as e:

~\Miniconda3\envs\bw25reg\lib\site-packages\urllib3\connectionpool.py in _validate_conn(self, conn)
1009 if not getattr(conn, "sock", None): # AppEngine might not have .sock
-> 1010 conn.connect()
1011

~\Miniconda3\envs\bw25reg\lib\site-packages\urllib3\connection.py in connect(self)
357 # Add certificate verification
--> 358 conn = self._new_conn()
359 hostname = self.host

~\Miniconda3\envs\bw25reg\lib\site-packages\urllib3\connection.py in _new_conn(self)
185 except SocketError as e:
--> 186 raise NewConnectionError(
187 self, "Failed to establish a new connection: %s" % e

NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x00000242B870B9A0>: Failed to establish a new connection: [Errno 11001] getaddrinfo failed

During handling of the above exception, another exception occurred:

MaxRetryError Traceback (most recent call last)
~\Miniconda3\envs\bw25reg\lib\site-packages\requests\adapters.py in send(self, request, stream, timeout, verify, cert, proxies)
438 if not chunked:
--> 439 resp = conn.urlopen(
440 method=request.method,

~\Miniconda3\envs\bw25reg\lib\site-packages\urllib3\connectionpool.py in urlopen(self, method, url, body, headers, retries, redirect, assert_same_host, timeout, pool_timeout, release_conn, chunked, body_pos, **response_kw)
754
--> 755 retries = retries.increment(
756 method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]

~\Miniconda3\envs\bw25reg\lib\site-packages\urllib3\util\retry.py in increment(self, method, url, response, error, _pool, _stacktrace)
573 if new_retry.is_exhausted():
--> 574 raise MaxRetryError(_pool, url, error or ResponseError(cause))
575

MaxRetryError: HTTPSConnectionPool(host='pandarus.brightway.dev%3c!doctype%20html%20public%20%22-', port=443): Max retries exceeded with url: //W3C//DTD%20HTML%203.2%20Final//EN%22%3E%0A%3Ctitle%3E409%20Conflict%3C/title%3E%0A%3Ch1%3EConflict%3C/h1%3E%0A%3Cp%3EThis%20intersection%20already%20exists%3C/p%3E%0A (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x00000242B870B9A0>: Failed to establish a new connection: [Errno 11001] getaddrinfo failed'))

During handling of the above exception, another exception occurred:

ConnectionError Traceback (most recent call last)
~\Miniconda3\envs\bw25reg\lib\site-packages\bw2regional\pandarus_remote.py in poll(self, interval)
49 while True:
---> 50 if self.status not in {"failed", "finished", "forgotten"}:
51 time.sleep(interval)

~\Miniconda3\envs\bw25reg\lib\site-packages\bw2regional\pandarus_remote.py in status(self)
40 def status(self):
---> 41 response = requests.get(self.url)
42 if response.status_code != 404:

~\Miniconda3\envs\bw25reg\lib\site-packages\requests\api.py in get(url, params, **kwargs)
74
---> 75 return request('get', url, params=params, **kwargs)
76

~\Miniconda3\envs\bw25reg\lib\site-packages\requests\api.py in request(method, url, **kwargs)
60 with sessions.Session() as session:
---> 61 return session.request(method=method, url=url, **kwargs)
62

~\Miniconda3\envs\bw25reg\lib\site-packages\requests\sessions.py in request(self, method, url, params, data, headers, cookies, files, auth, timeout, allow_redirects, proxies, hooks, stream, verify, cert, json)
541 send_kwargs.update(settings)
--> 542 resp = self.send(prep, **send_kwargs)
543

~\Miniconda3\envs\bw25reg\lib\site-packages\requests\sessions.py in send(self, request, **kwargs)
654 # Send the request
--> 655 r = adapter.send(request, **kwargs)
656

~\Miniconda3\envs\bw25reg\lib\site-packages\requests\adapters.py in send(self, request, stream, timeout, verify, cert, proxies)
515
--> 516 raise ConnectionError(e, request=request)
517

ConnectionError: HTTPSConnectionPool(host='pandarus.brightway.dev%3c!doctype%20html%20public%20%22-', port=443): Max retries exceeded with url: //W3C//DTD%20HTML%203.2%20Final//EN%22%3E%0A%3Ctitle%3E409%20Conflict%3C/title%3E%0A%3Ch1%3EConflict%3C/h1%3E%0A%3Cp%3EThis%20intersection%20already%20exists%3C/p%3E%0A (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x00000242B870B9A0>: Failed to establish a new connection: [Errno 11001] getaddrinfo failed'))

During handling of the above exception, another exception occurred:

gaierror Traceback (most recent call last)
~\Miniconda3\envs\bw25reg\lib\site-packages\urllib3\connection.py in _new_conn(self)
173 try:
--> 174 conn = connection.create_connection(
175 (self._dns_host, self.port), self.timeout, **extra_kw

~\Miniconda3\envs\bw25reg\lib\site-packages\urllib3\util\connection.py in create_connection(address, timeout, source_address, socket_options)
72
---> 73 for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
74 af, socktype, proto, canonname, sa = res

~\Miniconda3\envs\bw25reg\lib\socket.py in getaddrinfo(host, port, family, type, proto, flags)
953 addrlist = []
--> 954 for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
955 af, socktype, proto, canonname, sa = res

gaierror: [Errno 11001] getaddrinfo failed

During handling of the above exception, another exception occurred:

NewConnectionError Traceback (most recent call last)
~\Miniconda3\envs\bw25reg\lib\site-packages\urllib3\connectionpool.py in urlopen(self, method, url, body, headers, retries, redirect, assert_same_host, timeout, pool_timeout, release_conn, chunked, body_pos, **response_kw)
698 # Make the request on the httplib connection object.
--> 699 httplib_response = self._make_request(
700 conn,

~\Miniconda3\envs\bw25reg\lib\site-packages\urllib3\connectionpool.py in _make_request(self, conn, method, url, timeout, chunked, **httplib_request_kw)
381 try:
--> 382 self._validate_conn(conn)
383 except (SocketTimeout, BaseSSLError) as e:

~\Miniconda3\envs\bw25reg\lib\site-packages\urllib3\connectionpool.py in _validate_conn(self, conn)
1009 if not getattr(conn, "sock", None): # AppEngine might not have .sock
-> 1010 conn.connect()
1011

~\Miniconda3\envs\bw25reg\lib\site-packages\urllib3\connection.py in connect(self)
357 # Add certificate verification
--> 358 conn = self._new_conn()
359 hostname = self.host

~\Miniconda3\envs\bw25reg\lib\site-packages\urllib3\connection.py in _new_conn(self)
185 except SocketError as e:
--> 186 raise NewConnectionError(
187 self, "Failed to establish a new connection: %s" % e

NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x00000242B4C7AE80>: Failed to establish a new connection: [Errno 11001] getaddrinfo failed

During handling of the above exception, another exception occurred:

MaxRetryError Traceback (most recent call last)
~\Miniconda3\envs\bw25reg\lib\site-packages\requests\adapters.py in send(self, request, stream, timeout, verify, cert, proxies)
438 if not chunked:
--> 439 resp = conn.urlopen(
440 method=request.method,

~\Miniconda3\envs\bw25reg\lib\site-packages\urllib3\connectionpool.py in urlopen(self, method, url, body, headers, retries, redirect, assert_same_host, timeout, pool_timeout, release_conn, chunked, body_pos, **response_kw)
754
--> 755 retries = retries.increment(
756 method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]

~\Miniconda3\envs\bw25reg\lib\site-packages\urllib3\util\retry.py in increment(self, method, url, response, error, _pool, _stacktrace)
573 if new_retry.is_exhausted():
--> 574 raise MaxRetryError(_pool, url, error or ResponseError(cause))
575

MaxRetryError: HTTPSConnectionPool(host='pandarus.brightway.dev%3c!doctype%20html%20public%20%22-', port=443): Max retries exceeded with url: //W3C//DTD%20HTML%203.2%20Final//EN%22%3E%0A%3Ctitle%3E409%20Conflict%3C/title%3E%0A%3Ch1%3EConflict%3C/h1%3E%0A%3Cp%3EThis%20intersection%20already%20exists%3C/p%3E%0A (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x00000242B4C7AE80>: Failed to establish a new connection: [Errno 11001] getaddrinfo failed'))

During handling of the above exception, another exception occurred:

ConnectionError Traceback (most recent call last)
~\AppData\Local\Temp/ipykernel_780/103464265.py in
----> 1 aware.create_regionalized_intersections()

~\Miniconda3\envs\bw25reg\lib\site-packages\bw2_aware_init_.py in create_regionalized_intersections()
28 try:
29 job = remote.calculate_intersection("world", "watersheds-aware")
---> 30 job.poll(interval=2)
31 if job.status != "finished":
32 raise ValueError(

~\Miniconda3\envs\bw25reg\lib\site-packages\bw2regional\pandarus_remote.py in poll(self, interval)
55 pass
56 finally:
---> 57 print("\nJob ended with status '{}'".format(self.status))
58
59

~\Miniconda3\envs\bw25reg\lib\site-packages\bw2regional\pandarus_remote.py in status(self)
39 @Property
40 def status(self):
---> 41 response = requests.get(self.url)
42 if response.status_code != 404:
43 return response.text

~\Miniconda3\envs\bw25reg\lib\site-packages\requests\api.py in get(url, params, **kwargs)
73 """
74
---> 75 return request('get', url, params=params, **kwargs)
76
77

~\Miniconda3\envs\bw25reg\lib\site-packages\requests\api.py in request(method, url, **kwargs)
59 # cases, and look like a memory leak in others.
60 with sessions.Session() as session:
---> 61 return session.request(method=method, url=url, **kwargs)
62
63

~\Miniconda3\envs\bw25reg\lib\site-packages\requests\sessions.py in request(self, method, url, params, data, headers, cookies, files, auth, timeout, allow_redirects, proxies, hooks, stream, verify, cert, json)
540 }
541 send_kwargs.update(settings)
--> 542 resp = self.send(prep, **send_kwargs)
543
544 return resp

~\Miniconda3\envs\bw25reg\lib\site-packages\requests\sessions.py in send(self, request, **kwargs)
653
654 # Send the request
--> 655 r = adapter.send(request, **kwargs)
656
657 # Total elapsed time of the request (approximately)

~\Miniconda3\envs\bw25reg\lib\site-packages\requests\adapters.py in send(self, request, stream, timeout, verify, cert, proxies)
514 raise SSLError(e, request=request)
515
--> 516 raise ConnectionError(e, request=request)
517
518 except ClosedPoolError as e:

ConnectionError: HTTPSConnectionPool(host='pandarus.brightway.dev%3c!doctype%20html%20public%20%22-', port=443): Max retries exceeded with url: //W3C//DTD%20HTML%203.2%20Final//EN%22%3E%0A%3Ctitle%3E409%20Conflict%3C/title%3E%0A%3Ch1%3EConflict%3C/h1%3E%0A%3Cp%3EThis%20intersection%20already%20exists%3C/p%3E%0A (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x00000242B4C7AE80>: Failed to establish a new connection: [Errno 11001] getaddrinfo failed'))

When I call remote.intersection I get this error linked to SSL certificate :

SSLError: HTTPSConnectionPool(host='pandarus.brightwaylca.org', port=443): Max retries exceeded with url: /?verify=False (Caused by SSLError(SSLCertVerificationError("hostname 'pandarus.brightwaylca.org' doesn't match either of '*.parkingcrew.net', 'parkingcrew.net'")))

It seems that the website pandarus.brightwaylca.org is down.

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

This repository currently has no open or pending branches.

Detected dependencies

• Check this box to trigger a request for Renovate to run again on this repository