el-bid / urbanpy Goto Github PK

This issue is mainly concerned with discussing the best strategy to add UrbanPy to Anaconda such that users can choose between pip and conda by using

conda install urbanpy

instead of pip install. The conda build documentation has a "from scratch" and a build with conda skeleton.

This process seems fairly integratabtle into a GitHub action as our tagged release with pypi, given that we use conda skeleton to build the updated meta.yaml file and then log into an anaconda account for urbanpy. @bitsandbricks @Claudio9701 let me know what you think. I believe there is a default conda package build/upload action within the actions gallery.

Run start_osrm_server succesfully

But the docker image console reports there is no data in the ~/data/osrm dir

osrm_route raise ConnectionError ... Max retries exceeded ...

Hi!
I have noticed that 'download' functions have been changed. However when I install urbanpy via pip install urbanpy in a Google Colab, I still have the function hdx_fb_population() available, and I can not use the new functions: get_hdx_dataset() or search_hdx_dataset().
This is not really an issue, but I am opening it because I would really appreciate to be notified when the package version is updated.
Thanks!

The scripts in charge of creating the OSRM server are not being downloaded after installing urbanpy.

• unix_download.sh
• windows_download.ps1

A simple solution could be downloading the bash files directly with python using requests to github file.

This function :

peru_hexs = up.geom.gen_hexagons(4, peru_limit_clean)

Generates this warning:

/Users/claudio/.pyenv/versions/peru-connect-env/lib/python3.9/site-packages/urbanpy/geom/geom.py:168: FutureWarning: Currently, index_parts defaults to True, but in the future, it will default to False to be consistent with Pandas. Use `index_parts=True` to keep the current behavior and True/False to silence the warning.
  city_poly = city.explode().reset_index(drop=True)

Solution:
Define if True or False is adequate for this operation.

Because the foot profile is harcoded in line 78, routing/routing.py:

docker run -t --name osrm_extract -v $(pwd):/data osrm/osrm-backend osrm-extract -p **/opt/foot.lua** /data/{country}-latest.osm.pbf;

maybe

start_osrm_server(country, continent)

can be modified as start_osrm_server(country, continent, **profile**) to accept one of {'foot', 'car', 'bicycle'} as does osrm_route(origin, destination, profile)

and then run the docker line pasting in the chosen profile

Hi! I'm recreating the demo notebook with the Mexico City, but I have an issue with "routing.start_osrm_server", I can't make this line of code to work, aparentlye it doesn't found a file, could you help me??

An image of the problem:

I tried running Demo.ipynb and I was not able.
The following function:
tqdm_notebook().pandas()
generates this error:
Widget Javascript not detected. It may not be installed or enabled properly.

I have checked and Ipywidgets was installed and enabled for Jupyter.

If I deleted tqdm:
from tqdm import tqdm_notebook tqdm_notebook().pandas()
and I changed progress_apply to apply:
distance_duration = hex_lima.apply( lambda row: up.routing.osrm_route( origin=row.geometry.centroid, destination = fs.iloc[row['nearest_food_facility_ix']]['geometry'] ), result_type='expand', axis=1, )

Demo.ipynb worked perfectly.

Should I set something different in order to use tqdm? Thanks!

Hard to know if there is an error or something when the function takes to much time.

urbanpy.download.hdx_fb_population solo permite descargar datos de algunos paises de latinoamerica - segun https://el-bid.github.io/urbanpy/urbanpy.download.html#urbanpy.download.hdx_fb_population solo Argentina, Bolivia, Brazil, Chile, Colombia, Ecuador, Paraguay, Peru y Uruguay.

Hay forma de usar los datos de otros paises que estan disponibles en la pagina de humdata?

Fix the windows PowerShell script to create the OSRM Server

Hi 👋🏻 hoping for some help on an error I'm running into when using urbanpy

Using code from the urbanpy_workshop.ipynb when I adapt the line below for Brazil:

pop_brazil = up.download.hdx_fb_population('brazil', 'full')

I get the error

ValueError                                Traceback (most recent call last)
Cell In [30], line 1
----> 1 pop_brazil = up.download.hdx_fb_population('brazil', 'full')

File /opt/homebrew/lib/python3.10/site-packages/urbanpy/download/download.py:182, in hdx_fb_population(country, map_type)
    180     return pd.concat([pd.read_csv(file) for file in dataset_dict[country][map_type]])
    181 else:
--> 182     return pd.read_csv(dataset_dict[country][map_type])

File /opt/homebrew/lib/python3.10/site-packages/pandas/util/_decorators.py:211, in deprecate_kwarg.<locals>._deprecate_kwarg.<locals>.wrapper(*args, **kwargs)
    209     else:
    210         kwargs[new_arg_name] = new_arg_value
--> 211 return func(*args, **kwargs)

File /opt/homebrew/lib/python3.10/site-packages/pandas/util/_decorators.py:317, in deprecate_nonkeyword_arguments.<locals>.decorate.<locals>.wrapper(*args, **kwargs)
    311 if len(args) > num_allow_args:
    312     warnings.warn(
    313         msg.format(arguments=arguments),
    314         FutureWarning,
    315         stacklevel=find_stack_level(inspect.currentframe()),
    316     )
--> 317 return func(*args, **kwargs)

File /opt/homebrew/lib/python3.10/site-packages/pandas/io/parsers/readers.py:950, in read_csv(filepath_or_buffer, sep, delimiter, header, names, index_col, usecols, squeeze, prefix, mangle_dupe_cols, dtype, engine, converters, true_values, false_values, skipinitialspace, skiprows, skipfooter, nrows, na_values, keep_default_na, na_filter, verbose, skip_blank_lines, parse_dates, infer_datetime_format, keep_date_col, date_parser, dayfirst, cache_dates, iterator, chunksize, compression, thousands, decimal, lineterminator, quotechar, quoting, doublequote, escapechar, comment, encoding, encoding_errors, dialect, error_bad_lines, warn_bad_lines, on_bad_lines, delim_whitespace, low_memory, memory_map, float_precision, storage_options)
...
-> 1744     raise ValueError(msg)
   1746 try:
   1747     return mapping[engine](f, **self.options)

ValueError: Invalid file path or buffer object type: <class 'list'>

When is substitute children or youth instead of full the code runs fine. It's hard to tell from the metadata for this hdx data what the new keyword might be for "full population data". Wondering if you have ideas for how I can get the full population data.

Note this also means the line below from the tutorial does not work:

pop_arg = up.download.hdx_fb_population('argentina', 'full')

But it gives a 404 error.

Code example

hex_df = urbanpy.geom.merge_shape_hex(hex_df, pop_df, agg={'population': 'sum'})

TODO: Complete error raised

Hi all 👋🏻

As I was going through the urbanpy tutorial today in UrbanPy_workshop.ipynb the step to install urbanpy:

pip install urbanpy

threw an error when trying to install the gdal dependency. The error was as follows.

gdal_config_error: [Errno 2] No such file or directory: 'gdal-config'
Could not find gdal-config. Make sure you have installed the GDAL native library and development headers.

I found this post to help guide how to fix this in macOS by installing gdal with brew.

Once that ran, I could install urbanpy (using pipenv) via:

pipenv install urbanpy

However, after activating my pipenv, I still needed to run

pip install urbanpy in the jupyter notebook.

Ayer asistía a Worshop que dieron por el BID, estoy tratando de cargar con datos de Chile, pero no funciona para Chile, si para otros países como Argentina o Perú:

pop_cl= up.download.hdx_fb_population('chile', 'full')

Alguna sugerencia, y/o procedimiento para cargar manualmente, muchas gracias.

Hi,

In routing.py line 702 ox.get_nearest_nodes should be ox.nearest_nodes and I guess x and y should be reversed as well.

Currrent behaviour:

pop_search = up.download.search_hdx_dataset(country)
pop_country = up.download.get_hdx_dataset(pop_search, pop_index) # This takes too much time

Requested feature:

pop_search = up.download.search_hdx_dataset(country)
pop_country = up.download.get_hdx_dataset(pop_search, pop_index, mask=city_limits) # This is expected to be faster

Where city_limits represent either the city boundaries as a polygon or the city total bounds as a bounding box.

Hi all 👋🏻 I've got an edge case I wanted to present w/ starting up the OSRM server in a GitHub codespace

If I accidentally (😭) run up.routing.start_osrm_server('sul', 'south-america_brazil', 'foot'); without running the code below first

$ mkdir -p ~/data/osrm/
$ cd ~/data/osrm
$ wget https://download.geofabrik.de/south-america/brazil/sul-latest.osm.pbf

I get the output Server was started successfully and it even looks like the server starts at port 5000. However, I don't see all the output from docker in the jupyter notebook.

To get the server to really start successfully, I had to re-run the not just the code chunk above in the terminal, but I also had to do a full rebuild of the container in codespaces. After that rebuild, I had a successful server start in approx 2.5 minutes.

CC: @bitsandbricks, @Claudio9701