pysal / giddy Goto Github PK

Hi,

may I start by apologising if this isn't the right place to raise this issue. This might not be a 'bug' and I thought of putting this on the opensapce-list, let me know if that's more appropriate.

I am using the new feature of giddy that have the Markov states already discretised. The .gal file was made in GeoDa, edited by me to connect some islands, and then re-used in GeoDa to check that the new connections worked. When I read the .gal file using giddy it appears to read in with no errors. However when I run the spatial Markov code I get the following traceback. There is little guidance on what the issue might be, A KeyError means that something (‘1’?) is missing?

Traceback (most recent call last):

File "", line 1, in
runfile('D:/SDC/TOAC/giddy/TOAC.py', wdir='D:/SDC/TOAC/giddy')

File "C:\ProgramData\Anaconda3\lib\site-packages\spyder\utils\site\sitecustomize.py", line 710, in runfile
execfile(filename, namespace)

File "C:\ProgramData\Anaconda3\lib\site-packages\spyder\utils\site\sitecustomize.py", line 101, in execfile
exec(compile(f.read(), filename, 'exec'), namespace)

File "D:/SDC/TOAC/giddy/TOAC.py", line 20, in
sm=gd.Spatial_Markov(q8, w, discrete=True)

File "C:\ProgramData\Anaconda3\lib\site-packages\giddy-1.0.2-py3.6.egg\giddy\markov.py", line 409, in init
T, P = self._calc(y, w, classes, k=k)

File "C:\ProgramData\Anaconda3\lib\site-packages\giddy-1.0.2-py3.6.egg\giddy\markov.py", line 518, in _calc
ly = ps.lag_categorical(w,y)

File "C:\ProgramData\Anaconda3\lib\site-packages\libpysal\weights\spatial_lag.py", line 160, in lag_categorical
return np.vstack([lag_categorical(w,col) for col in y.T]).T

File "C:\ProgramData\Anaconda3\lib\site-packages\libpysal\weights\spatial_lag.py", line 160, in
return np.vstack([lag_categorical(w,col) for col in y.T]).T

File "C:\ProgramData\Anaconda3\lib\site-packages\libpysal\weights\spatial_lag.py", line 172, in lag_categorical
output[w.id2i[int(idx)]] = keys[int(outidx)]

KeyError: 1

The tests are failing
https://github.com/pysal/giddy/actions/runs/9648038464/job/26608252448#step:7:194

It seems to be due to the changed behavior of the output of numpy - in addition to outputting the value, the data type is output as well. Any thoughts on resolving this?

It would be great to have a method to use a trained spatial Markov chain to predict the next n states for a set of geometries.

sm = pysal.Spatial_Markov(data, weights, fixed = True, k = 7)
prediction = sm.predict( inital_state,  weights= weights, steps=10, k=7)

where inital_state would be a list of the current state of each regions and prediction would be a list of lists which would contain the predictions for the next n steps for each region.

In addition there could be a function

prediction = sm.predict_ensembl( inital_state,  weights= weights, steps=10, trials=100, k=7)

where trials is the number of randomly seeded predictions to make. The result would be a list of the mean and variance of the predictions for the next N steps

Where did the spatial_dynamics.interaction functions relocate to in the PySAL migration?

A bug in pysal.spatial_dynamics.rank class Tau: def _calc(self, x, y). Ties are counted incorrectly. Cause: initialization is made with ECount = 0, but should be ECount = 1

As per pysal/libpysal#178 and pysal/libpysal#181, "development guidelines" link in README.md is broken in the Contribute section.

from: http://pysal.readthedocs.io/en/latest/developers/index.html
to: https://github.com/pysal/pysal/wiki

Two minor issues need to be addressed:

• the link for Python Package Index goes to the pysal release, not the giddy release
• Eli's last name is not capitalized in the BibTeX citation

Hello, I am experiencing difficulties with "giddy.markov.Homogeneity_Results" command and wondering if I can get some help from giddy developers.

I am using the income information to estimate two classic Markov transition matrices (discrete Markov matrix using 5 quantiles) for two different periods. The first period is from the year 1981 to 1996 and the second period is from the year 2001 to 2016.

Below are the two transition probability matrices that I estimated. cm_p1 is a classic Markov's transition matrix using the income information of the period 1981-1996 and cm_p2 is a matrix using the income information of the period 2001-2016.

I tried to run the Homogeneity test (LR test and chi-square test) between these two Markov transition matrices using the "giddy.markov.Homogeneity_Results" command.

Based on the above information, it seems like I can conduct the homogeneity test using the "giddy.markov.Homogeneity_Results(cm_p1.p, cm_p2.p)" code. But when I run it, it shows an error message saying "not enough values to unpack (expected 3, got 2)"

I am a bit confused about what caused this error. Could anyone possibly help me with this problem?

I am seeing 2 failures locally when splot is not installed. Seems like we should try to get these failures to be bit more graceful or prevent them from happening.

FAILED giddy/tests/test_directional.py::Rose_Tester::test_rose - TypeError: cannot unpack non-iterable NoneType object
FAILED giddy/tests/test_directional.py::i::test_rose - TypeError: cannot unpack non-iterable NoneType object

I'd like to cut a fresh release (v2.3.4) for this month's PySAL meta-release.

Several items to decide for this release:

• Do we want to add gha workflow for publishing docs? #179
• Do we want to replace the current name giddy with estda (exploratory spatial-temporal data analysis) to increase its visibility?
• Anything else?

Following the merge of #199, we'll want to modernize the installation infrastructure.

xref:

• pysal/pointpats#117
• pysal/mapclassify#193
• etc.

@weikang9009 Does this sound OK to you?

• black repo – #201
• ruff repo – #202
• pre-commit for formatting and linting – #202
• setup tool_scm for versioning – #203
• prune files – #203
• etc.

It is possible that we run into non-ergodic Markov chains, e.g. it might be impossible to migrate from one Markov state to another. Right now, the steady_state function for calculating the limiting distribution and the fmpt function calculating first mean passage times can only deal with non-ergodic Markov chains. It will be useful to extend these functions to deal with non-ergodic Markov chains.

The Markov Chain is irreducible and is composed by:
1 Recurrent class (indices):
[0 1 2 3 4]
0 Transient class.
The Markov Chain has 0 absorbing state.

Should read:

The Markov Chain is irreducible and is composed by:
1 Recurrent class (indices):
[0 1 2 3 4]

Hello,

This is neither a bug nor a feature request but I hope that someone might take pity on me and answer this query. Is giddy.markov.Spatial_Markov stochastic in any way? The reason I ask is that I run the function:

sm = giddy.markov.Spatial_Markov(c,w,fixed=False,discrete=True,k=5,m=5)

and when I print out the F object

print(sm.F)

it changes from run to run. If it is not stochastic then I a bit flummoxed as to this behaviour. If it is, how to I fix the random seed? I have tried:

import random
random.seed(1)
sm = giddy.markov.Spatial_Markov(c,w,fixed=False,discrete=True,k=5,m=5)

the the 'random' feature persists.

Thanks.

the dependencies arent accurate (and dont match pyproject.toml)

remove api.py to be consistent with other pysal submodules.

How about instead of giddy, we go with

GeoDiDy = Geospatial DIstribution DYnamics ?

Hello,

I now understand that in the case of ties for the spatial lag a random number is used to break the tie. To explore this issue more I am looking at the lclass_ids attribute. I get these outputs:

[[4 4 4 4 4 4]
 [4 4 4 4 4 4]
 [4 4 4 4 4 4]
 ...
 [1 1 1 0 0 1]
 [2 3 2 0 2 0]
 [3 3 3 3 2 2]]

However if for my final polygon, # 7264, I manually identify the rank of spatial lags (minus 1) for this polygon's neighbours I get these lags:

7264	7							
7023	7028	7029	7032	7036	7037	7039	Mode	
3	3	3	3	4	4	4	3	t1
4	2	4	3	4	4	4	4	t2
4	3	4	4	4	4	4	4	t3
4	3	4	3	4	4	4	4	t4
4	2	4	3	2	4	4	4	t5
3	2	3	2	3	4	4	3	t6

which for the final row of lclass_ids my calculation is

[3 4 4 4 4 3]

and not

[3 3 3 3 2 2]

We should add in testing environments for Python 3.11.

separate from pysal

Hello, I got this message :

AttributeError: module 'numpy' has no attribute 'int'.
`np.int` was a deprecated alias for the builtin `int`. To avoid this error in existing code, use `int` by itself. Doing this will not modify any behavior and is safe. When replacing `np.int`, you may wish to use e.g. `np.int64` or `np.int32` to specify the precision. If you wish to review your current use, check the release note link for additional information.
The aliases was originally deprecated in NumPy 1.20; for more details and guidance see the original release note at:
    https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations

I am not sure if this is a genuine error message or I am doing something dumb.

My data is panel data 32k rows (areas) by 6 columns (time periods). My input data is pre-processed into quintiles that are NOT pooled over the 6 time periods. Quintiles are calculated within each time period. Thus there are very near 20% of each quintile in each time period. I have used GeoDa to create the .gal file.

I run with the command:

m = giddy.markov.Spatial_Markov(c, w, fixed = False, discrete = True, k = 5 ,m = 5)

I believe that the values I have given to fixed and discrete are the correct ones for my situation. When I do this I get the error message above. The message says something has been deprecated. Thanks.

CI is failing for the following tests:

• test_markov.py::test_Spatial_Markov::test_cutoff
• test_sequence.py::test_Sequence_unequal

ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions.

Hi, everyone, recently i use the pysal package to implement the spatial_markov. I notice the struction document adopt the "us48.gal", however, the document doesn't give how it form. So, there i have two questions needed to answer.
(1), how i get the us48.gal, in other word, how is the us48 file formed?
(2), i want to use the spatial_markov by importing my own data, how should i do?
I so looking forward to receiving your answer.
The official document has listed as following:
https://giddy.readthedocs.io/en/latest/MarkovBasedMethods.html

For example directional.ipynb

--> 276     import matplotlib
    277 
    278     if gui and gui != 'auto':

ModuleNotFoundError: No module named 'matplotlib'

after checking with @weikang9009, I put the metadata together for a release on conda-forge, and i think things are in pretty good shape. If possible, would @weikang9009 and @sjsrey mind replying to this issue to confirm you're willing to be listed as maintainers of the recipe? If not, i'm still happy to do it solo

on a clean ubuntu box with python 3.5 (and 3.6) i'm hitting this:

root@334aa6b92338:/# source activate test-env0
(test-env0) root@334aa6b92338:/# pip install giddy
Collecting giddy
  Downloading https://files.pythonhosted.org/packages/14/ba/7455a5d0f6bb50d46fcca0b10a37c038ce26a886d6bbc11d9b09cef0cc88/giddy-1.1.1.tar.gz
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-wx3i8mg2/giddy/setup.py", line 18, in <module>
        long_description = file.read()
      File "/miniconda/envs/test-env0/lib/python3.5/encodings/ascii.py", line 26, in decode
        return codecs.ascii_decode(input, self.errors)[0]
    UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 1891: ordinal not in range(128)

https://pysal.org/giddy/

Hello, I have made some progress on my previous issue, with the help of a colleague to get libpysal 3.0.3 installed. I now get the message below when I try to print the spatial ERGODIC values. It would be good if I could be provided with some pointers as to what is causing the matrix singularity? I am happy to share the input data and code. Thanks.

Spatial ERGODIC
C:\Users\tra6sdc\AppData\Local\Continuum\anaconda3\lib\site-packages\giddy-1.0.2-py3.6.egg\giddy\markov.py:438: ComplexWarning: Casting complex values to real discards the imaginary part
S[i] = STEADY_STATE(p)
Traceback (most recent call last):

File "", line 1, in
runfile('E:/TOAC.py', wdir='E:')

File "C:\Users\tra6sdc\AppData\Local\Continuum\anaconda3\lib\site-packages\spyder\utils\site\sitecustomize.py", line 710, in runfile
execfile(filename, namespace)

File "C:\Users\tra6sdc\AppData\Local\Continuum\anaconda3\lib\site-packages\spyder\utils\site\sitecustomize.py", line 101, in execfile
exec(compile(f.read(), filename, 'exec'), namespace)

File "E:/TOAC.py", line 37, in
for fq in sm.F:

File "C:\Users\tra6sdc\AppData\Local\Continuum\anaconda3\lib\site-packages\giddy-1.0.2-py3.6.egg\giddy\markov.py", line 447, in F
F[i] = fmpt(np.asmatrix(p))

File "C:\Users\tra6sdc\AppData\Local\Continuum\anaconda3\lib\site-packages\giddy-1.0.2-py3.6.egg\giddy\ergodic.py", line 106, in fmpt
Z = la.inv(I - P + A)

File "C:\Users\tra6sdc\AppData\Local\Continuum\anaconda3\lib\site-packages\numpy\linalg\linalg.py", line 513, in inv
ainv = _umath_linalg.inv(a, signature=signature, extobj=extobj)

File "C:\Users\tra6sdc\AppData\Local\Continuum\anaconda3\lib\site-packages\numpy\linalg\linalg.py", line 90, in _raise_linalgerror_singular
raise LinAlgError("Singular matrix")

LinAlgError: Singular matrix

Clicking on any of the three images at https://pysal.org/giddy/ will get a 404.
Looks like the change from master to main didn't make it into the docs that get rendered?

We should update the master branch to the main branch to conform with the push for reducing problematic language.

ImportError: cannot import name 'generated_jit' from 'numba' (/home/serge/miniforge3/envs/workshop-pysal-wrsa24/lib/python3.10/site-packages/numba/__init__.py)

This is for giddy 2.3.5 pyhd8ed1ab_0 conda-forge

Possibly related

Hi, everyone, recently i use the pysal package to implement the spatial_markov. I notiice the structions document adopt the "us48.gal", however, the document don't give how it form. So, there i have two questions needed to answer.
(1), how i get the us48.gal, in other word, how is the us48 file formed?
(2), i want to use the spatial_markov by importing my own data, how should i do?
I so looking forward to receiving your answer.
The offial docement has listed as following:
https://giddy.readthedocs.io/en/latest/MarkovBasedMethods.html

Similar to what happens in libpysal.

There seems to be something going wrong with how giddy exposes its package version (at least when installed via conda/mamba on Ubuntu). MWE:

conda create -n TEST giddy
conda activate TEST
conda list giddy  # shows version 2.3.5
pip show giddy  # shows version 0.0.0
python -c "import giddy; print(giddy.__version__)"  # shows version 0.0.0
pip install giddy --upgrade
conda list giddy  # shows version 2.3.5
pip show giddy  # shows version 2.3.5
python -c "import giddy; print(giddy.__version__)"  # shows version 2.3.5

So, when v2.3.5 is installed via conda, the package version is visible to conda, but both pip and Python itself see version "0.0.0". If I reinstall with pip, the version number is fine everywhere. Haven't seen this kind of behavior before.

prepare giddy v2.3.4 supporting python 3.7-3.10 before meta-package release in Jan 2023

Currently, travis CI is configured to test against libpysal on pypi and github. Since mapclassify and esda are also dependencies for giddy, dual testing should also incorporate them.

giddy 2.2.2 should be released soon to incorporate the bug fix to the rank method #118

Hello,

this follows on from my earlier query. I find the process of gaining access to a new version of the package that may solve my issue convoluted. Firstly I was pointed top this web page : https://github.com/pysal/pysal/wiki/GitHub-Standard-Operating-Procedures, but after: forking pysal; installing Git; and typing 'git clone https://github.com/tra6sdc/pysal.git' I get this in the window :

_steph_000@HOMEPC MINGW64 ~
$ git clone https://github.com/tra6sdc/pysal.git
Cloning into 'pysal'...
remote: Counting objects: 21397, done.
remote: Total 21397 (delta 0), reused 0 (delta 0), pack-reused 21397
Receiving objects: 100% (21397/21397), 57.91 MiB | 5.52 MiB/s, done.
Resolving deltas: 100% (13481/13481), done.
Checking out files: 100% (888/888), done.

steph_000@HOMEPC MINGW64 ~
$_

Now what? Is there now a local version of pysal somewhere on my PC? and if so, inside Anaconda/Spyder what do I do to access this local version? I am even unclear if I need to get pysal, libpysal or giddy? Is there a simple step-by-step list of instructions to do what I want, that doesn't assume that I have been steeped in git/pysal of years?

Alternatively, is there a timescale for a release of the new version? Thanks.

This is happening in giddy and elsewhere in pysal using the sphinx build:

The data has a set of 4 classes. But on calling
giddy.markov.Spatial_Markov(rpci, qW, fixed = True, k = 4), it gives out only 2 classes.
I don't really understand why it gives its own custom number of classes.

Last time CI was successfully run was 9 months ago. After this long period of no activity, much has changed in the dependency stack and the old ci has a lot of rough edges.

We need to update the CI so new enhancements can be added to the package.

.travis.yml potentially uses an archaic version of Miniconda dating back to 2015. The newer versions are explicitly labeled Miniconda2 or Miniconda3. Within .travis.yml, this should be updated to:

wget https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh -O miniconda.sh

Links broken to the source notebooks within https://github.com/pysal/giddy/tutorial.

i.e.

This page was generated from doc/notebooks/MarkovBasedMethods.ipynb

I will push up a fix for this based on what I did in spaghetti shortly.

Just curious: is it good practice to reuse a function name as the name of a class attribute?

An example is steady_state:

• steady_state is a function in giddy and can be imported as from giddy.ergodic import steady_state
• steady_state is an attribute of class Markov in giddy and can be used as:

from giddy.ergodic import Markov
mc = Markov(y)
mc.steady_state

docs building is failing https://readthedocs.org/projects/giddy/builds/9427109/

libpysal is now main by default, causing dev testing failures. We need to update CI accordingly.