simongreenhill / abvdget Goto Github PK

Hi Simmo,

Perhaps it's useful if Siva and I file some questions we had about ABVDget here rather than via emails? Let us know if that's not alright.

We've moved our "playpen" (as Harald calls it) out of Glottobank, it's here now. I've checked with Harald about what can be shared and how, all the GB and NTS data is handled appropriately.

What we want to do is get distance measurements of austronesian languages.

For starters, we've written two scripts: one to download all the json files (get_lgs.py) and one to mash them together into a wide csv-file just the way we wanted them (Make_ABVD_wide.R). We encountered some issues, most of them were solved and I learned a lot. Hurray!

Should you have time, we have these questions:

1. How come Futuna, West and Proto-Philippine are in ABVD when they have no entries?

2. Following your sample code here, we tried running this bit, but we couldn't get strict and uniques to work.

db = ABVDatabase(
strict=True, # Strict cognate coding (i.e. ignore dubious items like 1?)
uniques=True, # encode non-cognate items as unique states.
)

We commented those lines out and dealt with the issue in our own way. We want to be inclusive so we’re testing lumping cognacy classes with question marks together with those without, we'll see how it goes. The r-script has a way of dealing with turning non-cognate states into unique states, should be exactly the same result as what you were doing.

3. We weren’t sure what to do when there is more than one cognate class for an item. We set it up so that we only get the first of the cognate classes listed. Is this reasonable?

4. There are cases where, for one word (meaning/concept), in one language there is more than one entry. We weren’t sure how to deal with this when making a distance matrix. We picked the form with the lowest cognacy number for now.

We imagine that there's a better way of dealing with cases of multiple values/entries like (3) and (4), perhaps we'll figure it out on our own - but if there's a shortcut available that'd be appreciated too.