[T]he Political Instability Task Force’s Worldwide Atrocities Dataset [] records information from several international press sources about situations in which five or more civilians are deliberately killed in the context of some wider political conflict. (Jay Ulfelder)

The blog post already links to some R code. The recipe method is described in #10.

Travis CI fails on loading xlsx dependency.

@briatte Thanks for suggesting placing your QoG work together in one place with psData.

First attribution: This would be a really big contribution, so I'ld be more than happy to add you as a contributor to the repository and make you an author on the package overall. Just let me know.

Things to discuss/work on:

Unified syntax: I think to make the package really user friendly we should work on unifying the syntax across the commands that gather the different data sets. I've worked out kind of a start, but I'm not particularly wedded to anything. We should discuss this further.

Unified data object class and data handling methods: I really like what you've done with adding additional capabilities to lag/lead/graph the data. It would be great to apply this across the other sources.

Thoughts on moving forward:

If you want to merge psData and your work together I think:

1. I should first give you contributor status on the psData repo.
2. We start a qogDev branch to psData which you can patch over the code into.
3. We should start co-writing a JSS style package description paper while we unify the syntax and data handling methods across the commands. I think writing a paper while doing this will (a) help us clearly delineate the package structure and usage, (b) provide a guide to other possible contributors (who may also become co-authors), and (c) satisfy our need to appease the academic gods who are hungry for papers.

Quick question:

• Does the current version of qogdata pass CRAN check?

Just let me know what you think about this plan as a way forward.

Occurs with following command

DpiData <- DpiGet()

also there is what appears to be a typo on the main page. You refer to pdData as not yet being on CRAN

The default url in PolityGet() points to the p4v2012.sav file which does not exist anymore, and therefore it produces an error. It must point to the latest release, which currently is p4v2015.sav.

• add improvements to panel lag and shift based on recent updates in DataCombine. These are mostly error handling improvements, but also has capabilities for a number of different moving averages/'spread' for dummy variables.

I've put together the first draft of the presentation for csv,conf next week.

The slide deck is here: http://christophergandrud.github.io/psDataCSVConf/psDataPresentation.html

Feel free to make a pull request here: https://github.com/christophergandrud/psDataCSVConf

Noting this down as an idea for when the production version is out: it might be relevant to run an ECPR Summer School course with psData one day. They have courses on other software, probably teaching with the same kind of data.

Following discussions in #1 and #3, there seems to be a consensus emerging that we should focus on:

1. packages for downloading/cleaning individual data sets
2. at the same time, at least for data in country-year format, a common syntax/capabilities should be developed that makes it easy for users to download data sets with these individual packages and returns data frames that can be easily merged with one another.

As such, I think what we might want to do is

(a) Create a new text document repo that would be used to collaborate on a common psCountryData framework. (Probably starting with an .md document that would develop a framework checklist for individual country-year data packages to follow. Also, because my main professional incentive right now is to publish papers, this could be developed into a JSS style article laying out the framework and giving examples from packages that implement it. Any interested person could of course co-author.).

(a) Create a new package called psCountryData that would contain core functions shared by the individual data Getter/Variable Builder packages. For example, psData currently includes a CountryID function. This is a modified version of countrycode that is handy for creating merge ready country identifiers. It looks like there is some good stuff in @briatte's QoG package that could go in there too.

(c) Break up psData into individual Getter and Variable Builder packages that implement this framework. Similarly the two QoG packages could (depending on the authors' preferences), implement the unified syntax.

Any thoughts?

Hello, I am not sure if this package is still usable?
I tried to run the code, yet, R shows warning and said there is no "WinsetCreator".

Feel free to add data sets and variables that would be useful to include in psData. Code contributions are also always very helpful

• Quality of Governance Data
• Henisz's Political Constraints Index (Mark Hallerberg)--sign in required
• American National Elections Studies sign in required
• General Social Survey
• World Values Survey
• European Social Survey sign in required
• Comparative Study of Electoral Systems
• Correlates of War (COW)
  • Related data to COW
• Comparative Agendas Project
• Quality of Government
• Pew Research Center sign in required
• Congressional Bills Project
• Global Database of Events, Language, and Tone (GDELT)
  • Possible import from GDELTtools
• Dynamics of Collective Action (DoCA)

In terms of repository structure, I think it would be beneficial to split each data source into separate files. The idea would be to create a standarized "recipe" format that would include all info about the dataset (e.g. where to download, bibtex cite, name of cleaning script, date updated), and then a cleaning script that does all the magic we need.

I use something like that locally, where I have a YAML file that specifies all the info and then an accompanying python script that I use for cleaning.

This makes user contributions very easy. They just cut and paste another "recipe" and include an R script that does the cleaning. The only thing psData has to do is provide a proper API to parse the recipe, download the data, and activate the cleaning script.

Think of something like the homebrew install for mac and its library of "formulas":

https://github.com/Homebrew/homebrew/tree/master/Library/Formula

Following the consensus in #5 a number of major changes will be made to psData for the version 1 release. These include:

• Separating the present getter and variable builder functions into their own packages.
• Implement the framework laid out in the psData guidelines. Including:
• get_data a function for downloading panel data sets by calling getter and variable builder functions from associated packages.
• panel_set a function for cleaning the downloaded data into a political science panel-series (psData) object.
• panel_merge a function for merging psdata objects.

Allow users to pass a flag that will cache the data locally after it is downloaded. Goals:

1- Reproducibility. Even if the online version disappears, the user will still have a local copy
2- Quicker download on second request.

Should be accompanied by a standalone cleaning script if applicable.

This isn't really a psData topic, but I was wondering if anyone is interested in putting together a proposal for the CPS special issue on transparency in the social sciences.

I'm thinking that maybe a project on how social scientists use version control and (primarily) GitHub to make their research more transparent.

Anyone interested?

Thanks for a very useful package! One tiny bug. In the Polity IV Data import, Russia and USSR both have a 1922 year. This might not normally be a problem but they have the same iso2c code, so it creates a duplicate. Only 1922 has this problem.

Hi,

Here's a draft S4 class that creates a subclass of data.frame to implement functions for panel data and time series manipulation, or for any dyadic data.

Contents

• CountryID by @christophergandrud (wraps around countrycode by @vincentarelbundock, with edits)
• MoveFront from DataCombine by @christophergandrud
• class-HomogList and subclass_homog_list from DataFrameConstr by @jrnold
• class-psData (hacked from DataFrameConstr by @jrnold)
• constrained_data_frame from DataFrameConstr by @jrnold
• debt.rda (demo Reinhart and Rogoff data)
• get_data and a bunch of get (download) functions for e.g. Polity 4 and QOG data
• panel with functions by @zmjones
• utils (with snippets by more authors)

I'm sure many more people have written useful code for working with panel data, including for visualization purposes—the package should probably also look in that direction, and the draft includes a Reinhart and Rogoff plot.

Output

# install
devtools::install_github("briatte/psData")
library(psData)

# example
data(debt)

The object is of class data.frame:

> head(debt)
    Country Year    growth     ratio
1 Australia 1946 -3.557951 190.41908
2 Australia 1947  2.459475 177.32137
3 Australia 1948  6.437534 148.92981
4 Australia 1949  6.611994 125.82870
5 Australia 1950  6.920201 109.80940
6 Australia 1951  4.272612  87.09448

Now add panel design variables to quickly create a psData panel object:

> as.panel(debt, "Country", "Year")
 Panel data frame [ 1171 rows x  4 columns, 20 Country x 64 Year ]

    Country Year    growth     ratio
1 Australia 1946 -3.557951 190.41908
2 Australia 1947  2.459475 177.32137
3 Australia 1948  6.437534 148.92981
4 Australia 1949  6.611994 125.82870
5 Australia 1950  6.920201 109.80940
6 Australia 1951  4.272612  87.09448

The object is now a data.frame to which the package can pass functions similar to the xt and ts commands in Stata. Country codes and basic date formats are automatically detected and usable for safe merges between multiple panel data.

When using psData, the user would frequently see messages such as:

37 duplicated values were created when standardising the country ID with iso2c.
59 observations dropped based on missing values of the standardised ID variable.

I think it would be a good idea to add a verbose option that allows user to know exactly which id has been dropped / duplicated due to transforming which type of country ID to another. It would add a lot of peace of mind. In my experience, I always go back to check what has been dropped manually.

If @christophergandrud think this is a good idea, I would be happy to contribute a pull request.

Do any of you have experience with dat?

I've been trying to think about how this might relate to psData. Overall, they aim to address similar issues, with psData being a bit more focused on one type of data and usage from R.

I wonder if it is worth exploring using dat as a backend, or some other integration.

Sorry everyone, I've been kind of overwhelmed with other projects the past couple of weeks.

I was thinking that to get this thing started up again it might be good for interested parties to think of a few days to a week in the summer that they would be available to work together on this close to full time.

Any interest? Preferred times?

Would it be appropriate to file whatever version of psData is ready next year for the Google Summer of Code 2015?

Any thoughts on integrating testdat into psData?

The dataset will be added to the master branch version of psData, not the overhauled version.

I have code ready to add this.

Executing the following code

grep(pattern = "Northern America", x = psData::countrycode_data$region, value = T)

returns repeated values of "Northern America " with a trailing space.